Download Homework 2 with Solutions | Statistical Methodology | STAT 613 and more Assignments Statistics in PDF only on Docsity!
Stat613: Intermediate Theory of Statistics
Homework 2 (due 9/14/07)
Instructor: Jianhua Huang Students: Anna Dao
TA: Seokho Lee
Do Exercises 4.17, 4.28 from the text and the following additional problem.
Problem A. (All students are required to latex the solution to this problem. Try to be concise in your
presentation – clear and succinct.)
Multivariate normal distribution
Let Y 1
,... , Y
n
be a random sample from the N p
(μ, Ω) distribution. That is, Y j
= (Y
1 j
,... , Y
pj
T has the
multivariate normal distribution with mean vector μ = (μ 1
,... , μ p
T and p×p nonsingular covariance matrix
Ω, whose (r, s) element is ω rs
. Denote the sample mean and sample covariance matrix as
Y = n
− 1
n ∑
j=
Yj , S = (n − 1)
− 1
n ∑
j=
(Yj −
Y )(Yj −
Y )
T .
a. Show that the MLE of μ and Ω are given by ˆμ =
Y and
Ω = n
− 1 (n − 1)S. [Hint: If you are not familiar
with matrix differentiation, it might be useful to do a good search on matrix cookbook for useful facts.] You
can derive the results by solving the likelihood equations. But for students who want rigor, can you prove
that the solution of the likelihood equation indeed maximizes the likelihood?
Let Y 1
,... , Y
n
be from N p
(μ, Ω). Then the likelihood of Y 1
,... , Y
n
is
L(Y 1 ,... , Yn|μ, Ω) =
(2π)
np/ 2 |Ω|
n/ 2
exp
n ∑
i=
(Yi − μ)
T Ω
− 1 (Yi − μ)
Now take the natural log from the likelihood above,
l(Y 1
,... , Y
n
|μ, Ω) = constant −
n
ln |Ω| −
n ∑
i=
(Y
i
− μ)
T Ω
− 1 (Y i
− μ)
The MLE of μ and Ω can be obtained by setting ∂l/∂μ = 0 p
and ∂l/∂Ω = 0 p×p
. Using some results from a
matrix cookbook:
∂l
∂μ
− 1 (Y i
− μ)(−1) =
− 1 (Y i
− μ)
⇒ ̂μM LE = Y
and
∂l
n
− 1 −
− 1 (Yi − μ)(Yi − μ)
T Ω
− 1 (−1)
n
− 1
− 1 (Yi − μ)(Yi − μ)
T Ω
− 1
As intended above, set ∂l/∂Ω = 0p×p:
(^0) p×p = −
n
− 1
− 1
(Yi − μ)(Yi − μ)
T
Ω
− 1
⇔ nΩ
− 1
− 1 (Yi − μ)(Yi − μ)
T Ω
− 1
⇔ nIp×p =
− 1 (Yi − μ)(Yi − μ)
T
M LE
n
(Y
i
− μ̂)(Y i
− μ̂)
T
n
(Y
i
− Y )(Y
i
− Y )
T
b. Show that apart from constants, the value of the maximized log likelihood is −(1/2)n log |
Ω|, and deduce
that the likelihood ratio statistic for comparison of the full model and a submodel obtained by constraining
elements of Ω may be written as n log |
Ω 0 |, where
Ω 0 is the MLE of Ω under a specified constraint. (Part
c shows an example of such constraint.)
Due to the nature of ̂μM LE and
ΩM LE , the maximized log likelihood is attained at μ̂M LE and
ΩM LE.
l(Y 1
,... , Y
n
|μ, Ω) = constant −
n
ln |
n ∑
i=
(Y
i
− Y )
T
n
n ∑
j=
(Y
j
− Y )(Y
j
− Y )
T
− 1
(Y
i
− Y )
= constant −
n
ln |
T race
n ∑
i=
(Yi − Y )
T
n
n ∑
j=
(Yj − Y )(Yj − Y )
T
− 1
(Yi − Y )
= constant −
n
ln |
T race
n ∑
i=
n
n ∑
j=
(Yj − Y )(Yj − Y )
T
− 1
(Yi − Y )(Yi − Y )
T
= constant −
n
ln |
T race {nI p×p
= constant −
n
ln |
Now we have to test the significance of a submodel:
- H 0 : θ = (μ, Ω 0 ) ∈ Θ 0
• H
a
: θ = (μ, Ω) not in Θ 0
where Θ 0
Then LRT would be:
T = 2 ln
maxθ∈Θ L(θ)
max θ∈Θ 0 L(θ)
Figure 1: Histograms of LRTs
Histogram of statistics
statistics
Density
0 5 10 15 20
Histogram of statistics
statistics
Density
0 5 10 15
Histogram of statistics
statistics
Density
0 5 10 15 20
Histogram of statistics
statistics
Density
0 50 100 150 200
For each picture, 1000 sample were taken. Each sample is of size 50. p = 3 also remains the same in
simulation. The upper left figure is multivariate-t with 300 degree of freedom. The upper right figure is
multivariate-t with 50 degree of freedom. The lower left figure is multivariate-t with 10 degree of freedom.
The lower right figure is multivariate-t with 3 degree of freedom.
