Download Stat 411 Homework 7: Minimum Variance Unbiased Estimation in Exponential Families and more Assignments Statistics in PDF only on Docsity!
Stat 411 – Homework 07 Due: Wednesday 03/
Undergraduates may solve the “Graduate only” problem(s) for possible extra credit.
- Let X 1 ,... , Xn be iid N(θ, 1). Find the MVUE of η = θ^2.
- Problem 7.5.3 on page 392.
- Problem 7.5.6 on page 393.
- Problem 7.5.13 on page 394.
- (Graduate only) Let X 1 ,... , Xn be iid from a distribution with PDF/PMF fθ(x) in the (regular) exponential family, i.e.,
fθ(x) = exp
p(θ)K(x) + S(x) + q(θ)
Define m(θ) = Eθ[K(X 1 )] and the estimator m̂ (θ) = (^1) n
∑n i=1 K(Xi).
(a) Without using part (b) below, argue that m̂ (θ) is the MVUE of m(θ).
(b) Show that the variance of m̂ (θ) is equal to the Cramer–Rao lower bound. (Hints: You’ll need to calculate the Fisher information I(θ) for fθ(x) define above. Also, remember that m(θ) is a function of the parameter θ, so the Cramer–Rao lower bound is not simply [nI(θ)]−^1. Finally, the results in The- orem 7.5.1 should be helpful.)
Stat 411 – Homework 07 Solutions
- The distribution N(θ, 1) is a regular one-parameter exponential family problem with K(x) = x. Therefore, T =
∑n i=1 Xi^ is a complete sufficient statistic for^ θ^ and, consequently, the MVUE of θ is X = T /n. It is easy to check that ˆη = X 2 − 1 /n is an unbiased estimator of η = θ^2. Since it’s a function of the complete sufficient statistic, by the Lehmann–Scheffe theorem, it must be the MVUE.
- Problem 7.5.3 from the text.
(a) The beta distribution with PDF fθ(x) = θxθ−^1 , with x ∈ (0, 1) and θ > 0, is a regular exponential family. That is,
fθ(x) = exp{log θ + (θ − 1) log x}.
Since K(x) = log x here, a complete sufficient statistic, based on X 1 ,... , Xn iid ∼ fθ(x), is T 0 =
∑n i=1 log^ Xi. So too is^ T^ = exp{T^0 /n}^ = (X^1 X^2 · · ·^ Xn) 1 /n, the geometric mean, since the function t 7 → et/n^ is one-to-one. (b) The log-likelihood function `(θ) =
∑n i=1 log^ fθ(Xi) is
`(θ) = n log θ + (θ − 1)
∑^ n
i=
log Xi.
Differentiating and setting equal to zero gives the likelihood equation:
n θ
∑^ n
i=
log Xi = 0.
Therefore, the MLE is θˆ = −n/
∑n i=1 log^ Xi^ =^ −^1 /^ log^ T^ , a function of the geometric mean T from part (a).
- Problem 7.5.6 from the text. Let Y = K(X). Then the moment-generating function MY (t) of Y is given by
MY (t) = Eθ(etK(X)) =
etK(x)eθK(x)+S(x)+q(θ)^ dx
eq(θ)−q(θ+t)e(θ+t)K(x)+S(x)+q(θ+t)^ dx
= eq(θ)−q(θ+t)
e(θ+t)K(x)+S(x)+q(θ+t)^ dx
= eq(θ)−q(θ+t),
where the last inequality is because the integrand is the exponential family PDF with parameter “θ + t” instead of “θ.”
The next-to-last line follows from a change of variable: u = t − 1. Therefore, T /(n + T − 1) is an unbiased estimator of θ; moreover, since it’s a function of the complete sufficient statistic, it must be the MVUE.
- (Graduate only) To keep notation simple, write ˆm = m̂ (θ).
(a) The estimator ˆm = (^) n^1
∑n i=1 K(Xi) is unbiased for^ m(θ) =^ Eθ[K(X^1 )] by defini- tion. Since it’s a function of the complete sufficient statistic T =
∑n i=1 K(Xi) it must be the MVUE according to Lehmann–Scheffe. (b) Here I prove that ˆm is the MVUE of m(θ) by showing that its variance is equal to the Cramer–Rao lower bound.
- Observe that Vθ( ˆm) = Vθ(T /n) = Vθ(T )/n^2. By Theorem 7.5.1(3),
Vθ(T ) = n{p′′(θ)q′(θ) − p′(θ)q′′(θ)}/p′(θ)^3 ,
which implies Vθ( ˆm) =
p′′(θ)q′(θ) − p′(θ)q′′(θ) np′(θ)^3
- For the Cramer–Rao lower bound, I first need the Fisher information. The second derivative of log fθ(x) is
(∂^2 /∂θ^2 ) log fθ(x) = p′′(θ)K(x) + q′′(θ).
Therefore,
I(θ) = −Eθ
[ ∂ 2
∂θ^2
log fθ(X 1 )
]
= −p′′(θ)Eθ[K(X 1 )] − q′′(θ) =
p′′(θ)q′(θ) − p′(θ)q′′(θ) p′(θ)
where the last equality uses the result in Theorem 7.5.1(2) to express Eθ[K(X 1 )] in terms of p and q. Since the estimand is m(θ), not just θ, the Cramer–Rao lower bound for estimating m(θ) involves the derivative of m(θ). Since m(θ) = Eθ[K(X 1 )] = −q′(θ)/p′(θ) from Theorem 7.5.1(2), we get m′(θ) =
p′′(θ)q′(θ) − p′(θ)q′′(θ) p′(θ)^2
Therefore, the Cramer–Rao lower bound is
LB =
m′(θ)^2 nI(θ)
p′′(θ)q′(θ) − p′(θ)q′′(θ) np′(θ)^3
Since the Cramer–Rao lower bound is the same as Vθ( ˆm), there’s no other estimator with smaller variance, i.e., it’s the MVUE. This result in this exercise showed that the Cramer–Rao lower bound is at- tained in an exponential family problem. There is a partial converse to this result which says, roughly, if the Cramer–Rao lower bound is attained, then the problem must be an exponential family type—very interesting! See Lehmann and Casella (1998, p. 121) for the precise statement of this result.
