Math 6010, Fall 2004: Homework
Homework 4
#1, page 136: As in the hint let Ii=1 if Eiis incorrect; else, let Ii=0. Then
∑iIidenotes the number of incorrect statements, and E[∑iIi] = ∑iE[Ii] =
∑iP(Ei) = ∑iαiis the corresponding expectation.
#2, page 136: Fix k>1, and define
f(α) = 1−α
kk−(1−α)0≤α≤1.
Evidently,
f0(α) = 1−1−α
kk−1
>0,
for all α>0. This proves that the minimum of foccurs uniquely at α=0;
i.e., f(α)>f(0), which is the desired result.
#4, page 136: A solution will be posted soon.
#5, page 136: Every time we add a new variable we increase the variance
(§5.4). Here, however, is a direct argument: Suppose we have the new
model,
G:Y=Xβ+Zγ+ε=Wδ+ε,
where W= (X,Z)columnwise, and δ= (β0,γ0)0. The least-squares pre-
dictor, under G, is
ˆ
δG= (W0W)−1W0Y.
Thus, the new predictor at (x0
0,γ0
0)0is:
ˆ
Y0G=x0
0,γ0
00ˆ
δG.
By Theorem 3.6(iv) (p. 55),
Varˆ
δG=(X0X)−1+LM L0−L M0
−M L0M,
where L= (X0X)−1X0Z,M= (Z0R Z)−1, and R=In−W(W0W)−1W0.
Therefore,
Varˆ
Y0G=x0
0,γ0
00Varˆ
δGx0
0,γ0
0
=σ2x0
0X0X−1x0+x0
0LM L0x0−γ0
0M L0x0−x0
0LM 0γ0+γ00Mγ0
=σ2x0
0X0X−1x0+x0
0LM L0x0−2γ0
0M L0x0+γ00Mγ0
=σ2x0
0X0X−1x0+L0x0−γ00ML0x0−γ0
≥σ2x0
0X0X−1x0,
as long as Mis positive definite. It remains to prove that Mis p.d. We can
note that M−1is positive definite, because w0M w =kRZw k2for all w.
(This uses R0R=R.) Therefore, M−1has all strictly positive eigenvalues,
1
