EECS 501 SOLUTIONS TO PROBLEM SET #7 Fall 2001
1. Let fxi(X) = 1
A,0≤X≤A; 0 otherwise (Ais unknown). Let y=max[x1.. . xn].
Fy(Y) = P r[y≤Y] = P r[max[x1. . . xn]≤Y] = Qn
i=1 P r[xi≤Y] = ( Y
A)n, Y ≤A.
fy(Y) = d
dY Fy(Y) = nY n−1/An,0≤Y≤A; 0 otherwise (previous result scaled).
1a. E[ˆ
A] = E[y] = RA
0(Y nY n−1/An)dY =n
n+1 A6=A→ˆ
Ais a biased estimator.
LIM
n→∞ E[ˆ
A] = LIM
n→∞
n
n+1 A=A→ˆ
Ais an asymptotically unbiased estimator.
1b. P r[|ˆ
A−A|> ²] = P r[ˆ
A < A −²] = RA−²
0(nY n−1/An)dY = (1 −²
A)n.
LIM
n→∞ P r[|ˆ
A−A|> ²] = LIM
n→∞ (1 −²
A)n= 0 →ˆ
Ais a (weakly) consistent estimator.
2. fx|λ(X|λ) = Q5
i=1 λe−λXi=λ5e−λP5
i=1 Xifor Xi≥0.
0 = ∂
∂λ log fx|λ(X|λ) = ∂
∂λ [5 log λ−λP5
i=1 Xi] = 5
λ−P5
i=1 Xi→ˆ
λMLE = 5/P5
i=1 Xi.
Note that P5
i=1 Xiis a sufficient statistic for ˆ
λMLE : Don’t need individual Xi.
3. Given: fr|λ(R|λ) = λe−λR, R ≥0. fλ(λ) = 1
Te−λ/T , λ ≥0.
3a. MLE: 0 = ∂
∂λ log fr|λ(R|λ) = ∂
∂λ [log λ−λR] = 1
λ−R→ˆ
λMLE (R) = 1
R. Note units.
3b. MAP: 0 = ∂
∂λ [log fr|λ(R|λ) +log fλ(λ)] = ∂
∂λ [log λ−λR−log T−λ/T ] = 1
λ−R−1
T→
ˆ
λMAP (R) = 1/(R+ 1/T ) = T /(RT + 1)=1/(a priori+a posteriori).
(T→0) →λ= 0 →ˆ
λMAP (R)→0 (a priori information about λpredominates).
(T→ ∞)→fλ(λ) uniform→ˆ
λMAP (R)→ˆ
λMLE (R) (a posteriori info predominates).
3c. MSE: ˆ
λLSE (R) = Rλfr|λ(R|λ)fλ(λ)dλ
Rfr|λ(R|λ)fλ(λ)dλ =Rλ2
Te−λ(R+1/T )dλ
Rλ
Te−λ(R+1/T )dλ = 2/(R+ 1/T ) = 2ˆ
λMAP .
NOTE: fλ|R(λ|R) = (R+ 1/T )2λe−λ(R+1/T ), λ > 0 = 2nd -order Erlang pdf.
Recognizing this, read off ˆ
λLSE (R) = E[λ|r=R] = 2/(R+ 1/T ) without integrals.
4. px1...x100|p(X1. . . X100 |P) = PK(1 −P)100−Kwhere K=P100
i=1 Xi=#heads.
4a. ∂
∂P [Klog P+ (100 −K) log(1 −P)] = K
P−100−K
1−P→ˆpMLE =K
100 .
4b. ˆpLLSE (K) = E[p] + λkp
σ2
k
(K−E[k]). Need to plug into formula:
E[p] = 1
2. E[k] = Ep[E[k|p]] = E[100p] = 100( 1
2) = 50.
E[kp] = EpE[kp|p] = E[100p2] = 100 R1
0P2dP = 33.33.
λkp =E[kp]−E[k]E[p] = 33.33 −50( 1
2) = 8.33.
σ2
k=Ep[σ2
k|p] + V arp[E[k|p]] = E[100p(1 −p)] + σ2
100p=100
2−100
3+ 104(1−0)2
12 = 850.
ˆpLLSE (K) = 1
2+8.33
850 (K−50) = 0.009804(K+ 1) = K+1
102 (look familiar?).
NOTE: Can also compute ˆpLSE (K) = K+1
102 and note this is a linear estimator!