Solutions to Exam 1
Problem 1 (12 points) Let A1, A2, . . . be a sequence of independent random variables, with
P[Ai= 1] = P[Ai=1
2] = 1
2for all i. Let Bk=A1· · · Ak.
(a) Does limk→∞ Bkexist in the m.s. sense? Justify your anwswer.
(b) Does limk→∞ Bkexist in the a.s. sense? Justify your anwswer.
(c) Let Sn=B1+. . .+Bn. You can use without proof (time is short!) the fact that limm,n→∞ E[SmSn] =
35
3,which implies that limn→∞ Snexists in the m.s. sense. Find the mean and variance of the limit
random variable.
(d) Does limn→∞ a.s. Snexist? Justify your anwswer.
(a) E[(Bk−0)2] = E[A2
1]k= ( 5
8)k→0 as k→ ∞. Thus, limk→∞ m.s. Bk= 0.
(b) Each sample path of the sequence Bkis monotone nonincreasing and bounded below by zero, and is hence conver-
gent. Thus, limk→∞ a.s. Bkexists. (The limit has to b e the same as the m.s. limit, so Bkconverges to zero almost surely.)
(c) Mean square convergence implies convergence of the mean. Thus, the mean of the limit is limn→∞ E[Sn] = limn→∞ Pn
k=1 E[Bk] =
P∞
k=1(3
4)k= 3. By the property given in the problem statement, the second moment of the limit is 35
3, so the variance of
the limit is 35
3−32=8
3.
(d) Each sample path of the sequence Snis monotone nondecreasing and is hence convergent. Thus, limk→∞ a.s. Bk
exists. (The limit has to be the same as the m.s. limit.)
Here is a proof of the claim given in part (c) of the problem statement, although the proof was not asked for on the exam.
If j≤k, then E[BjBk] = E[A2
1···A2
jAj+1 ···Ak] = ( 5
8)j(3
4)k−j, and a similar expression holds for j≥k. Therefore,
E[SnSm] = E[
n
X
j=1
Bj
n
X
k=1
Bk] =
n
X
j=1
n
X
k=1
E[BjBk]
→
∞
X
j=1
∞
X
k=1
E[BjBk]
= 2
∞
X
j=1
∞
X
k=j+1 „5
8«j„3
4«k−j
+
∞
X
j=1 „5
8«j
= 2
∞
X
j=1
∞
X
l=1 „5
8«j„3
4«l
+
∞
X
j=1 „5
8«j
= (
∞
X
j=1 „5
8«j
)(2
∞
X
l=1 „3
4«l
+ 1)
=5
3(2 ·3 + 1) = 35
3
Problem 2 (12 points) Let X, Y , and Zbe random variables with finite second moments and suppose
Xis to be estimated. For each of the following, if true, give a brief explanation. If false, give a counter
example.
(a) TRUE or FALSE: E[|X−E[X|Y]|2]≤E[|X−
b
E[X|Y, Y 2]|2].
(b) TRUE or FALSE: E[|X−E[X|Y]|2] = E[|X−
b
E[X|Y, Y 2]|2] if Xand Yare jointly Gaussian.
(c) TRUE or FALSE? E[|X−E[E[X|Z]|Y]|2]≤E[|X−E[X|Y]|2].
(d) TRUE or FALSE? If E[|X−E[X|Y]|2] = Var(X) then Xand Yare independent.
(a) TRUE. The estimator E[X|Y] yields a smaller MSE than any other function of Y, including b
E[X|Y, Y 2].
(b) TRUE. Equality holds because the unconstrained estimator with the smallest mean squre error, E[X|Y], is linear,
and the MSE for b
E[X|Y, Y 2] is less than or equal to the MSE of any linear estimator.
(c) FALSE. For example if Xis a nonconstant random variable, Y=X, and Z≡0, then, on one hand, E[X|Z] =
E[X], so E[E[X|Z]|Y] = E[X], and thus E[|X−E[E[X|Z]|Y]|2] = Var(X).On the other hand, E[X|Y] = Xso that
E[|X−E[X|Y]|2] = 0.
(d) FALSE. For example, suppose X=Y W , where Wis a random variable independent of Ywith mean zero and variance
one. Then given Y, the conditional distribution of Xhas mean zero and variance Y2. In particular, E[X|Y] = 0, so that
E[|X−E[X|Y]|2] = Var(X), but Xand Yare not independent.
1
