ECE 534 RANDOM PROCESSES FALL 2004
PROBLEM SET 3 Due Wednesday, October 6
Random Vectors and Minimum Mean Squared Error Estimation
Assigned Reading: Chapter 3 and the section on matrices in the Appendix, in the course notes.
Problems to be handed in:
1. Linear approximation of the cosine function over [0, π]
Let Θ be uniformly distributed on the interval [0, π] (yes, [0, π], not [0,2π]). Suppose Y= cos(Θ)
is to be estimated by an estimator of the form a+bΘ. What numerical values of aand bminimize
the mean square error?
2. Projections onto nested linear subspaces
(a) Use the Orthogonality Principle to prove the following statement: Suppose V0and V1are
two closed linear spaces of second order random variables, such that V0⊃ V1, and suppose X
is a random variable with finite second moment. Let Z∗
ibe the random variable in Viwith the
minimum mean square distance from X. Then Z∗
1is the variable in V1with the minimum mean
square distance from Z∗
0. (b) Suppose that X, Y1,and Y2are random variables with finite second
moments. For each of the following three statements, identify the choice of subspace V0and V1
such that the statement follows from part (a):
(i) b
E[X|Y1] = b
E[b
E[X|Y1, Y2]|Y1].
(ii) E[X|Y1] = E[E[X|Y1, Y2]|Y1]. (Sometimes called the “tower property.”)
(iii) E[X] = E[b
E[X|Y1]]. (Think of the expectation of a random variable as the constant closest to
the random variable, in the m.s. sense. )
3. Diagonalizing a two-dimensional Gaussian distribution
Let X= X1
X2!be a mean zero Gaussian random vector with correlation matrix 1ρ
ρ1!,
where |ρ|<1. Find an orthonormal 2 by 2 matrix Usuch that X=UY for a Gaussian vector
Y= Y1
Y2!such that Y1is independent of Y2. Also, find the variances of Y1and Y2.
Note: The following identity might be useful for some of the problems that follow. If A, B, C,
and Dare jointly Gaussian and mean zero, then E[ABCD] = E[AB]E[CD] + E[AC]E[B D] +
E[AD]E[BC ]. This implies that E[A4]=3E[A2]2, Var(A2)=2E[A2], and Cov(A2, B2) =
2Cov(A, B)2.Also, E[A2B] = 0.
4. A quadratic estimator
Suppose Yhas the N(0,1) distribution and that X=|Y|. Find the estimator for Xof the form
b
X=a+bY +cY 2which minimizes the mean square error. (You can use the following numerical
values: E[|Y|] = 0.8, E[Y4] = 3, E[|Y|Y2] = 1.6.)
(a) Use the orthogonality principle to derive equations for a, b, and c.
(b) Find the estimator b
X.
(c) Find the resulting minimum mean square error.
1
