University of Illinois Spring 2009
ECE 313: Problem Set 13
Joint Gaussian random variables, MMSE, limit theorems
Due: Wednesday May 6 at 4 p.m.
Reading: Ross Chapter 8
1. [Jointly Distributed Random Variables]
Let the random variables Xand Ybe such that E[X] = 1, E[Y] = 4, V ar(X) = 4, V ar(Y) = 9, and
ρ= 0.1. Let W= 3X+Y+ 2.
(a) Find E[W] and V ar(W).
(b) If Xand Yare jointly Gaussian random variables, what is P{W > 0}?
2. [Coordinate Transformations for Jointly Distributed Gaussian Random Variables]
Let Xand Ybe jointly Gaussian zero-mean random variables, with variances σ2
Xand σ2
Y(respectively)
and with correlation coefficient ρ. Find the angle θsuch that the random variables Z=Xcos θ+Ysin θ
and W=−Xsin θ+Ycos θare independent Gaussian random variables. Note that this coordinate
transformation corresponds to a rotation of the axes by θ. Thus, in this problem you will show how to
transform two dependent Gaussian random variables to independent Gaussian random variables by a
simple rotation.
3. [Linear Minimum Mean Square Error Estimation]
Suppose that the value of Yis to be estimated in terms of the uncorrelated random variables X1and
X2by the linear predictor g(X1, X2) = a+bX1+cX2. Determine the values for a,band cthat
minimize E[(Y−(a+bX1+cX2))2]. Express your answer in terms of the means and variances of X1
and X2and the covariances Cov(X , Y1) and Cov(X, Y2).
4. [Minimum Mean Square Error Estimation]
This problem is a continuation of Problem 8 from Problem Set 11. The random point (X, Y ) is
uniformly distributed on the shaded region shown.
u
v
1
1
1/2
1/2
(a) Given that the value of Xis u, the (conditional) minimum-mean-square-error estimate of Y
is E[Y|X=u], the conditional mean of Y, and the (conditional minimum) mean-square error
achieved is V AR(Y|X=u).
Use your answers to Problem 8 from Problem Set 11 to sketch graphs of E[Y|X=u] and
V AR(Y|X=u) as functions of ufor 0 < u < 1.
(b) Since V AR(Y|X=u) depends on the value of X, it is a function of X. Use LOTUS to compute
E[V AR(Y|X=u)], the expected value of this function. This is the (unconditional) mean-square
error that we achieve when we estimate Yas E[Y|X=u], and no other estimate can have smaller
mean-square error than this.
