Download Problem Set 3 with Solutions - Random Processes | ECE 534 and more Assignments Electrical and Electronics Engineering in PDF only on Docsity!
ECE 534 RANDOM PROCESSES SPRING 2008
SOLUTIONS TO PROBLEM SET 3
3.1. Rotation of a joint normal distribution yielding independence
(a) |Cov(X)| = 1 and Cov(X)−^1 =
so
fX (x) =
2 π
exp
x 1 − 10 x 2 − 5
)T (
x 1 − 10 x 2 − 5
2 π
exp
{(x 1 − 10)^2 − 2(x 1 − 10)(x 2 − 5) + 2(x 2 − 5)^2 }
(b) det
2 − λ 1 1 1 − λ
= λ^2 − 3 λ + 1, so the eigenvalues are 3 ±
√ 5 2.^ The eigenvectors satisfy ( 2 1 1 1
v 1 v 2
√ 5 2
v 1 v 2
or v 2 =
− 1 ±√ 5 2
v 1. Unit length eigenvectors are thus given
by
√ 2 √ 5 ∓
√ 5
− 1 ±√ 5 2
and U can be taken to be the matrix with columns given by the two
eigenvectors:
U =
2 5 −
√ 5
2 5+
√ √ 5 2 5+
√ 5 −
2 5 −
√ 5
The other possible choices for U can be obtained by swapping the two columns or multiplying either
column by −1. Note that U has the form
cos(θ) − sin(θ) sin(θ) cos(θ)
3.3. Calculation of some minimum mean square error estimators (a) Note that Cov(X, Y ) = Var(X) and Var(Y ) = Var(X) + Var(N ). By the formulas for linear MMSE estimation,
Ê [X|Y ] = E[X] + Cov(X,Y^ ) Var(Y ) (Y^ −^ E[Y^ ]) =^
1 λ +^
1 /λ^2 1 /λ^2 +σ^2 (Y^ −^
1 λ ) =^
1 λ +^
1 1+λ^2 σ^2 (Y^ −^
1 λ ) =^
Y +λσ^2 1+λ^2 σ^2
E[e^2 ] = Var(X) − Cov(X, Y )^2 /Var(Y ) = (^) λ^12 − (1/λ^4 )/( (^) λ^12 + σ^2 ) = σ
2 1+λ^2 σ^2 (b) No. Observe that X is nonnegative, but the estimator Ê [X|Y ] can be negative. An estimator with smaller MSE is X̂ = max{ 0 , Ê [X|Y ]}, because (X − X̂ )^2 ≤ (X − Ê [X|Y ])^2 with probability one, and the inequality is strict whenever Ê [X|Y ] < 0.
3.5. Conditional probabilities with joint Gaussians I (a) Given Y , X is N (ρY, 1 − ρ^2 ). Thus, P [X ≤ 1 |Y ] = P [ √X− 1 −ρYρ 2 ≤ √^1 − 1 −ρYρ 2 | Y ] = Φ( √^1 − 1 −ρYρ 2 ).
(b) Given Y = y, X is N (ρy, 1 − ρ^2 ). Therefore, given Y = y, X − y is N ((ρ − 1)y, 1 − ρ^2 ). Thus, E[(X − Y )^2 |Y = y] = E[(X − y)^2 |Y = y] = (ρ − 1)^2 y^2 + 1 − ρ^2.
3.7. An estimation error bound (a) There’s not enough information to compute E[e^2 ]. However, the LMMSE estimator Ê [X|Y ] has error
E[(X − Eˆ[X|Y ])^2 ] = Var(X) − Cov(X, Y )^2 /Var(Y ) = 8 −
This is an upper bound on E[(X − E[X|Y ])^2 ], since the MMSE estimate has MSE less than or equal to that of the best linear estimator. (b) It is sufficient (and necessary) that E[X|Y ] be linear, for then and only then is it true that E[X|Y ] = Eˆ[X|Y ], and that the error for the MMSE estimator E[X|Y ] achieves its worst case value: E[e^2 ] = 7/2. There is more than one distribution satisfying the given moment conditions such that E[X|Y ] = Eˆ[X|Y ]. One possibility is to take X and Y to be jointly Gaussian. For
another possibility, let
X
Y
be discrete with the joint probability mass function given in the
following table: X = 2 +
8 X = 2 −
Y = −2 +
2 14 + α 14 − α Y = − 2 −
2 14 − α 14 + α
This joint distribution has the given mean vector and covariance matrix if α = 163. Since Y has
only two possible values, any function of Y has the form aY + b, so Ê [X|Y ] = E[X|Y ].
3.9. Diagonalizing a two-dimensional Gaussian distribution
If U = √^12
, and Y = U T^ X, then X = U Y and Cov(Y ) = U T^ KU =
λ 1 0 0 λ 2
where the variances are λ 1 = 1 + ρ and λ 2 = 1 − ρ. The columns of U could be swapped, in which case the eigenvalues would be swapped, and either column of U could be multiplied by minus one. Otherwise the choice of U is unique. One way to find U is to first find the eigenvalues by solving the characteristic equation det(Iλ − K) = 0, find the eigenvectors by solving the eigen-equation Kv = λv, and let the columns of U be the normalized eigenvectors.
3.15. A quadratic estimator (a) By the Orthogonality Principle, we want (|Y |−a−bY −cY 2 ) ⊥ Y i^ for i = 0, 1 , 2. Equivalently,
E[|Y |] − a − bE[Y ] − cE[Y 2 ] = 0 E[|Y |Y ] − aE[Y ] − bE[Y 2 ] − cE[Y 3 ] = 0 E[|Y |Y 2 ] − aE[Y 2 ] − bE[Y 3 ] − cE[Y 4 ] = 0
or equivalently,
- 8 − a − c = 0 −b = 0
- 6 − a − 3 c = 0
yielding a = c = 0.4 and b = 0, so X̂ = 0.4 + 0. 4 Y 2. MSE=E[X^2 ] − E[ Xˆ^2 ] = E[Y 2 ] − (0.4)^2 (1 + 2 E[Y 2 ] + E[Y 4 ]) = 1 − 0 .16(1 + 2 + 3) = 0.04.
3.17. Estimation for an additive Gaussian noise model The vectors X and Y are jointly Gaussian since they are obtained from X and Z via linear operations. Thus, the conditional distribution of X given Y is a Gaussian distribution. For a given value of Y , that distribution has mean μX + KXY K Y− 1 (Y − μY ) and covariance matrix equal to
KX − KXY K− Y 1 KY X.
