Download Midterm Exam 1 Solutions - Random Processes | ECE 534 and more Exams Electrical and Electronics Engineering in PDF only on Docsity!
University of Illinois at Urbana-Champaign
ECE 534: RANDOM PROCESSES
Fall 2006
Midterm 1 Solutions
Monday, October 16, 2006
1 (Note that each part of this problem can be solved indepen- dently by using what has been shown in the previous part.) Let (Ω, F, P ) be a probability space. Define:
An ∈ F, n = 1, 2 ,... Bm =
n≥m
An, m = 1, 2 ,... (1)
a) EXTRA CREDIT (although this doesn’t take much effort to show:) Show that
A , {ω s.t. ω ∈ An finitely often } (2)
can be expressed as
A =
⋃^ ∞
m=
n≥m
Acn
Solution: if the An occur finitely many times then for some m they occur never again after m. This is equivalent to saying that A =
n≥m A
c n for some^ m. Thus
A =
⋃^ ∞
m=
n≥m
Acn
b) Show that if P (Bm) → 0 then P (A) = 1. Hint: use the result of part a).
Solution: Note that P (Bm) → 0 if and only if P (Bmc) → 1.
Bmc =
n≥m
Acn ⊆
⋃^ ∞
m=
n≥m
Acn = A
⇒ P (A) ≥ P (Bcm) ⇒ P (A) ≥ lim m→∞ P (Bmc) = 1.
c) Let S be a random variable along with the sequence S 1 , S 2 ,... of random variables, all defined on (Ω, F, P ). Define An() = {ω : |Sn(ω) − S(ω)| > } and Bm() along with A() in terms of An() according to (1) and (2). Show that if (^) ∞ ∑
n=
P (An()) < ∞ for all > 0
then Sn →a.s. S Hint: use the result of part b).
identically to X. Prove the strong law of large numbers. Hint: use the result of part c). Solution: Let Sn = (^) n^1
∑n i=1 Xi^ and^ S^ =^ μ.
P (An()) = P (|Sn − μ| > ) = P (Sn − μ > ) + P (Sn − μ < −) = P (Sn > μ + ) + P (−Sn > −μ + ) ≤ exp(−nl(μ + )) + exp(−nl(μ − ))
where the final inequality follows from the Chernoff bound and noting that E
[
eθ(−X)
]
= M (−θ). For any > 0, since l(μ + ) and l(μ − ) are both positive, it follows that the geometric sum
n=1 P^ (An()) is finite.^ Thus from (c) we have that Sn →a.s. μ.
2 Let X and Y be independent random variables where:
X ∼ N (0, σ^2 ), Y =
− 1 w/ prob. (^12) 1 w/ prob. (^12)
Let Z = X + Y and W = XY. a) Find the MMSE estimator of Z given Y. Solution: The MMSE estimator is given by E[Z|Y ] = E[X + Y |Y ], which because X and Y are independent, is equal to E[0 + Y |Y ] = Y. b) Find the linear MMSE estimator of Z given Y. Solution: Since the solution in a) is already linear in Y , the answer is the same as a): Eˆ[Z|Y ] = Y. c) Are X and W uncorrelated? Are X and W independent?
Solution: To determine whether or not they are correlated, we must evaluate Cov (X, W ):
Cov (X, W ) = E [(X − EX)(W − EW )] = E [X(W − EW )]
= E
[
X
W −
))]
= E [XW ] = E
[
X^2 Y
]
E
[
X^2
]
E
[
X^2
]
So yes, they are uncorrelated. As for independence, no they are not indepen- dent. This can be shown in numerous ways. One way is: conditioned upon X, W can only take on two possible values. However, W has a marginal distribu- tion that permits it to take on any range of values with non-zero probability.
d) Are X and W Gaussian? Are they jointly Gaussian?
Solution: We are already given that X is Gaussian. As for W ,
fW (w) =
fX (w) +
fX (−w) = fX (w)
where the last equality follows from the symmetry of the Gaussian distribu- tion around 0. Thus W is Gaussian. They are not jointly Gaussian because we know that for jointly Gaussian random variables, uncorrelatedness implies independence. But in c) we showed that they are uncorrelated but not inde- pendent.
