Download Solutions to Problem Set 2 on Random Processes | ECE 534 and more Assignments Electrical and Electronics Engineering in PDF only on Docsity!
ECE 534 RANDOM PROCESSES FALL 2008
SOLUTIONS TO PROBLEM SET 2
1 The reciprocal of the limit is the limit of the reciprocal Let > 0. Let ′^ = min{ |x∞ 2 |, x
(^2) ∞ 2 }.^ By the hypothesis, there exists a value of^ no^ so large that for all n ≥ no, |xn − x∞| ≤ ′. This condition implies that |xn| ≥ |x∞|/2, because of the choice of ′. Therefore, for all n ≥ no, (^) ∣ ∣ ∣∣^1 xn
x∞
∣∣ = |xn^ −^ x∞| |xn||x∞|
x^2 ∞
which, by definition, shows that (1/xn) converges to 1/x∞.
2 Limits of some deterministic series (a) Convergent. This is the power series expansion for ex, which is everywhere convergent, evalu- ated at x = 3. The value of the sum is thus e^3. Another way to show the series is convergent is to notice that for n ≥ 3 the nth^ term can be bounded above by 3 n n! =^
33 3!
3 4
3 5 · · ·^
3 n ≤^ (4.5)(^
3 4 )
n− (^3). Thus,
the sum is bounded by a constant plus a geometric series, so it is convergent. (b) Convergent. Let 0 < η < 1. Then ln n < nη^ for all large enough n. Also, n + 2 ≤ 2 n for all large enough n, and n+5 ≥ n for all n. Therefore, the nth^ term in the series is bounded above, for all suffi- ciently large n, by 2 n·n η n^3 = 2n
η− (^2). Therefore, the sum in (b) is bounded above by finitely many terms
of the sum, plus 2
n=1 n η− (^2) , which is finite, because, for α > 1, ∑∞ n=1 n −α (^) < 1+∫^ ∞ 1 x
−αdx = α α− 1 , as shown in an example in the appendix of the notes. (c) Not convergent. Let 0 < η < 0. 2. Then log(n + 1) ≤ nη^ for all n large enough, so for n large enough the nth^ term in the series is greater than or equal to n−^5 η. The series is therefore divergent. We used the fact that
n=1 n −α (^) is infinite for any 0 ≤ α ≤ 1 , because it is greater than or equal
to the integral
1 x
−αdx, which is infinite for 0 ≤ α ≤ 1.
3 Convergence of random variables on (0,1], version 2 (a) (a.s, p., d., not m.s.) For any ω ∈ Ω fixed, the deterministic sequence Xn(ω) converges to zero. So Xn → 0 a.s. The sequence thus also converges in p. and d. If the sequence converged in the m.s. sense, the limit would also have to be zero, but
E[|Xn − 0 |^2 ] = E[Xn|^2 ] =
n^2
0
ω dω = +∞ 6 → 0.
The sequence thus does not converge in the m.s. sense. (b) (a.s, p., d., not m.s.) For any ω ∈ Ω fixed, except the single point 1 which has zero probability, the deterministic sequence Xn(ω) converges to zero. So Xn → 0 a.s. The sequence also converges in p. and d. If the sequence converged in the m.s. sense, the limit would also have to be zero, but
E[|Xn − 0 |^2 ] = E[Xn|^2 ] = n^2
0
ω^2 ndω =
n^2 2 n + 1
The sequence thus does not converge in the m.s. sense. (c) (d. only) For ω fixed and irrational, the sequence does not even come close to settling down,
so intuitively we expect the sequence does not converge in any of the three strongest senses: a.s., m.s., or p. To prove this, it suffices to prove that the sequence doesn’t converge in p. Since the sequence is bounded, convergence in probability is equivalent to convergence in the m.s. sense, so it also would suffice to prove the sequence does not converge in the m.s. sense. The Cauchy criteria for m.s. convergence would be violated if E[(Xn − X 2 n)^2 ] 6 → 0 as n → ∞. By the double angle formula, X 2 n(ω) = 2ω sin(2πnω) cos(2πnω) so that
E[(Xn − X 2 n)^2 ] =
0
ω^2 (sin(2πnω))^2 (1 − 2 cos(2πnω))^2 dω
and this integral clearly does not converge to zero as n → ∞. In fact, following the heuristic reasoning below, the limit can be shown to equal E[sin^2 (Θ)(1 − 2 cos(Θ))^2 ]/ 3 , where Θ is uniformly distributed over the interval [0, 2 π]. So the sequence (Xn) does not converge in m.s., p., or a.s. senses. The sequence does converge in the distribution sense. We shall give a heuristic derivation of the limiting CDF. Note that the CDF of Xn is given by
FXn (c) =
0
I{f (ω) sin(2πnω)≤c}dω (1)
where f is the function defined by f (ω) = ω. As n → ∞, the integrand in (1) jumps between zero and one more and more frequently. For any small > 0, we can imagine partitioning [0, 1] into intervals of length . The number of oscillations of the integrand within each interval converges to infinity, and the factor f (ω) is roughly constant over each interval. The fraction of a small interval for which the integrand is one nearly converges to P {f (ω) sin(Θ) ≤ c} , where Θ is a random variable that is uniformly distributed over the interval [0, 2 π], and ω is a fixed point in the small interval. So the CDF of Xn converges for all constants c to: ∫ (^1)
0
P {f (ω) sin(Θ) ≤ c} dω. (2)
(Note: The following observations can be used to make the above argument rigorous. The integrals in (1) and (2) would be equal if f were constant within each interval of the form ( (^) ni , i+1 n ). If f is continuous on [0, 1], it can be approximated by such step functions with maximum approximation error converging to zero as n → ∞. Details are left to the reader.)
4 On the maximum of a random walk with negative drift (a) By the strong law of large numbers, P {Sn/n → − 1 } = 1. Therefore, with probability one, Sn/n ≤ 0 for all sufficiently large n. That is, with probability one, Sn > 0 only finitely many times. The random variable Z, with probability one, is thus the maximum of only finitely many nonnegative numbers. So Z is finite with probability one. (b) Suppose P {X 1 = c − 1 } = P {X 1 = −c − 1 } = 0.5 for a constant c > 0. Then X 1 has mean -1 as required. Following the hint, for c ≥ 1, we have E[Z] ≥ E[max{ 0 , X 1 }] = (c − 1)/ 2. Observe that E[Z] can be made arbitrarily large by taking c arbitrarily large. So the answer to the question is no. (Note: More can be said about E[Z] if the variance of X 1 is known. A celebrated bound of
J.F.C. Kingman is that E[Z] ≤ Var − 2 E([XX^11 )] .)
E[SnSm] = E[
∑^ n
j=
Bj
∑m
k=
Bk] =
∑^ n
j=
∑^ m
k=
E[Bj Bk] →
∑^ ∞
j=
∑^ ∞
k=
E[Bj Bk] (3)
∑^ ∞
j=
∑^ ∞
k=j+
)j ( 3 4
)k−j
∑^ ∞
j=
)j
∑^ ∞
j=
∑^ ∞
l=
)j ( 3 4
)l
∑^ ∞
j=
)j
∑^ ∞
j=
)j
∑^ ∞
l=
)l
A visual way to derive (4), is to note that (3) is the sum of all entries in the infinite 2-d array:
.. .
8
4
8
4
8
5 8
4
8
8
4
8
8
4
8
4
Therefore,
8
)j ( 2
l=
4
)l
is readily seen to be the sum of the jth^ term on the diagonal,
plus all terms directly above or directly to the right of that term. (d) Mean square convergence implies convergence of the mean. Thus, the mean of the limit is limn→∞ E[Sn] = limn→∞
∑n k=1 E[Bk] =^
k=1(^
3 4 ) k (^) = 3. The second moment of the limit is the
limit of the second moments, namely 353 , so the variance of the limit is 353 − 32 = 83. (e) Yes. Each sample path of the sequence Sn is monotone nondecreasing and is hence convergent. Thus, limn→∞ a.s. Sn exists. The limit has to be the same as the m.s. limit. (To show that the limit is finite with probability one, and hence an ordinary random variable, the monotone conver- gence theorem could be applied, yielding E[limn→∞ S∞] = limn→∞ E[Sn] = 1. )
8 Large deviation exponent for a mixture distribution (a) ˜l(a) = maxθ {θa − MX (θ)} where
MX (θ) = log E[exp(θX)] = log{f E[exp(θY )] + (1 − f )E[exp(θZ)]} = log{f exp(MY (θ)) + (1 − f ) exp(MZ (θ))}
(b) View f exp(MY (θ)) + (1 − f ) exp(MZ (θ)) as an average of exp(MY (θ)) and exp(MZ (θ)). The definition of concavity (or Jensen’s inequality) applied to the concave function log u implies that log(average) ≥ average(log), so that log{f exp(MY (θ)) + (1 − f ) exp(MZ (θ)) ≥ f MY (θ) + (1 − f )MZ (θ), where we also used the fact that log exp MY (θ) = MY (θ). Therefore, ˜l(a) ≤ l(a) for all a. Remark: This means that Se nn is more likely to have large deviations than S nn. That is reasonable,
because Se nn has randomness due not only to FY and FZ , but also due to the random coin flips. This point is particularly clear in case the Y ’s and Z’s are constant, or nearly constant, random variables.
