Download Probability and Statistics: Finding Probabilities and Expectations of Random Variables and more Exams Statistics in PDF only on Docsity!
University of Illinois Spring 2008
ECE 313: Solutions to the Final Exam
- (a) A and B are two events such that 0 < P (A) < 1 and 0 < P (B) < 1.
TRUE P (A ∪ B) ≥ max{P (A), P (B)}. (A ⊂ A ∪ B, B ⊂ A ∪ B) TRUE P (A ∩ B) ≤ min{P (A), P (B)}. (A ∩ B ⊂ A, A ∩ B ⊂ B) TRUE If A and B are independent, P (A|B) + P (Ac|Bc) = 1. TRUE P (A|B)P (B) + P (Ac|Bc)P (Bc) = 1 − P (A ⊕ B). FALSE P (A|B)P (B) + P (Ac|B)P (B) = P (A). (true if RHS were P (B)) TRUE If P (A) = P (B), then P (A|B) = P (B|A). FALSE If P (A|B) = P (B|A), then P (A) = P (B). Conclusion does not hold if P (A|B) = P (B|A) = 0. TRUE If P (A|B) = P (A), then P (Bc|A) = 1 − P (B). A and B are independent events (b) X and Y are random variables such that var(X ) = var(Y) = σ^2 < ∞. var(2X + 3Y + 4) = 4σ^2 + 9σ^2 + 12 · cov(X , Y) = var(3X − 2 Y + 1) = 9σ^2 + 4σ^2 − 12 · cov(X , Y) implies that cov(X , Y) = 0. TRUE X and Y are uncorrelated random variables. FALSE X and Y are independent random variables. TRUE var(2X + 3Y + 4) = var(2X − 3 Y + 1). TRUE cov(X + Y, X − Y) = 0.
- At the Democratic National Convention (DNC), Hillary Clinton and Barack Obama have equal numbers of delegates committed to them, and neither candidate can win the nomination on a ballot. In desperation, the DNC decides to have a series of debates between the candidiates to decide the Democratic nominee. Hillary wins a debate (event H) with probability p, and Barack wins a debate (event B) with probability q = 1 − p. There are no draws. The debates continue until one of the candidates wins two debates in a row and is declared the Democratic nominee. Successive debates can be regarded as independent trials of an experiment, and X denotes the total number of debates. Express the answers to the following questions in terms of p and q, that is, do not write (say) pq as p(1 − p) or multiply it out as p − p^2. On the other hand, feel free to simplify p + q = 1. (a) For n > 0, find the probability that more than 2 n debates occur at the DNC. Solution: If 2n debates have not produced a winner, then the winners must have alternated, that is, the result of these 2n debates must have been either HBHB · · · HB or BHBH · · · BH. Hence,
P {X > 2 n} = pqpq · · · pq + qpqp · · · qp = 2(pq)n. Note that this formula does not hold for n = 0. (Why not? and what is the value of P {X > 0 } anyway?) (b) For n ≥ 0, find the probability that more than 2 n + 1 debates occur at the DNC. Solution: By the same argument as in part (a) that the winners must alternate, we get that P {X > 2 n + 1} = (pq)np + (qp)nq = (pq)n^ which does hold for n = 0.
(c) Find E[X ]. Hint: use the results of parts (a) and (b). Solution: Since X is a positive integer-valued random variable, we have that
E[X ] =
∑^ ∞
k=
P (X > k) =
∑^ ∞
n=
P {X > 2 n} +
∑^ ∞
n=
P {X > 2 n + 1}
∑^ ∞
n=
P {X > 2 n} + 1 +
∑^ ∞
n=
P {X > 2 n + 1}
∑^ ∞
n=
2(pq)n^ + 1 +
∑^ ∞
n=
(pq)n^ = −1 + 3
[
1 + (pq) + (pq)^2 + · · ·
]
= (^1) −^3 pq − 1 = 2 + 1 −^ pqpq.
Notice that E[X ] = 2 if p = 0 or q = 0 which makes perfect sense. On the other hand, symmetry indicates that the maximum value of E[X ] should occur when p = q = 12 , and surprisingly, this maximum value is only 3; on average, three or fewer debates can be expected to occur at the DNC! Alternatively, we can steal the result from part (d) that P {X = 2n + 1} = (pq)n for n ≥ 1, and use the same argument as in part (d) to deduce that for n ≥ 1, P {X = 2n} = P {X > 2 n − 1 } − P {X > 2 n} = (pq)n−^1 − 2(pq)n = (pq)n−^1 (1 − 2 pq) = (p^2 + q^2 )(pq)n−^1. Since X ≥ 2, we get
E[X ] =
∑^ ∞
k=
k · P {X = k} = (p^2 + q^2 )
∑^ ∞
n=
2 n · (pq)n−^1 +
∑^ ∞
n=
(2n + 1)(pq)n
= 2(p^2 + q^2 )
∑^ ∞
n=
n(pq)n−^1 + 2(pq)
∑^ ∞
n=
n(pq)n−^1 +
∑^ ∞
n=
(pq)n
= 2(p^2 + q^2 + pq) (^) (1 −^1 pq) 2 + (^1) −^1 pq − 1
2 p^2 + 2q^2 + 2pq + 1 − pq − 1 + 2pq − (pq)^2 (1 − pq)^2
= 2(p
(^2) + q (^2) + 2pq) − pq − (pq) 2 (1 − pq)^2
= 2(p^ +^ q)
(^2) − pq − (pq) 2 (1 − pq)^2
=^2 −^ pq^ −^ (pq)
2 (1 − pq)^2 =
(2 + pq)(1 − pq) (1 − pq)^2 =
2 + pq 1 − pq just as before.
(d) Find P {X = 2n + 1} and P {X = 2n + 1
X > 2 n}. Solution: For integer-valued X , P (X = k + 1) = P {X > k} − P {X > k + 1}. So, P {X = 2n + 1} = P {X > 2 n} − P {X > 2 n + 1} = 2(pq)n^ − (pq)n^ = (pq)n. Note that since P {X > 0 } = 1 6 = 2(pq)^0 , the result P {X = 2n + 1} = (pq)n^ does not hold for n = 0. For n = 0, P {X = 1} = P {X > 0 } − P {X > 1 } = 1 − 1 = 0 as should be obvious if we just think about the problem a bit! From this we get
lies below the parabola in the right-hand figure above. Thus, we have that
P {real solutions} = P {X 2 ≥ Y} =
u=
∫ (^) u 2
v=
(u + v)dvdu
u=
[
uv +
v^2 2
u^2
v=
du =
u=
[
u^3 +
u^4 2
]
dv
[
u^4 4
5 10
1
0
=^1
+^1
(c) Find the conditional pdf fY^ ∣∣X (v
β) of Y given that X = β, where 0 < β < 1. Be sure to specify the value of fY^ ∣∣X (v
β) for all real numbers v.
Solution: fY^ ∣∣X (v
β) =
fX ,Y (β, v) fX (β) =^
β + v β + (^12)
= 2( 2 ββ^ ++ 1^ v) for 0 < v < 1, and fY|X (v
β) = 0 otherwise. The pdf is illustrated in the left-hand figure below. Using the formula for the area of a trapezoid, it is easily verified that the area under the curve equals 1.
2 3
v v
f Y^ ∣∣X (v
v
f Y^ ∣∣X (v
β) f Y^ ∣∣ X (v
2 β 2 β+
2 β+
4 3
(d) Find the minimum-mean-square-error (MMSE) estimate of Y given that X = β where 0 < β < 1. Solution: The MMSE estimator of Y given that X = β is the mean of the conditional pdf fY^ ∣∣X (v
β), viz.,
E[X |Y = β] =
0
2 v(β + v) 2 β + 1 dv^ =^
2 β + 1
[
βv^2 +
2 v^3 3
1
0
β + 2/ 3 2 β + 1 =
3 β + 2 6 β + 3. Note that the MMSE estimator is a nonlinear function of β, varying from value 2 3 at^ β^ = 0 to^
5 9 <^
2 3 at^ β^ = 1. Since both^ X^ and^ Y^ are more likely to be close to 1 than to 0, the estimated value of Y is pretty large even when β, the observed value of X , is close to 0. When β is close to 1, the estimated value of Y is still large but closer to 12 because the conditional pdf of Y is much “more uniform” when β is large than when β is small (e.g.. compare the middle and right-hand sketches in the figure on the previous page).
- Consider the following binary hypothesis testing problem. If hypothesis H 0 is true, the continuous random variable X ∼ Uniform(− 2 , 2), while if hypothesis H 1 is true,
the pdf of X is f 1 (u) =
4 (2^ − |u|),^ |u|^ <^2 , 0 , otherwise.
(a) The maximum-likelihood decision rule can be stated in the form |X |
Hx ≷ H 1 −x
η.
Specify whether x denotes 0 or 1, and find the values of η, the probability of false alarm PFA, and the probability of missed detection PMD. Solution: The easiest way to solve this problem is to sketch the two pdfs as shown in the left-hand figure below.
π 1 f 1 (u)
π 0 f 0 (u)
u
1 3
f 1 (u)
f 0 (u)
1 2
1 4
u
1 12
It is obvious that the maximum-likelihood decision is in favor of H 1 if |X | < 1, and hence x = 0, η = 1. By inspection, we get that PFA = 2 × 14 = 12 while PMD = 2 ×
2 ×^1 ×^
1 4
The graphically-challenged can proceed as follows. For − 2 < u < 2, the likelihood ratio is Λ(u) = f^1 (u) f 0 (u)
=^0 .25(2^ − |u|)
- 25
= 2 − |u|. When X = u is the observation, the maximum-likelihood decision rule decides in favor of H 1 if Λ(u) > 1. Hence Γ 1 = {u : |u| < 1 } and Γ 0 = {u : 1 < |u| < 2 }, that is, the ML decision rule is that if |X | > 1, the decision is that H 0 is the true hypothesis. Thus, we have x = 0, and η = 1. PFA =
Γ 1
f 0 (u) du =
− 1
4 du^ =
PMD =
Γ 0
f 1 (u) du = 2
1
(2 − u) du =^1 2
2 u − u
2 2
2
1
=^1
(b) Suppose that the hypotheses have a priori probabilities π 0 = 1/3 and π 1 = 2/3. What is the error probability P (E) of the maximum-likelihood decision rule? Solution: The probability of error of the ML decision rule is
P (E) = π 0 PFA + π 1 PMD =
3 ×^
3 ×^
(c) The MAP (also known as the minimum-error-probability or Bayesian) decision
rule can be stated in the form |X |
Hx ≷ H 1 −x
ξ. Specify whether x denotes 0 or 1,
and find the values of ξ and the error probability P (E). Solution: Sketching π 0 f 0 (u) and π 1 f 1 (u) as in the right-hand figure above, we easily see that the MAP decision is in favor of H 1 if |X | < 1 .5, and hence x = 0, ξ = 1.5. By inspection, we get that π 0 PFA = 3 × 121 = 14 = 13 × 34 while π 1 PMD = 2 ×
2 ×^
1 2 ×^
1 12
= 241 = 23 × 161 , that is, PFA = 34 , and PMD = 161.
- If α > 0, then the integrand is 0 for 0 < u < α, and we get fZ (α) =
α
exp(−u) exp(α − u) du = exp(α)^12 exp(− 2 u)
∞
α
=^12 exp(−α).
- If α < 0, then the integrand is nonzero for 0 < u < ∞, and we get fZ (α) =
0
exp(−u) exp(α − u) du = exp(α)
2 exp(−^2 u)
∞
0
2 exp(α).
We can combine these two cases and write fZ (α) = 12 exp(−|α|), −∞ < α < ∞.
- Suppose X and Y are jointly Gaussian random variables with means 0 and 2 respec- tively, variances 4 and 16 respectively, and correlation coefficient 12. Let W = −X +bY and Z = 7X − Y where b is a number whose value you will determine below. (a) For what value(s) of b does E[W] equal 0? Solution: E[W] = −E[X ] + bE[Y] = 0 + 2b = 0 ⇒ b = 0. (b) [10 points] For what value(s) of b does var(W) equal 3? Solution: 3 = var(W) = var(X ) + b^2 var(Y) − 2 bcov(X , Y) = 4 + 16b^2 − 2 b 12 2 · 4 implies that 16b^2 − 8 b + 1 = 0 ⇒ (4b − 1)^2 = 0 ⇒ b = 14. (c) For what value(s) of b are W and Z independent random variables? Solution: Since X and Y are jointly Gaussian random variables, W and Z are also jointly Gaussian random variables, and hence independent if cov(W, Z) = 0. But, cov(W, Z) = cov(−X + bY, 7 X − Y) = − 7 · cov(X , X ) + 7b · cov(Y, X ) + cov(X , Y) − b · cov(Y, Y) = − 7 · 4 + 7b · 4 + 4 − b · 16 = 12b − 24
and we see that cov(W, Z) = 0 when b = 2.
