Download Probability and Expectation Calculations for Random Variables and more Assignments Music in PDF only on Docsity!
ECE 541
Stochastic Signals and Systems
Problem Set 1 Solutions
Sept 2005
Problem Solutions : Yates and Goodman, 2.2.8 2.3.13 2.5.9 2.6.6 2.7.7 2.7.8 2.8.8 2.10. 3.2.5 3.4.14 3.5.10 3.7.2 3.8.8 and 3.8.
Problem 2.2.8 Solution
From the problem statement, a single is twice as likely as a double, which is twice as likely as a triple, which is twice as likely as a home-run. If p is the probability of a home run, then PB (4) = p PB (3) = 2p PB (2) = 4p PB (1) = 8p (1)
Since a hit of any kind occurs with probability of .300, p + 2p + 4p + 8p = 0.300 which implies p = 0.02. Hence, the PMF of B is
PB (b) =
- 70 b = 0
- 16 b = 1
- 08 b = 2
- 04 b = 3
- 02 b = 4 0 otherwise
Problem 2.3.13 Solution
(a) Let Sn denote the event that the Sixers win the series in n games. Similarly, Cn is the event that the Celtics in in n games. The Sixers win the series in 3 games if they win three straight, which occurs with probability
P [S 3 ] = (1/2)^3 = 1/ 8 (1)
The Sixers win the series in 4 games if they win two out of the first three games and they win the fourth game so that
P [S 4 ] =
(1/2)^3 (1/2) = 3/ 16 (2)
The Sixers win the series in five games if they win two out of the first four games and then win game five. Hence,
P [S 5 ] =
(1/2)^4 (1/2) = 3/ 16 (3)
By symmetry, P [Cn] = P [Sn]. Further we observe that the series last n games if either the Sixers or the Celtics win the series in n games. Thus,
P [N = n] = P [Sn] + P [Cn] = 2P [Sn] (4)
Consequently, the total number of games, N , played in a best of 5 series between the Celtics and the Sixers can be described by the PMF
PN (n) =
2(1/2)^3 = 1/ 4 n = 3 2
1
(1/2)^4 = 3/ 8 n = 4 2
2
(1/2)^5 = 3/ 8 n = 5 0 otherwise
(b) For the total number of Celtic wins W , we note that if the Celtics get w < 3 wins, then the Sixers won the series in 3 + w games. Also, the Celtics win 3 games if they win the series in 3,4, or 5 games. Mathematically,
P [W = w] =
P [S3+w] w = 0, 1 , 2 P [C 3 ] + P [C 4 ] + P [C 5 ] w = 3
Thus, the number of wins by the Celtics, W , has the PMF shown below.
PW (w) =
P [S 3 ] = 1/ 8 w = 0 P [S 4 ] = 3/ 16 w = 1 P [S 5 ] = 3/ 16 w = 2 1 /8 + 3/16 + 3/16 = 1/ 2 w = 3 0 otherwise
(c) The number of Celtic losses L equals the number of Sixers’ wins WS. This implies PL(l) = PWS (l). Since either team is equally likely to win any game, by symmetry, PWS (w) = PW (w). This implies PL(l) = PWS (l) = PW (l). The complete expression of for the PMF of L is
PL (l) = PW (l) =
1 / 8 l = 0 3 / 16 l = 1 3 / 16 l = 2 1 / 2 l = 3 0 otherwise
Problem 2.5.9 Solution
In this ”double-or-nothing” type game, there are only two possible payoffs. The first is zero dollars, which happens when we lose 6 straight bets, and the second payoff is 64 dollars which happens unless we lose 6 straight bets. So the PMF of Y is
PY (y) =
(1/2)^6 = 1/ 64 y = 0 1 − (1/2)^6 = 63/ 64 y = 64 0 otherwise
For the ultra-reliable case,
Ru = ru(W ) =
− 30 W = 0,
k − 30 W = 1, PW (w) =
1 − (1 − q/2)^10 w = 0, (1 − q/2)^10 w = 1, 0 otherwise.
Thus we can express the expected profit as
E [ru(W )] =
∑^1
w=
PW (w) ru(w) (6)
= PW (0) (−30) + PW (1) (k − 30) (7) = (1 − (1 − q/2)^10 )(−30) + (1 − q/2)^10 (k − 30) = (0.95)^10 k − 30 (8)
To determine which implementation generates the most profit, we solve E[Ru] ≥ E[Rs], yielding k ≥ 20 /[(0.95)^10 − (0.9)^10 ] = 80.21. So for k < $80.21 using all standard devices results in greater revenue, while for k > $80.21 more revenue will be generated by imple- menting all ultra-reliable devices. That is, when the price commanded for a working circuit is sufficiently high, we should build more-expensive higher-reliability circuits. If you have read ahead to Section 2.9 and learned about conditional expected values, you might prefer the following solution. If not, you might want to come back and review this alternate approach after reading Section 2.9. Let W denote the event that a circuit works. The circuit works and generates revenue of k dollars if all of its 10 constituent devices work. For each implementation, standard or ultra-reliable, let R denote the profit on a device. We can express the expected profit as
E [R] = P [W ] E [R|W ] + P [W c] E [R|W c] (9)
Let’s first consider the case when only standard devices are used. In this case, a circuit works with probability P [W ] = (1 − q)^10. The profit made on a working device is k − 10 dollars while a nonworking circuit has a profit of -10 dollars. That is, E[R|W ] = k − 10 and E[R|W c] = −10. Of course, a negative profit is actually a loss. Using Rs to denote the profit using standard circuits, the expected profit is
E [Rs] = (1 − q)^10 (k − 10) + (1 − (1 − q)^10 )(−10) = (0.9)^10 k − 10 (10)
And for the ultra-reliable case, the circuit works with probability P [W ] = (1 − q/2)^10. The profit per working circuit is E[R|W ] = k − 30 dollars while the profit for a nonworking circuit is E[R|W c] = −30 dollars. The expected profit is
E [Ru] = (1 − q/2)^10 (k − 30) + (1 − (1 − q/2)^10 )(−30) = (0.95)^10 k − 30 (11)
Not surprisingly, we get the same answers for E[Ru] and E[Rs] as in the first solution by performing essentially the same calculations. it should be apparent that indicator random variable W in the first solution indicates the occurrence of the conditioning event W in the second solution. That is, indicators are a way to track conditioning events.
Problem 2.7.8 Solution
(a) There are
6
equally likely winning combinations so that
q =
6
≈ 1. 07 × 10 −^7 (1)
(b) Assuming each ticket is chosen randomly, each of the 2n − 1 other tickets is indepen- dently a winner with probability q. The number of other winning tickets Kn has the binomial PMF
PKn (k) =
{ ( 2 n− 1 k
qk(1 − q)^2 n−^1 −k^ k = 0, 1 ,... , 2 n − 1 0 otherwise
(c) Since there are Kn + 1 winning tickets in all, the value of your winning ticket is Wn = n/(Kn + 1) which has mean
E [Wn] = nE
[
Kn + 1
]
Calculating the expected value
E
[
Kn + 1
]
(^2) ∑n− 1
k=
k + 1
PKn (k) (4)
is fairly complicated. The trick is to express the sum in terms of the sum of a binomial PMF.
E
[
Kn + 1
]
(^2) ∑n− 1
k=
k + 1
(2n − 1)! k!(2n − 1 − k)!
qk(1 − q)^2 n−^1 −k^ (5)
2 n
(^2) ∑n− 1
k=
(2n)! (k + 1)!(2n − (k + 1))! qk(1 − q)^2 n−(k+1)^ (6)
By factoring out 1/q, we obtain
E
[
Kn + 1
]
2 nq
(^2) ∑n− 1
k=
2 n k + 1
qk+1(1 − q)^2 n−(k+1)^ (7)
2 nq
∑^2 n
j=
2 n j
qj^ (1 − q)^2 n−j ︸ ︷︷ ︸ A
We observe that the above sum labeled A is the sum of a binomial PMF for 2n trials and success probability q over all possible values except j = 0. Thus
A = 1 −
2 n 0
q^0 (1 − q)^2 n−^0 = 1 − (1 − q)^2 n^ (9)
Problem 2.10.4 Solution
Suppose Xn is a Zipf (n, α = 1) random variable and thus has PMF
PX (x) =
c(n)/x x = 1, 2 ,... , n 0 otherwise
The problem asks us to find the smallest value of k such that P [Xn ≤ k] ≥ 0 .75. That is, if the server caches the k most popular files, then with P [Xn ≤ k] the request is for one of the k cached files. First, we might as well solve this problem for any probability p rather than just p = 0.75. Thus, in math terms, we are looking for
k = min
k′|P
[
Xn ≤ k′
]
≥ p
What makes the Zipf distribution hard to analyze is that there is no closed form expression for
c(n) =
( (^) n ∑
x=
x
Thus, we use Matlab to grind through the calculations. The following simple program generates the Zipf distributions and returns the correct value of k.
function k=zipfcache(n,p); %Usage: k=zipfcache(n,p); %for the Zipf (n,alpha=1) distribution, returns the smallest k %such that the first k items have total probability p pmf=1./(1:n); pmf=pmf/sum(pmf); %normalize to sum to 1 cdf=cumsum(pmf); k=1+sum(cdf<=p);
The program zipfcache generalizes 0.75 to be the probability p. Although this program is sufficient, the problem asks us to find k for all values of n from 1 to 10^3 !. One way to do this is to call zipfcache a thousand times to find k for each value of n. A better way is to use the properties of the Zipf PDF. In particular,
P
[
Xn ≤ k′
]
= c(n)
∑^ k′
x=
x
c(n) c(k′)
Thus we wish to find
k = min
k′| c(n) c(k′)
≥ p
= min
k′|
c(k′)
p c(n)
Note that the definition of k implies that 1 c(k′)
p c(n)
, k′^ = 1,... , k − 1. (6)
Using the notation |A| to denote the number of elements in the set A, we can write
k = 1 +
k′|
c(k′)
p c(n)
This is the basis for a very short Matlab program:
function k=zipfcacheall(n,p); %Usage: k=zipfcacheall(n,p); %returns vector k such that the first %k(m) items have total probability >= p %for the Zipf(m,1) distribution. c=1./cumsum(1./(1:n)); k=1+countless(1./c,p./c);
Note that zipfcacheall uses a short Matlab program countless.m that is almost the same as count.m introduced in Example 2.47. If n=countless(x,y), then n(i) is the number of elements of x that are strictly less than y(i) while count returns the number of elements less than or equal to y(i). In any case, the commands k=zipfcacheall(1000,0.75); plot(1:1000,k);
is sufficient to produce this figure of k as a function of m:
0 100 200 300 400 500 600 700 800 900 1000 0
50
100
150
200
n
k
We see in the figure that the number of files that must be cached grows slowly with the total number of files n. Finally, we make one last observation. It is generally desirable for Matlab to execute operations in parallel. The program zipfcacheall generally will run faster than n calls to zipfcache. However, to do its counting all at once, countless generates and n × n array. When n is not too large, say n ≤ 1000, the resulting array with n^2 = 1, 000 ,000 elements fits in memory. For much large values of n, say n = 10^6 (as was proposed in the original printing of this edition of the text, countless will cause an “out of memory” error.
Problem 3.2.5 Solution
fX (x) =
ax^2 + bx 0 ≤ x ≤ 1 0 otherwise
First, we note that a and b must be chosen such that the above PDF integrates to 1. ∫ (^1)
0
(ax^2 + bx) dx = a/3 + b/2 = 1 (2)
Hence, b = 2 − 2 a/3 and our PDF becomes
fX (x) = x(ax + 2 − 2 a/3) (3)
(c) We can use the integration by parts formula
u dv = uv −
v du by defining u = 1 − FX (x) and dv = dx. This yields ∫ (^) ∞
0
[1 − FX (x)] dx = x[1 − FX (x)]|∞ 0 +
0
xfX (x) dx (5)
By applying part (a), we now observe that
x [1 − FX (x)]|∞ 0 = lim r→∞ r[1 − FX (r)] − 0 = lim r→∞ rP [X > r] (6)
By part (b), limr→∞ rP [X > r] = 0 and this implies x[1 − FX (x)]|∞ 0 = 0. Thus, ∫ (^) ∞
0
[1 − FX (x)] dx =
0
xfX (x) dx = E [X] (7)
Problem 3.5.10 Solution
This problem is mostly calculus and only a little probability. From the problem statement, the SNR Y is an exponential (1/γ) random variable with PDF
fY (y) =
(1/γ)e−y/γ^ y ≥ 0 , 0 otherwise.
Thus, from the problem statement, the BER is
P (^) e = E [Pe(Y )] =
−∞
Q(
2 y)fY (y) dy =
0
Q(
2 y) y γ
e−y/γ^ dy (2)
Like most integrals with exponential factors, its a good idea to try integration by parts. Before doing so, we recall that if X is a Gaussian (0, 1) random variable with CDF FX (x), then Q(x) = 1 − FX (x). (3)
It follows that Q(x) has derivative
Q′(x) = dQ(x) dx
dFX (x) dx
= −fX (x) = −
2 π
e−x (^2) / 2 (4)
To solve the integral, we use the integration by parts formula
∫ (^) b a u dv^ =^ uv|
b a −^
∫ (^) b a v du, where
u = Q(
2 y) dv =
γ
e−y/γ^ dy (5)
du = Q′(
2 y)
2 y
e−y 2 √πy v = −e−y/γ^ (6)
From integration by parts, it follows that
P (^) e = uv|∞ 0 −
0
v du = −Q(
2 y)e−y/γ
∞ 0
0
y
e−y[1+(1/γ)]^ dy (7)
= 0 + Q(0)e−^0 −
π
0
y−^1 /^2 e−y/γ¯^ dy (8)
where ¯γ = γ/(1 + γ). Next, recalling that Q(0) = 1/2 and making the substitution t = y/¯γ, we obtain
P (^) e =
¯γ π
0
t−^1 /^2 e−t^ dt (9)
From Math Fact B.11, we see that the remaining integral is the Γ(z) function evaluated z = 1/2. Since Γ(1/2) =
π,
P (^) e =
¯γ π
[
γ¯
]
[
γ 1 + γ
]
Problem 3.7.2 Solution
Since Y =
X, the fact that X is nonegative and that we asume the squre root is always positive implies FY (y) = 0 for y < 0. In addition, for y ≥ 0, we can find the CDF of Y by writing FY (y) = P [Y ≤ y] = P
[√
X ≤ y
]
= P
[
X ≤ y^2
]
= FX
y^2
For x ≥ 0, FX (x) = 1 − e−λx. Thus,
FY (y) =
1 − e−λy 2 y ≥ 0 0 otherwise
By taking the derivative with respect to y, it follows that the PDF of Y is
fY (y) =
2 λye−λy^2 y ≥ 0 0 otherwise
In comparing this result to the Rayleigh PDF given in Appendix A, we observe that Y is a Rayleigh (a) random variable with a =
2 λ.
Problem 3.8.8 Solution
(a) The event Bi that Y = ∆/2 + i∆ occurs if and only if i∆ ≤ X < (i + 1)∆. In particular, since X has the uniform (−r/ 2 , r/2) PDF
fX (x) =
1 /r −r/ 2 ≤ x < r/ 2 , 0 otherwise,
we observe that P [Bi] =
∫ (^) (i+1)∆
i∆
r
dx =
r
In addition, the conditional PDF of X given Bi is
fX|Bi (x) =
fX (x) /P [B] x ∈ Bi 0 otherwise =
1 /∆ i∆ ≤ x < (i + 1)∆ 0 otherwise (3) It follows that given Bi, Z = X − Y = X − ∆/ 2 − i∆, which is a uniform (−∆/ 2 , ∆/2) random variable. That is,
fZ|Bi (z) =
1 /∆ −∆/ 2 ≤ z < ∆/ 2 0 otherwise
