Download Sample Mean-Probability and Statistics-Assignment Solution and more Exercises Probability and Statistics in PDF only on Docsity!
EE 5375/7375 Random Processes October 7, 2003
Homework #5 Solutions
Problem 1. textbook problem 6. A discrete-time random process Xn is defined as follows. A fair coin is tossed. If the outcome is heads, Xn = 1 for all n; if the outcome is tails, Xn = −1 for all n. (a) Sketch some sample paths of the process. (b) Find the PMF for Xn. (c) Find the joint PMF for Xn and Xn+k. (d) Find the mean and autocovariance functions of Xn.
(a) If the outcome is heads, then Xn = 1, 1 , 1 ,.. .. If the outcome is tails, then Xn = − 1 , − 1 , − 1 ,... (b) Since a head or tail is equally likely, it is equally likely that the process will be +1 or -1:
P (Xn = 1) = P (Xn = −1) = 1/ 2
(c) In either case of heads or tails, Xn and Xn+k will have the same value:
P (Xn = 1, Xn+k = 1) = P (Xn = − 1 , Xn+k = −1) =
In neither case will Xn and Xn+k have different values:
P (Xn = 1, Xn+k = −1) = P (Xn = − 1 , Xn+k = 1) = 0
(d) Since Xn is 1 or -1 with equal likelihood, the mean is E(Xn) = 1 · ( 12 ) − 1 · ( 12 ) = 0. The autocovariance function is
C(n, n + k) = E(XnXn+k) − E(Xn)E(Xn+k) = E(XnXn+k) = 1 · P (Xn = 1, Xn+k = 1) + (−1)(−1) · P (Xn = − 1 , Xn+k = −1) =
Problem 2. textbook problem 6. Let Sn denote a binomial counting process. (a) Show that P (Sn = j, Sm = i) 6 = P (Sn = j)P (Sm = i). (b) Find P (Sn = j|Sm = i) where n > m. (c) Show that P (Sn = j|Sm = i, Sl = k) = P (Sn = j|Sm = i) where n > m > l.
(a) We know that Sn has independent increments that follow a binomial distribution. That is, for n > m, the increment Sn − Sm is the sum of n − m Bernoulli trials, which has the same distribution as Sn−m. Hence,
P (Sn = j, Sm = i) = P (Sn − Sm = j − i, Sm = i) = P (Sn−m = j − i)P (Sm = i) 6 = P (Sn = j)P (Sm = i)
The last step follows because Sn−m is the sum of n − m Bernoulli trials whereas Sn is the sum of n Bernoulli trials. (b) Given Sm = i, the event Sn = j is the same as the increment Sn − Sm = j − i. This increment is the sum of n − m Bernoulli trials, so it is a binomial:
P (Sn = j|Sm = i) = P (Sn − Sm = j − i) =
n − m j − i
pj−i(1 − p)n−m−(j−i)
(c) We have to use the property of independent increments.
P (Sn = j|Sm = i, Sl = k) = P (Sn = j, Sm = i, Sl = k) P (Sm = i, Sl = k) = P (Sn − Sm = j − i, Sm − Sl = i − k, Sl = k) P (Sm − Sl = i − k, Sl = k) = P (Sn − Sm = j − i)P (Sm − Sl = i − k)P (Sl = k) P (Sm − Sl = i − k)P (Sl = k) = P (Sn − Sm = j − i) = P (Sn = j|Sm = i)
Problem 3. textbook problem 6. Consider the following moving average processes:
Yn =
(Xn + Xn− 1 ) , X 0 = 0
Zn =
Xn +
Xn− 1 , X 0 = 0
(a) Flip a coin 10 times to obtain a realization of a Bernoulli random process Xn. Find the resulting realizations of Yn and Zn. (b) Repeat part (a) with Xn given by the random step process introduced in Example 6.12 (i.e., Dn = 2In − 1 where In is the Bernoulli random process. At any time, the process takes value +1 with probability p and value -1 with probability 1 − p). (c) Find the mean, variance,and covariance of Yn and Zn if Xn is a Bernoulli random process. Are the sample means of Yn and Zn in part (a) close to their respective means? (d) Repeat part (c) if Xn is the random step process.
(a) Realizations of Xn, Yn, and Zn are shown in Figure 1. (b) If Xn is a random step process, realizations are shown in Figure 2. At any time, the process is +1 with probability p or -1 with probability 1 − p. (c) If Xn is a Bernoulli random process, the mean and variance of Yn are
E(Yn) =
2 E(Xn) +
2 E(Xn−^1 ) =
2 p^ +
2 p^ =^ p
E(Y (^) n^2 ) = E
X n^2 +^1 2 XnXn− 1 +^1 4 X n^2 − 1
E(X n^2 ) +
E(Xn)E(Xn− 1 ) +
E(X n^2 − 1 )
=
1 · p +
p · p +
1 · p
=
p(1 + p)
var(Yn) = E(Y (^) n^2 ) − (E(Yn))^2 =^1 2 p(1 + p) − p^2 =^1 2 p(1 − p)
The autocovariance of Yn is found by
E(YnYn+1) = E
(Xn + Xn− 1 )^1 2 (Xn+1 + Xn)
E(XnXn+1 + Xn− 1 Xn+1 + XnXn + Xn− 1 Xn)
=
(p^2 + p^2 + p + p^2 )
=
(3p^2 + p)
2
Notice that this differs from the sample mean of Xn by only (^21) n Xn but this difference becomes negligible when n is large. Hence the sample mean of Yn will be close to the sample mean of Xn, which in turn will be close to its mean p. By implication, the sample mean of Yn will be close to p which is also the mean of Yn. By a similar approach, the sample mean of Zn can be shown to be close to the mean of Zn which is p.
(d) If Xn is the random step process, then E(Xn) = p − (1 − p) = 2p − 1 and E(X n^2 ) = p + (1 − p) = 1 and E(XnXn+k) = 1 · p^2 − 1 · p(1 − p) − 1 · p(1 − p) + (−1)(−1) · (1 − p)(1 − p) = 4p^2 − 4 p + 1. The mean and variance of Yn are
E(Yn) =
2 E(Xn) +
2 E(Xn−^1 ) = 2p^ −^1
E(Y (^) n^2 ) = E
X n^2 +
XnXn− 1 +
X n^2 − 1
=^1
E(X n^2 ) +^1 2 E(Xn)E(Xn− 1 ) +^1 4 E(X n^2 − 1 )
=^1 4
+^1
(4p^2 − 4 p + 1) +^1 4 = 2p^2 − 2 p + 1 var(Yn) = E(Y (^) n^2 ) − (E(Yn))^2 = 2p^2 − 2 p + 1 − (2p − 1)^2 = 2p(1 − p)
The autocovariance of Yn is found by
E(YnYn+1) = E
(Xn + Xn− 1 )
(Xn+1 + Xn)
4 E(XnXn+1^ +^ Xn−^1 Xn+1^ +^ XnXn^ +^ Xn−^1 Xn)
4 (4p
(^2) − 4 p + 1 + 4p (^2) − 4 p + 1 + 1 + 4p (^2) − 4 p + 1)
= 3p^2 − 3 p + 1
E(YnYn+k) = E
(Xn + Xn− 1 )
(Xn+k + Xn+k− 1 )
, k > 1
=^1 4 E(XnXn+k + Xn− 1 Xn+k + XnXn+k− 1 + Xn− 1 Xn+k− 1 )
=^1 4 4(4p^2 − 4 p + 1) = 4p^2 − 4 p + 1
CY (n, n + 1) = E(YnYn+1) − E(Yn)E(Yn+1) = 3p^2 − 3 p + 1 − (2p − 1)^2 = p(1 − p) CY (n, n + k) = E(YnYn+k) − E(Yn)E(Yn+k) , k > 1 = 4p^2 − 4 p + 1 − (2p − 1)^2 = 0
Similarly, the mean and variance of Zn are
E(Zn) =
E(Xn) +
E(Xn− 1 ) =
(2p − 1) +
(2p − 1) = 2p − 1
E(Z n^2 ) = E
X n^2 +^4 9 XnXn− 1 +^1 9 X^2 n− 1
(4p^2 − 4 p + 1) +
p^2 −
p + 1
4
var(Zn) = E(Z^2 n) − (E(Zn))^2 =
p^2 −
p + 1 − (2p − 1)^2 =
p(1 − p)
The autocovariance of Zn is found by
E(ZnZn+1) = E
3 Xn^ +
3 Xn−^1
3 Xn+1^ +
3 Xn
= E
9 XnXn+1^ +
9 Xn−^1 Xn+1^ +
9 XnXn^ +
9 Xn−^1 Xn
(4p^2 − 4 p + 1) +
(4p^2 − 4 p + 1) +
(4p^2 − 4 p + 1)
=
p^2 −
p + 1
E(ZnZn+k) = E
Xn +
Xn− 1
Xn+k +
Xn+k− 1
, k > 1
= E
XnXn+k +
Xn− 1 Xn+k +
XnXn+k− 1 +
Xn− 1 Xn+k− 1
(4p^2 − 4 p + 1) = 4p^2 − 4 p + 1
CZ (n, n + 1) = E(ZnZn+1) − E(Zn)E(Zn+1)
p^2 −
p + 1 − (2p − 1)^2 =
p(1 − p) CZ (n, n + k) = E(ZnZn+k) − E(Zn)E(Zn+k) , k > 1 = 4p^2 − 4 p + 1 − (2p − 1)^2 = 0
Problem 4. textbook problem 6. Find the pdf of the processes defined in Problem 3 if the Xn are an iid sequence of zero-mean, unit-variance Gaussian random variables.
If the Xn are an iid Gaussian random variables, then Yn and Zn are linear combinations of independent Gaussians so Yn and Zn are Gaussian processes (i.e., any point in the processes is a Gaussian random variable). We know that the Gaussian pdf has two parameters, the mean and variance, so it is sufficient to find the mean and variance of Yn and Zn.
E(Yn) =
E(Xn) +
E(Xn− 1 ) =
E(Zn) =
E(Xn) +
E(Xn− 1 ) = 0
var(Yn) = E(Y (^) n^2 ) =
4 E(X
n^2 ) +^1 2 E(Xn)E(Xn−^1 ) +
4 E(X
(^2) n− 1 )
=
4 ·^ 1 +
2 ·^ 0 +
4 ·^ 1 =
var(Zn) = E
X n^2 +
XnXn− 1 +
X n^2 − 1
=^4
· 1 +^4
· 0 +^1
· 1 =^5
Problem 5. textbook problem 6. Suppose that a secretary receives calls that arrive according to a Poisson process with a rate of 10 calls per hour. What is the probability that no calls go unanswered if the secretary is away from the office for the first and last 15 minutes of an hour?
5
(a) The number of noise impulses during the message has a Poisson PMF with mean λt. According to the Poisson PMF, the probability of 0 impulses is e−λt. (b) A message is error-free or correctable if the number of impulses is 0 or 1. According to the Poisson PMF,
P (0 or 1 impulses) = P (0 impulse) + P (1 impulse) = e−λt^ + λte−λt
Problem 8. textbook problem 6. Messages arrive at a computer from 2 telephone lines according to independent Poisson processes with rates λ 1 and λ 2 , respectively. (a) Find the probability that a message arrives first on line 1. (b) Find the pdf for the time until a message arrives on either line. (c) Find the PMF for N (t), the total number of messages that arrive in an interval of length t. (d) Generalize the result of part (c) for the “merging” of k independent Poisson processes of rate λ 1 ,... , λk, respectively:
N (t) = N 1 (t) + · · · + Nk(t)
(a) The easiest approach is to use conditional probabilities. Starting at time 0, let T 1 and T 2 denote the waiting time until the next arrival on line 1 and 2, respectively. For the Poisson process, we know that T 1 and T 2 are exponential random variables with mean (^) λ^11 and (^) λ^12 , respectively. A message arrives first on line 1 if T 1 < T 2. Given T 2 = x, the conditional probability that T 1 < T 2 is
P (T 1 < T 2 |T 2 = x) =
∫ (^) x
0
λ 1 e−λ^1 tdt = 1 − e−λ^1 x
The conditional probability can be unconditioned by the exponential pdf for T 2 :
P (T 1 < T 2 ) =
0
(1 − e−λ^1 x)λ 2 e−λ^2 xdx
= 1 − λ 2
0
e−(λ^1 +λ^2 )xdx
= 1 − λ 2 λ 1 + λ 2 = λ 1 λ 1 + λ 2
(b) The time T until the first message arrives on either line is T = min(T 1 , T 2 ), i.e., the earlier of line 1 or line 2. The PDF for T is P (T ≤ t) = P (min(T 1 , T 2 ) ≤ t) = 1 − P (T 1 > t, T 2 > t) = 1 − P (T 1 > t)P (T 2 > t) = 1 − e−λ^1 te−λ^2 t^ = 1 − e−(λ^1 +λ^2 )t
The pdf is found by taking the derivative of the PDF which yields
pT (t) = (λ 1 + λ 2 )e−(λ^1 +λ^2 )t
It can be seen that T is exponentially distributed. (c) The exponential pdf found in part (b) implies that arrivals on either line is a Poisson arrival process with rate λ 1 + λ 2. Indeed, this is a general property of Poisson processes. The sum of two independent Poisson processes with rates λ 1 and λ 2 is a Poisson process with rate λ 1 + λ 2. Hence the total number of messages, N (t), will be a Poisson random variable with mean (λ 1 + λ 2 )t. (d) The sum of k independent Poisson processes of rate λ 1 ,... , λk, will be a Poisson process with rate λ 1 +· · ·+λk. The total number of messages, N (t) will be a Poisson random variable with mean (λ 1 +· · ·+λk)t.
7
Problem 9. textbook problem 6. Messages arrive at a message center according to a Poisson process of rate λ. Every hour the messages that have arrived during the previous hour are forwarded to their destination. Find the mean of the total time waited by all the messages that arrive during the hour. Hint: condition on the number of arrivals.
Given k arrivals during the hour, there will be k arrival times, T 1 ,... , Tk. The Poisson process has the property that these arrival times will be uniformly distributed over the time interval. If a message arrives at time Tj , it will wait 60 − Tj minutes. The conditional mean total time waited by all messages is
E
∑^ k j=
(60 − Tj )|k arrivals
∑^ k j=
E(60 − Tj ) = k
0
(60 − T )^1
dT = 30k
The number of arrivals during the hour, k, is Poisson. Unconditioning the conditional mean, the mean total time waited by all messages is ∑∞
k=
30 k (λ · 1)k k! e
−λ· (^1) = 30λ
Problem 10. textbook problem 6. Let Y (t) = X(t) + μt where X(t) is the Wiener process. (a) Find the pdf of Y (t). (b) Find the joint pdf of Y (t) and Y (t + s).
(a) Suppose the Wiener process has parameters 0 and σ^2 , meaning that X(t) is Gaussian with 0 mean and variance σ^2 t. The PDF of Y (t) can be found by
P (Y (t) ≤ y) = P (X(t) + μt ≤ y) = P (X(t) ≤ y − μt)
= P
√X(t) σ^2 t
≤ y √^ −^ μt σ^2 t
y √ − μt σ^2 t
This identifies Y (t) as a Gaussian with mean μt and variance σ^2 t. The pdf of Y (t) can be found by differentiating the PDF above, or simply referring to the Gaussian pdf:
fY (t)(y) = √^1 2 πσ^2 t
e−(y−μt) (^2) / 2 σ (^2) t
(b) The joint pdf can be calculated in different ways. One way is to calculate the joint PDF of Y (t) and Y (t + s) and then differentiate. Another way is to note that the Wiener process has independent Gaussian increments. If Y (t) = i, the increment to Y (t+s) = j is Y (t+s)−Y (t) = j −i = X(t+s)+μ(t+s)−X(t)−μt. This means that X(t) makes an increment of X(t + s) − X(t) = j − i − μs. The joint pdf can now be written as the product of the Gaussian pdf of Y (t) and the Gaussian pdf of the increment of X(t + s) − X(t):
fY (t)Y (t+s)(y 1 , y 2 ) = √^1 2 πσ^2 t
e−(y^1 −μt) (^2) / 2 σ (^2) t 1 √ 2 πσ^2 s
e−(y^2 −y^1 −μs) (^2) / 2 σ (^2) s
Problem 11. textbook problem 6. Let Z(t) = X(t) + X(t − s) where X(t) is the Wiener process. (a) Find the pdf of Z(t). (b) Find the mean mZ (t) and autocovariance function CZ (t 1 , t 2 ).
8
In summary, Y (t) is a Gaussian process with mean (1 − a)μ and variance (1 + a^2 )CX (0) − 2 aCX (s).
Problem 13. Poisson Process For a Poisson counting process N (t) with mean rate λ, show that the autocovariance function is CN (s, t) = λ min(s, t).
Supposing t > s, the autocovariance is
CN (s, t) = E((N (s) − E(N (s)))(N (t) − E(N (t)))) = E((N (s) − λs)(N (t) − λt)) = E(N (s)N (t) − λsN (t) − λtN (s) + λ^2 st) = E(N (s)(N (s) + N (t) − N (s))) − λ^2 st = E(N (s)N (s)) + E(N (s))E(N (t) − N (s)) − λ^2 st = λs + λ^2 s^2 + (λs)(λ(t − s)) − λ^2 st = λs = λ min(s, t)
Problem 14. Poisson Process (optional for EE 5375) Show that the sum of two independent Poisson processes with mean rates λ 1 and λ 2 , respectively, is a Poisson process with mean rate λ 1 + λ 2. Hint: show that the interarrival time for the new process is exponentially distributed.
Starting from time 0, let T 1 be the time of the next arrival in the first Poisson process and T 2 be the time of the next arrival in the second Poisson process. In the combined process, the time T until the next arrival is the earlier of T 1 or T 2 , i.e., T = min(T 1 , T 2 ). The PDF for T is
P (T ≤ t) = P (min(T 1 , T 2 ) ≤ t) = 1 − P (T 1 > t, T 2 > t) = 1 − P (T 1 > t)P (T 2 > t) = 1 − e−λ^1 te−λ^2 t^ = 1 − e−(λ^1 +λ^2 )t
The pdf is found by taking the derivative of the PDF which yields
pT (t) = (λ 1 + λ 2 )e−(λ^1 +λ^2 )t
It can be seen that T is exponentially distributed with mean (^) λ 1 +^1 λ 2 and implies that the combined process is Poisson with rate λ 1 + λ 2.
Problem 15. Matlab (optional for EE 5375) This exercise will generate a first-order autoregressive process Yn = αYn− 1 + Xn where the Xn are iid standard normal random variables. Generate 100 N (0, 1) random samples by typing: > x = normrnd(0,1,100,1); The Xn can be viewed by a stem plot: > stem(x) Now the autoregressive process Yn will be derived from the Xn. First, choose a value of α within the range (0,1). For example, if you choose α = 0.5, type: > alpha = 0.5; Next, initialize the process by typing: > y(1) = x(1); Generate the remainder of the sequence by typing: > for n=2:100 y(n) = (alpha * y(n-1)) + x(n); end; The Yn can be viewed by a stem plot:
10
> stem(y) If a regression is done on the Yn, then the estimate of α should be close to the value of α originally chosen. Estimate α by typing: > alphahat = (y(1:99) * y(2:100)’)/(y(1:99) * y(1:99)’) Compare the result with the value of α originally chosen; are they close? Try with a different number of samples (e.g., more than 100).
