Download Homework 9: Autocorrelation & Markov Chains - Prof. Majeed M. Hayat and more Assignments Probability and Statistics in PDF only on Docsity!
ECE 541, Probability and Stochastic Processes Fall 2008 Homework Assignment 9 Due date: Thursday, Dec. 4, 2008
Problem 1. Consider the Gauss-Markov model
Bn = ρBn− 1 + W (n), n > 0 , 0 < |ρ| < 1 ,
where the initial state B 0 is a zero-mean Gaussian random variable with variance σ^2 B 0. Assume that W (1), W (2),... , are iid, Gaussian, zero mean, and independent of B 0. Let σ^2 W denote the variance of W (n). Derive a difference equation characterizing the autocorrelation function RB (m+n, n) =^4 E[Bn+mBn]. Solve the equation and comment on your result. Be sure to specify any initial conditions needed. Hint: First find a recursion for RB (n, n), and then find a recursion for RB (m+n, n) (in the variable m) for each fixed n.
Solution: For m ≥ 1, we write
RB (m + n, n) = E[Bn+mBn] = E[(ρBn+m− 1 + W (n + m))Bn] = E[ρBn+m− 1 Bn + W (n + m)Bn] = ρE[Bn+m− 1 Bn] = ρRB (m + n − 1 , n)
since W (n + m) and Bn are independent. (Why?) Hence, by repeating the above step, we obtain
RB (m + n, n) = ρmRB (n, n).
Moreover,
RB (n, n) = E[B n^2 ] = E[(ρBn− 1 + W (n))^2 ] = E[ρ^2 B^2 n− 1 + 2ρBn− 1 W (n) + W (n)^2 ] = ρ^2 E[B^2 n− 1 ] + 2ρE[Bn− 1 W (n)] + E[W (n)^2 ] = ρ^2 RB (n − 1 , n − 1) + σ^2 W
and we obtain
RB (n, n) = ρ^2 nRB (0, 0) = ρ^2 nE[B 02 ] + σ W^2
1 − ρ^2 n 1 − ρ^2 .
Finally, for m ≥ 1, m ≥ 1, we have
RB (m + n, n) = ρm
( ρ^2 nE[B^20 ] + σ W^2 1 − ρ^2 n 1 − ρ^2
) .
Note that if σ W^2 1 −^1 ρ 2 = E[B^20 ], then RB (m + n, n) becomes a function of m, independent of n.
Problem 2. (From Hoel, Port, and Stone) Consider an irreducible birth and death chain on the nonnegative integers with
qi/pi = ( i i + 1
)^2.
Show that the chain is transient.
Solution: For irreducible birth and death chain, the states are transient if and only if
∑^ ∞ m=
γm < ∞.
Since
γm =
∏^ m
i=
qi pi =
∏^ m
i=
( i i + 1 )^2 = ( 1 m + 1 )^2
we have (^) ∞ ∑ m=
γm =
∑^ ∞ m=
( 1 m + 1
)^2 = π^2 6
− 1 < ∞
The chain is transient.
Problem 3. (From Hoel, Port, and Stone) Consider a Markov chain on the nonnegative integers having transition probabilities given by pi,i+1 = p and pi, 0 = 1 − p, where 0 < p < 1. Show that this chain has a unique stationary distribution π and find it.
Solution: ∀i, j, ∃n, such that pnij > 0, therefore the Markov chain is irreducible. Since the probability to be in origin at time n is positive for all integer values of n, the state 0 has period 1. Since the Markov chain is irreducible, so does each state x > 0. Therefore, the Markov chain is aperiodic. Let π = [π 0 π 1.. .],
IP =
1 − p p 0 0 0... 1 − p 0 p 0 0... 1 − p 0 0 p 0... .. .
.. .
.. .
.. .
.. .
...
Use the equation π = πIP to obtain πk = pkπ 0 , k ≥ 1. Since ∑∞ k=0 πk^ = 1, the first scalar equation in π = πIP yields π 0 = 1 − p. This stationary distribution is unique by the relevant theorem in the notes since the chain is irreducible, aperiodic, and a stationary distribution exists.
Problem 4. There are n nodes in a certain cluster of computers that are initially operating at time k = 0. During each unit of time, each node may fail with probability p (independently of previous times and other processors) and remains dysfunctional from that point and on. For k = 0, 1 , 2 ,.. .,
Solution
a) The number of surviving nodes at time k depends only on the number of surviving nodes at time k -1, as such the number of surviving nodes is a Markov chain with n + states. The transition matrix is described as follows:
P 00 =1, P 0 i = 0, i =1,2, …, n ,
Pij = 0 if j > i ,
Pij = (1 ) j^ i^ j ,.
i p p j i n j
b) Let mk = E[ Nk ] (i.e., the mean sequence subject to N 0 = n , the initial condition given in the problem). The idea is to use that fact that E[ Nk ]= E[E[ N (^) k | Nk -1 ]], where inner conditional expectation can be computed as follows: E[ Nk | N (^) k -1 = i ] = (1- p ) i , since each PC is likely to survive one time transition with probability 1- p. We now take the expectation of the above equation and obtain E[ Nk ] = (1- p ) E[ Nk -1 ], or
mk = (1- p ) mk -1 , k >0, with initial condition m 0 = n.
Hence, mk = n (1- p ) k^.
c) Let r (^) k = E[ Nk^2 ]. Note that E[ Nk^2 ]= E[E[ Nk^2 | Nk -1 ]], and E[ Nk^2 | Nk -1 = i ] = p (1- p ) i +(1- p ) 2 i^2. (The last assertion follows from the fact that E[ Nk^2 | Nk -1 = i ] is equal to the second moment of a Bernoulli (i.e., {0,1}-valued) random variable of size i and success probability (1- p ).) Thus, by taking the expectation of the above equation we obtain E[ Nk^2 ] = p (1- p ) E[ Nk -1 ] + (1- p )^2 E[ Nk -1^2 ], or
r (^) k = p (1- p ) mk -1 + (1- p ) 2 r (^) k -1 , k >0, with initial condition r 0 = n^2.
The homogeneous solution to the above equation is of the form A (1- p ) 2 k , and the particular solution is n (1- p ) k^. Hence, the total solution is r (^) k = A (1- p ) k^ + n (1- p ) k. Now we apply the initial condition to obtain rk =( n^2 -n ) (1- p ) 2 k^ + n (1- p ) k.
The variance of Nk is therefore r (^) k - mk^2 = n (1- p ) k [1-(1- p ) k ].
d+e) This is the idea of the calculation: Let T be the exact time when all PCs fail. Then, Nk = 0 if and only if T = k. Now, P{ T = k } = P({ Nk -1 > 0} ∩ { Nk = 0}) = P([{ Nk -1 = 1} ∪ ... ∪ ([{ Nk -1 = n }] ∩ { Nk = 0}) = P([{ Nk -1 = 1}∩ { Nk = 0}] ∪ ... ∪ [{ Nk -1 = n } ∩ { Nk = 0}])
= 1 P { N 0 | N 1 j } P { Nk 1 j }.
n ∑ j = k =^ k − = − =
Now P{ N (^) k = 0 | Nk -1 = i } = pi , and P{ Nk -1 = i }= [( 1 p )(^ k^^1 )] i [ 1 ( 1 p )( k^1 )] ni , i
n (^) − − − ⎟⎟ − − − ⎠
where
in the last expression we realized that each PC remains functional from the beginning to time ( k -1) with probability (1- p ) k -1^. Thus,
{ } ∑ = 1 ⎟⎟[( 1 − )(−^1 )][ 1 −( 1 − )(−^1 )]−.
n i p k^ i p k nipi i
n P T k This expression can be further
simplified to
{ } ( 1 ( 1 )(^1 )) (( 1 )(^1 )[ 1 ] 1 ).
k n k n P T = k =− − − p − + − p − p − +
(To see the above simplification, observe that if B is a binomial rv with size n and success probability Q , then E[ aB ] = (1- Q + aQ ) n. To see this, write B as a sum of independent Bernoulli rv’s and exploit their independence in simplifying the expectation.)
Using Matlab, we can numerically calculate the sum ∑
∞
=
1
i
iP T i using the result in part
(e) and obtain E[ T ] = 4497.5 as the average time to total failure.
Note that E[ T ] = E[E[ T | N 1 ]]. Let us now calculate the conditional expectation E[ T | N 1 = i ]. To do so, first define Tk as the time to total failure given that we start with k PCs. For example, T 50 = T. So, we calculate E[ Tk | N 1 = i ] as follows: E[ T (^) k | N 1 = 0] = 1, and more generally, for i >0, E[ Tk | N 1 = i ] = 1+ Ti. This is because if i PCs survive in the first time increment, then for us to loose the remaining i PCs in Tk steps is equivalent to reaching total failure given that we start, just after the first step, with i PCs. We have to add the one because it had already taken us one unit of time to get to the first step. We now average over all possible realizations of N 1 and obtain: E[ Tk ] = E[E[ Tk | N 1 ]] = E[1 + TN 1 ]. If we define ak = E[ Tk ], then we can carry out the averaging and obtain
ak = 1 + ∑
=
k
j
aj Pk N j 0
where P k { N 1 = i }= pi^ pki i
k (^) − ⎟⎟ − ⎠
( 1 ) (from part (c)). If we use the initial condition a 1 = 1/ p
(prove this), we can calculate a 2 , a 3 , etc. Note that a 0 = 0 by definition. Also note that the
k value on the left-hand side of Eq.(1) is the same as the one used in pi^ pki i
k (^) − ⎟⎟ − ⎠
( 1 ). As
we move from k to k +1, we modify p i^ pki i
k (^) − ⎟⎟ − ⎠
( 1 ) to pi^ pk i i
k (^) +− ⎟⎟ − ⎠
⎛ +^1
. For example,
we calculate a 2 as follows:
a 2 = 1+ ∑
=
2
0
j
a (^) j Pk N j = 1+ a 0 ( 1 )^02 0
⎟⎟ − p p ⎠
⎟⎟ − p p ⎠
⎟⎟ − p p ⎠
With a 1 = 1/ p and a 0 = 0, we can solve for a 2 , and so on.
0 0.5 1 1.5 2 x 10 4
0
2
4
6
8
10
12
14
k
var[N
]k^
experimental theoretical
clear all close all n=50; K=20000; %(length of PMF P{T=k} array) I=10; k=[2:1:K]; p=.001; q=1-p; M=2000; %number of experiments N=ones(1,n); %initial condition: all PCs are up N_cum=zeros(1,K); N_cum2=zeros(1,K); N_i=zeros(1,K); P(k)=((q.^(k-1)).*(p-1) + 1).^n - (1-q.^(k-1)).^n; P(1)=p^n;
for j=1:M N=ones(1,n); %initial condition: all PCs are up for i=2:K X=(rand(1,n)>p); N=X.*N; N_i(i)=sum(N); N_cum(i)=N_cum(i)+N_i(i); N_cum2(i)=N_cum2(i)+N_i(i)^2; if sum(N)==
T(j)=i; break end end end
N_ave=N_cum/M; N_ave(1)=n; varN_ave=N_cum2/M - N_ave.^2; varN_ave(1)=0; T_ave=mean(T) mk=n(1-p).^([1:K]-1); vk=n( (q.^([1:K]-1)).*(1-q.^([1:K]-1)) );
figure(1) plot([1:K],N_ave,'r:',[1:K],mk,'b:') p=gca; set(p,'fontsize',16) xlabel('k','fontsize',16) ylabel('E[N_k]','fontsize',16) legend('experimental','theoretical')
figure(2) plot([1:K],varN_ave,'r:',[1:K],vk,'b:') xlabel('k','fontsize',16) ylabel('var[N_k]','fontsize',16) legend('experimental','theoretical') p=gca; set(p,'fontsize',16)
f) State 1 is not reachable from state 0 and the chain is therefore not irreducible.
g) From the very definition of recurrence, we note that state 0 is recurrent because for any k , P{ Nk =0| N 0 =0}=1, so the probability that 0 is revisited at some point is trivially 1.
On the other hand, all other states are transient. This is because for i =1, …, n , P{ Nk = i |
N 0 = i }=(1- p ) ki^ , and therefore (^0) 1
{ (^) k | } k
P N i N i
∞
=
∑ =^ =^ < ∞. So by the Borel-Cantelli Lemma,
the probability that state i is revisited infinitely often is 0.
