ECE 541, Probability and Stochastic Processes
Fall 2008
Homework Assignment 9
Due date: Thursday, Dec. 4, 2008
Problem 1. Consider the Gauss-Markov model
Bn=ρBn−1+W(n), n > 0,0<|ρ|<1,
where the initial state B0is a zero-mean Gaussian random variable with variance σ2
B0. Assume
that W(1), W (2), . . . , are iid, Gaussian, zero mean, and independent of B0. Let σ2
Wdenote the
variance of W(n).
Derive a difference equation characterizing the autocorrelation function RB(m+n, n)4
=E[Bn+mBn].
Solve the equation and comment on your result. Be sure to specify any initial conditions needed.
Hint: First find a recursion for RB(n, n),and then find a recursion for RB(m+n, n) (in the variable
m) for each fixed n.
Problem 2. (From Hoel, Port, and Stone) Consider an irreducible birth and death chain on the
nonnegative integers with
qi/pi= ( i
i+ 1)2.
Show that the chain is transient.
Problem 3. (From Hoel, Port, and Stone) Consider a Markov chain on the nonnegative integers
having transition probabilities given by pi,i+1 =pand pi,0= 1 −p, where 0 <p<1. Show that
this chain has a unique stationary distribution πand find it.
Problem 4. There are nnodes in a certain cluster of computers that are initially operating at time
k= 0. During each unit of time, each node may fail with probability p(independently of previous
times and other processors) and remains dysfunctional from that point and on. For k= 0,1,2, . . .,
let Nkdenote the number of processors that are up at time k.
a) Argue that Nkis a Markov chain and derive its transition probability matrix IP.
b) Use conditional expectations to derive a recursion for E[Nk] and solve for E[Nk], k≥0.
c) Determine a recursion for the second moment of Nk, specify the initial condition, and solve for
the second moment.
d) Calculate P{Nk= 0}.
e) Let Tbe the first time when all nodes have failed, that is, T= min{i:Ni= 0}. Show that
P{T=k}=³1+[p−1](1 −p)k−1´n−³1−(1 −p)k−1´n.
Hint: {T=k}={Nk−1>0} ∩ {Nk= 0}.
f) Is the chain irreducible? Justify your answer.
g) Which sates are recurrent and which states are transient? Justify your answer.