ECE 534 RANDOM PROCESSES FALL 2008
PROBLEM SET 5 Due Wednesday, November 5
5. Two faces of Markov processes–inference and dynamics
Assigned Reading: Sections 4.9 and 4.10, Chapter 5, and Chapter 6 through section 8.
Problems to be handed in:
1 A two station pipeline in continuous time
This is a continuous-time version of Example 4.9.1. Consider a pipeline consisting of two single-
buffer stages in series. Model the system as a continuous-time Markov process. Suppose new packets
are offered to the first stage according to a rate λPoisson process. A new packet is accepted at
stage one if the buffer in stage one is empty at the time of arrival. Otherwise the new packet is
lost. If at a fixed time tthere is a packet in stage one and no packet in stage two, then the packet is
transfered during [t, t+h) to stage two with probability hµ1+o(h). Similarly, if at time tthe second
stage has a packet, then the packet leaves the system during [t, t +h) with probability hµ2+o(h),
independently of the state of stage one. Finally, the probability of two or more arrival, transfer, or
departure events during [t, t +h) is o(h). (a) What is an appropriate state-space for this model?
(b) Sketch a transition rate diagram. (c) Write down the Qmatrix. (d) Derive the throughput,
assuming that λ=µ1=µ2= 1.
2 A variance estimation problem with Poisson observation
The input voltage to an optical device is Xand the number of photons observed at a detector is
N. Suppose Xis a Gaussian random variable with mean zero and variance σ2, and that given
X, the random variable Nhas the Poisson distribution with mean X2.(Recall that the Poisson
distribution with mean λhas probability mass function λne−λ/n! for n≥0.)
(a) Express P[N=n|σ2] as an integral. You do not have to perform the integration.
(b) Find the maximum likelihood estimator of σ2given N. (Caution: Estimate σ2, not X. Be as
explicit as possible–the final answer has a simple form. Hint: You can first simplify your answer to
part (a) by using the fact that if Xis a N(0,eσ2) random variable, then E[X2n] =
eσ2n(2n)!
n!2n. )
3 Finding a most likely path
Consider an HMM with state space S={0,1}, observation space {0,1,2},and parameter
θ= (π, A, B) given by:
π= (a, a3)A=a a3
a3aB=ca ca2ca3
ca2ca3ca
Here aand care positive constants. Their actual numerical values aren’t important, other than
the fact that a < 1.Find the MAP state sequence for the observation sequence 021201, using the
Viterbi algorithm. Show your work.
4 Sp ecialization of Baum-Welch algorithm for no hidden data
Suppose that the matrix Bis the identity matrix, so that Xt=Ytfor all t. (a) Determine how
the Baum-Welch algorithm simplifies in the special case that Bis the identity matrix, so that
Xt=Ytfor all t. (b) Still assuming that Bis the identity matrix, suppose that S={0,1}and the
observation sequence is 0001110001110001110001. Find the ML estimator for πand A.
1
