Download Solutions to Problem Set 5 of ECE 534 Random Processes, Fall 2008 and more Assignments Electrical and Electronics Engineering in PDF only on Docsity!
ECE 534 RANDOM PROCESSES FALL 2008
SOLUTIONS TO PROBLEM SET 5
1 A two station pipeline in continuous time (a) S = { 00 , 01 , 10 , 11 } (b)
μ
00 01
10 11
! ! (^) μ 1
μ 2
2
(c) Q =
−λ 0 λ 0 μ 2 −μ 2 − λ 0 λ 0 μ 1 −μ 1 0 0 0 μ 2 −μ 2
(d) η = (π 00 + π 01 )λ = (π 01 + π 11 )μ 2 = π 10 μ 1. If λ = μ 1 = μ 2 = 1.0 then π = (0. 2 , 0. 2 , 0. 4 , 0 .2) and η = 0.4.
2 A variance estimation problem with Poisson observation (a)
P {N = n} = E[P [N = n|X]] = E[
(X^2 )ne−X^2 n!
]
−∞
x^2 ne−x 2
n!
e−^
x^2 2 σ^2 √ 2 πσ^2
dx
(b) To arrive at a simple answer, we could set the derivative of P {N = n} with respect to σ^2 equal to zero either before or after simplifying. Here we simplify first, using the fact that if X is a N (0, ˜σ^2 ) random variable, then E[X^2 n] = eσ
2 n(2n)! n!2n^.^ Let ˜σ
(^2) be such that 1 2˜σ^2 = 1 +^
1 2 σ^2 ,^ or equivalently, ˜σ^2 = σ 2 1+2σ^2.^ Then the above integral can be written as follows:
P {N = n} = ˜σ σ
−∞
x^2 n n!
e
−x^2 2 eσ^2 √ 2 π˜σ^2
dx
c 1 σ˜^2 n+ σ
c 1 σ^2 n (1 + 2σ^2 )
2 n+ 2
where the constant c 1 depends on n but not on σ^2. Taking the logarithm of P {N = n} and calcu- lating the derivative with respect to σ^2 , we find that P {N = n} is maximized at σ^2 = n. That is, ̂ σ M L^2 (n) = n.
3 Finding a most likely path Finding the path z to maximize the posterior probability given the sequence 021201 is the same as maximizing pcd(y, z|θ). Due to the form of the parameter θ = (π, A, B), for any path z = (z 1 ,... , z 6 ), pcd(y, z|θ) has the form c^6 ai^ for some i ≥ 0. Similarly, the variable δj (t) has the form ctai^ for some i ≥ 0. Since a < 1, larger values for pcd(y, z|θ) and δj (t) correspond to smaller values of i. Rather than keeping track of products, such as aiaj^ , we keep track of the exponents of the products, which for aiaj^ would be i + j. Thus, the problem at hand is equivalent to finding a path from left to right in trellis indicated in Figure 1(a) with minimum weight, where the weight of a
’s 3 1
1 3 3 1
1 3 3 1
1 3 3 1
1 3 3 1
1 3
3 1
1 3 3 1
1 3 3 1
1 3 3 1
1 3 3 1
1 3
0 2 1 2 0 1
0
1
Observations
State
t=1 t=
3 2 3 1 2
3+2 1 3 1 2 3
1+
(a)
(b) 6 9 13 15 18
6 10 12 15
2
0 2 1 2 0 1
0
1
Observations
State
3 2 3 1 2
3+2 1 3 1 2 3
1+
t=1 (^) t= 6
5 19
!
Figure 1: Trellis diagram for finding a MAP path.
path is the sum of all the numbers indicated on the vertices and edges of the graph. Figure 1(b) shows the result of running the Viterbi algorithm. The value of δj (t) has the form ctai, where for i is indicated by the numbers in boxes. Of the two paths reaching the final states of the trellis, the upper one, namely the path 000000, has the smaller exponent, 18, and therefore, the larger probability, namely c^6 a^18. Therefore, 000000 is the MAP path.
4 Specialization of Baum-Welch algorithm for no hidden data (a) Suppose the sequence y = (y 1 ,... , yT ) is oberserved. If θ(0)^ = θ = (π, A, B) is such that B is the identity matrix, and all entries of π and A are nonzero, then directly by the definitions (without using the α’s and β’s):
γi(t) 4 = P [Zt = i|Y 1 = y 1 ,... , YT = yT , θ] = I{yt=i}
ξij (t) 4 = P [Zt = i, Zt+1 = j|Y 1 = y 1 ,... , YT = yT , θ] = I{(yt,yt+1)=(i,j)}
the Baum-Welch algorithm to be such that aij > 0 if and only if aij = 1 and bil > 0 if and only if bil = 1. (These constraints are added in addition to the usual constraints that π, A, and B have the appropriate dimensions, with π and each row of A and b being probability vectors.) After all, it makes sense for the initial parameter value to respect the constraint. And if it does, then the same constraint will be satisfied after each iteration, and no changes are needed to the algorithm itself.
7 Mean hitting time for a simple Markov process (a)
1
0 1 2 3
1 a 0.
1 !a (^) 0.
Since this is a discrete-time process, the equilibrium distributions are the probability distributions π on the state space satisfying π = πP. For this problem we find that for all a ∈ [0, 1] there is a unique equilibrium distribution, π = ( (^) 2(1+^1 −aa) , (^) 2(1+^1 a) , (^) 2(1+^2 aa) , (^) 2(1+aa) ). Remark: If a = 0 or a = 1 the process is not irreducible in the strong sense used in the notes. Specifically, if a = 0 then states 2 and 3 cannot be reached from states 0 or 1, and if a = 1 then state 0 cannot be reached from states 1, 2, or
- But for any a, there do not exist two disjoint sets of states such that it is impossible to get from either set to the other, so there cannot be more than one equilibrium distribution. (b) A general way to solve this is to let hi = E[min{n ≥ 0 |X(n) = 3}|X(0) = i], for 0 ≤ i ≤ 3. Our goal is to find h 0. Clearly, h 3 = 0. Derive equations for the other values by conditioning on the first step of the pro-
cess: hi = 1 +
j pij^ hj^ for^ i^6 = 3.^ Or
h 0 = 1 + h 1 h 1 = 1 + (1 − a)h 0 + ah 2 h 2 = 1 + (0.5)h 1
yielding
h 0 h 1 h 2
4 a + 1 4 2 a a + 1
Thus, h 0 = (^4) a + 1 is the required answer. (If a = 0 then h 0 = h 1 = h 2 = +∞, because for initial state 0,1, or 2, there is positive probability that state 3 is never reached.)
8 A birth-death process with periodic rates (a) Since the diagonal entries qii are bounded in magnitude, the process is not explosive (by Propo- sition 6.3.7), so the process is positive recurrent if and only if there is a probability distribution π satisfying πQ = 0 (by Proposition 6.3.6 (a)). Let θ = (^) μλaaλμbb. Then S 1 =
n=0 θ
n(1 + λa μa ), which is finite if and only if θ < 1. Thus, the process is positive recurrent if and only if θ < 1. (In case θ < 1, π 2 n = θn/S 1 and π 2 n+1 = θn^ Sλ 1 aμa .) Note: An alternative approach to (a) is to first note that S 2 =
n=0 θ −n(1 + μa λa ) = +∞^ if and only if θ ≤ 1. Therefore, the process is recurrent if and only if θ ≤ 1. In particular, the process is nonexplosive if θ ≤ 1. Then, finding that S 1 < ∞ if and only if θ < 1 implies that the process is positive recurrent if and only if θ < 1. (b) λa = λb. In general, rk = πkλk/λ, so that r ≡ π if and only if the arrival rates are all equal (corresponding to Poission arrivals, and then PASTA holds.)
