Download Stochastic Processes - Homework 2 Solutions | MATH 285 and more Assignments Stochastic Processes in PDF only on Docsity!
Homework 2
Troy Kravitz
April 11, 2007
Problem 1.1.
Suppose that {Xm} is Markov (λ, P ). If Yn = Xkn, show that {Yn} is Markov (λ, P k).
We first verify that {Xkn} is Markov. (As the steps progress, we’ll abbreviate notation, but no confusion should result.)
P (Xk(n+1) = ik(n+1)|Xk(0) = ik(0), ..., Xk(n−1) = ik(n−1), Xk(n) = ik(n))
=
P (Xk(n+1), Xk(0) = ik(0), ..., Xk(n−1) = ik(n−1), Xk(n) = ik(n)) P (Xk(0) = ik(0), ..., Xk(n−1) = ik(n−1), Xk(n) = ik(n))
=
P (Xk(n+1), Xk(n)|Xk(0), ..., Xk(n−1)) · P (Xk(0), ..., Xk(n−1)) P (Xk(0), ..., Xk(n−1), Xk(n))
=
P (Xk(n+1), Xk(n)|Xk(0), ..., Xk(n−1)) · P (Xk(0), ..., Xk(n−1)) P (Xk(n)|Xk(0), ..., Xk(n−1)) · P (Xk(0), ..., Xk(n−1))
=
P (Xk(n+1), Xk(n)|Xk(0), ..., Xk(n−1)) P (Xk(n)|Xk(0), ..., Xk(n−1))
=
P (Xk(n+1)|Xk(n)) · P (Xk(n)|Xk(0), ..., Xk(n−1)) P (Xk(n)|Xk(0), ..., Xk(n−1)) = P (Xk(n+1)|Xk(n))
Thus, {Xkn} is Markov. The initial probability distribution remains λ as we are not conditioning on a later realization. The transition matrix, P , is given by Theorem 1.1.3, (ii): P (Xn+m = j|Xm = i) = p( ijn ). Letting m = kn and n = k, so (n + 1)k = kn + k = m + n and nk = m gives the result. Thus, {Xkn} is Markov(λ, P (k)).
Problem 1.1.3, part B
Suppose that Z 0 , Z 1 , ... are iid random variables such that Zi = 1 with probability p and Zi = 0 with probability (1-p). Set S 0 = 0, Sn = Z 1 + ... + Zn. In each of the following cases, determine whether {Xn} is Markov. In the cases where {Xn} is Markov, find it’s state-space and transition matrix, and in the cases where it is not Markov, give an example where P (Xn+1 = i|Xn = k) is not independent of k.
(A) Xn = Zn: This is a Markov chain with state-space S = { 0 , 1 } and transition matrix
P =
[
p 1 − p p 1 − p
]
(B) Xn = Sn: This is also Markov, but with state space S = { 0 , 1 , ...} = N and transition matrix
P =
p 1 − p 0 ... ... ... 0 p 1 − p 0 ... ... 0 0 p 1 − p 0 ... ... ... ... ... ... ...
(C) Xn = S 0 + ... + Sn: This is not Markov. Merely knowing the state of the system is insufficient to fully specify the transition probabilities, as the following example will demonstrate. (Note that the additional information needed is the current index.) Compare the following two set-ups:
Case 1 =
Z : 1 1 1
S : 1 2 3
X : 1 3 6
(^) Case 2 =
Z : 1 0 1 0
S : 1 1 2 2
X : 1 2 4 6
In case 1, given that Xn− 1 = 6, the probability of Xn = 9 is 1−p (corresponding to Zn = 0) and the probability of Xn = 10 is p (corresponding to Zn = 1). All other probabilities are zero. In case 2, on the other hand, we also have Xm− 1 = 6. But now the probabilty of Xm = 8 is 1 − p (corresponding to Zm = 0) and the probability of Xm = 9 is p (corresponding to Zm = 1). Thus, merely being given that Xi = ii is insufficient to fully specify the transition probabilities.
(D) Xn = (Sn, S 0 + ... + Sn): This example is Markov. (We have sufficient information to distinguish this example from example (C).) The state-space will be the subset of N^2 that lies weakly above the 45 degree line. This space will contain vectors of the form: (0, 0), (1, m 1 ), (2, m 2 ), (3, m 2 ), ... where mi ≥ i. More generally, S = {(Sn, mn) : Sn ∈ N and mn ≥ Sn}. The transition probabilities from (Sn, mn) are: with probability p to (Sn, 2 · mn) and with probability 1 − p to (Sn + 1, 2 · mn + 1).
Someone suggests that the record of successive choices (a sequence of As and Bs) might arise from a two-state Markov chain with con- stant transition probabilities. Discuss, with reference to the value of Pn+1(A) that you have found, whether this is possible.
I believe that this is correct. The calculations for finding Pn+1(A) above relied upon Bayes’ law and a few simple identities, namely that the probability of being in either state 1, state 2, or state 3 must be unity. If there were only two states, the steps above would still apply with minimal changes. Thus, I conclude the record of successive choices could result from a two-state system.
