Download Homework 6 Solution - Stochastic Processes - Spring 2009 | MATH 285 and more Assignments Stochastic Processes in PDF only on Docsity!
Math 285, Spring 2009 Homework 6 Solutions
5.8. For this problem Sn = a + X 1 + · · · + Xn, where the Xk are iid with P[Xk = 1] = p ∈ (0, 1 /2) and P[Xk = −1] = 1 − p =: q. Notice that E[Xk] = 2p − 1. (a) Evidently
E[Sn|Fn− 1 ] = Sn− 1 + E[Xn|Fn− 1 ] = Sn− 1 + E[Xn] = Sn− 1 + (2p − 1).
Therefore
E[Mn|Fn− 1 ] = E[Sn − n(2p − 1)|Fn− 1 ] = Sn− 1 + (2p − 1) − n(2p − 1) = Sn− 1 − (n − 1)(2p − 1) = Mn− 1 ,
as required. (b) By the elementary inequality (A + B)^2 ≤ 2 A^2 + 2B^2 , we have
E[Z n^2 ] ≤ 2 E[S T^2 ∧n] + 2(2p − 1)^2 E[(T ∧ n)^2 ].
But |ST ∧n| ≤ N , so the first expectation on the right side of the above display is at most N 2. On the other hand, E[(T ∧ n)^2 ] ≤ E[T 2 ], which is finite because of Exercise 1.7. Indeed that exercise tells us that there are constants C > 0 and 0 < ρ < 1 such that
P[T > n] ≤ Cρn, n = 0, 1 , 2 ,....
Let’s now use the tail formula for E[T 2 ]:
E[T 2 ] =
0
2 tP[T > t] dt =
∑^ ∞
n=
∫ (^) n+
n
2 tP[T > t] dt
∑^ ∞
n=
2(n + 1)P[T > n]
≤ 2 C
∑^ ∞
n=
(n + 1)ρn^ < ∞.
(This final infinite series is convergent by the Ratio Test, for example.) Putting things together we find that E[Z n^2 ] ≤ 2 N 2 + 2(2p − 1)^2 E[T 2 ], ∀n.
(c) In view of part (b), the stopped martingale Mn∧T is bounded in L^2 , hence uniformly integrable. Therefore, a = E[M 0 ] = E[MT ] = E[ST + (1 − 2 p)T ].
But, by the result of Exercise 5.7,
P[ST = 0] =
ρa^ − ρN 1 − ρN^
where ρ = (1 − p)/p. Therefore P[ST = N ] = 1 −ρ
a 1 −ρN^ , so
E[ST ] = N · P[ST = N ] = N
1 − ρa 1 − ρN^
Consequently,
E[T ] = (1 − 2 p)−^1 {a − E[ST ]} = (1 − 2 p)−^1
a − N 1 − ρa 1 − ρN
5.9. (As already noted, the condition on the function g should read P f (x) − f (x) = g(x) for x ∈ S \ A. Also, in the formula for Mn, the expression g(XTj ) coincides with g(Xj ), because j < Tn = min(T, n) ≤ T , so Tj = min(T, j) = j.) Because the state space S is finite and the Markov chain is irreducible, it is recurrent as well. In particular, Px[T < ∞] = 1 for all x ∈ S. (a) It was noted in class that if we define h = P f − f then the “compensated” process
Kn := f (Xn) −
n∑− 1
j=
h(Xj ), n = 0, 1 , 2 ,...
is a martingale. Consequently the stopped process
KTn = f (XTn ) −
T∑n− 1
j=
h(Xj ), n = 0, 1 , 2 ,... ,
is also a martingale. But if j = 0, 1 ,... Tn − 1 then Xj must lie in S \ A (the set A has not yet been hit before time Tn), so h(Xj ) = g(Xj ) because P f (x) − f (x) = g(x) for all x ∈ S \ A. In other words, Mn = KTn , and we conclude that (Mn) is a martingale. (b) By Optional Stopping,
f (x) = Ex[Mn] = Ex[f (XTn )] − Ex
T∑n − 1
j=
g(Xj )
Since the state space S is finite the function f is bounded, so the sequence f (XTn ) converges boundedly to F (XT ) (because XT ∈ A and f = F on A) , and the first expectation on the
right above converges to Ex[F (XT )]. The second expectation converges to Ex
[∑
T − 1 j=0 g(Xj^ )
]
by
dominated convergence because ∣ ∣ ∣∣ ∣∣
T∑n − 1
j=
g(Xj )
∣∣ ≤^ T^ ·^ L,^ ∀n,
and because Ex[T ] < ∞ by Exercise 1.7. (Here L := max{|g(x)| : x ∈ S \ A}.) We conclude that
f (x) = Ex[f (XT )] − Ex
T∑ − 1
j=
g(Xj )
