Download Solutions Assignment 3 - Stochastic Processes | MATH 285 and more Assignments Stochastic Processes in PDF only on Docsity!
Math 285, Spring 2009 Homework 3 Solutions
2.7. (a) Because P 0 [T 0 > n] = (1/2)(2/3) · · · (n/(n + 1)) = 1/(n + 1), we have
P 0 [T 0 < ∞] = (^) nlim→∞ P 0 [T 0 ≤ n] = (^) nlim→∞ n/(n + 1) = 1,
so this chain is recurrent. By the same token,
E 0 [T 0 ] =
∑^ ∞
n=
P 0 [T 0 > n] =
∑^ ∞
n=
n + 1
so the chain is null recurrent. (b) For this chain, P 0 [T 0 > n] = (1/2)(1/3) · · · (n + 1) ≤ 1 /(n + 1), and so the chain is recurrent just like the chain of part (a). On the other hand, a more careful estimate of P 0 [T 0 > n] is
P 0 [T 0 > n] = (1/2)(1/3) · · · (n + 1) ≤
n(n + 1)
n^2
which is the term of a convergent p-series. The tail sum
E 0 [T 0 ] =
∑^ ∞
n=
P 0 [T 0 > n] ≤
∑^ ∞
n=
n^2
therefore converges, and this chain is positive recurrent. The stationary distribution to be found is the unique probability distribution π such that π = πP. That is,
(2. 7 .1) π(0) =
∑^ ∞
k=
π(k) · k^ + 1 k + 2
and π(n) = π(n − 1) ·
n + 1 , n = 1, 2 ,....
Solving this recursion we arrive at
π(n) = π(0) (n + 1)! , n = 1, 2 ,....
The condition
n=0 π(n) = 1 forces
1 = π(0)
∑^ ∞
n=
(n + 1)! =^ π(0)^ ·^ (e^ −^ 1).
Thus,
π(n) = (e − 1)−^1
(n + 1)! ,^ n^ = 0,^1 ,^2 ,....
This is consistent with the condition (2.7.1):
∑^ ∞ k=
π(k) · k + 1 k + 2 = (e − 1)−^1
∑^ ∞
k=
(k + 1)!
k + 1 k + 2
= (e − 1)−^1
∑^ ∞
k=
(k + 1)!
k + 2
= (e − 1)−^1
∑^ ∞
k=
(k + 1)! −^ (e^ −^ 1)
− 1
∑^ ∞
k=
(k + 2)!
= 1 − e − 2 e − 1
e − 1 = π(0).
(c) For this chain, P 0 [T 0 > n] = 1 · 2 · 5 · · · (n − 1)^2 + 1 2 · 3 · 6 · · · (n − 1)^2 + 2
Thus,
− log P 0 [T 0 > n] = −
n∑− 1
k=
log
k^2 + 2
which is the partial sum of a convergent (positive term) series, by the Limit Comparison Test (The ratio between − log(1 − 1 /(k^2 + 2)) and k−^2 converges to 1 as k → ∞.) This means that
− (^) nlim→∞ log P 0 [T 0 > n] < ∞,
which in turn implies that
P 0 [T 0 = ∞] = (^) nlim→∞ P 0 [T 0 > n] > 0 ,
and this chain is therefore transient.
2.8. (a) φ(s) = (5 + 8s + 7s^2 )/20. The mean value μ is equal to φ′(1) = 1.1. The smaller root of the fixed point equation is a = 5/7, the extinction probability. (b) φ(s) = (5+s+4s^3 )/10, so μ = φ′(1) = 1.2. The fixed point equation is 4s^3 − 9 s+5 = 0, which has roots 1 and − 1 ±
because 4s^3 − 9 s + 5 = (x − 1)(4x^2 + 4x − 5). The one of these roots that lies in (0, 1) is the desired fixed point: a = (
6 − 1)/ 2 ≈ .72. (c) For this distribution the mean μ is equal to
. 29 < 1, so the extinction probability is 1. (d) φ(s) = (1 − q)/(1 − qs), so μ = φ′(1) = q/(1 − q). If q ≤ 1 /2, then μ ≤ 1 and the extinction probability is 1. If q > 1 /2, Then the fixed point equation has roots 1 and a = (1 − q)/q, which is the extinction probability.
