Math 597/697: Homework 3
You are encouraged to work in groups. But please write your solutions
yourself. If you have questions about the homework, please ask them in
class, the other students will profit from them too.
1. The Smiths receive the paper every morning and place it on a pile
after reading it. Each afternoon, with probability 1/3 someone takes
all papers in the pile and put them in the recycling bin. Also if ever
there at least five papers in the pile, Mr Smith, with probability one,
take the papers to the bin in the afternoon. Consider the number of
papers in the pile in the evening and describe it with a Markov chain.
(a) What are the state space and the transition matrix?
(b) After a long time what would be the expected number of papers
in the pile?
(c) Assume that the piles starts with 0 papers. What is the expected
time until the pile will have again 0 papers.
(d) Same question as (c) but assume now that the pile starts with 2
papers.
2. Consider an irreducible Markov chain with a finite state space. Let
Mij =E[τj|x0=i] (1)
denote the expected return time to the state j, starting from i.
(a) Analyzing the first step show that
Mij = 1 + X
k
PikMkj (2)
(b) Let πibe the stationary distribution. Multiplying both sides by
πiand summing over ishow that
πj=1
Mjj
.(3)
3. The Ehrenfest urn model, continued from Problem 3 of HWK #2. This
is a Markov chain on {−a,−a+ 1, . . . , a −1, a}with nonzero transition
probabilities given by Pii+1 = (a−i)/2aand Pii−1= (a+i)/2a.
1
