Download Solutions for Prob Set 9: Markov Chains & Expected Trials and more Exams Probability and Statistics in PDF only on Docsity!
Probability: Problem Set 9 Solutions
Fall 2009
Instructor: W. D. Gillam
(1) Recall the negative binomial distribution which counts the number of trials of a
two outcome (0, 1 say, with probabilities 1 − p, p) experiment up to and including
the n
th success. Explain how to view this as a (finite, homogeneous, absorbing)
Markov chain with state space X = { 0 , 1 ,... , n}. What probability distribution μ
should you use for the “initial” probabilities? What is... ...the transition matrix
P? ...the matrix Q of transition probabilities between transient states? ...the
fundamental matrix (Id −Q)
− 1 ? Use the general theory to compute the expected
number of trials needed to reach the n
th success.
Solution. I think of the states as being the number of successes up to a given
time point, so we should set μ(i) equal to zero unless i = 0 (the process starts
from zero successes). The transition matrix P is then
P =
q p 0 0 · · · 0 0
0 q p 0 · · · 0 0
0 0 q p · · · 0 0
. . .
0 0 0 0 · · · q p
0 0 0 0 · · · 0 1
The matrix (Id −Q) is the n × n (really: { 0 ,... , n − 1 } × { 0 ,... , n − 1 }) matrix
Id −Q =
p −p 0 0 · · · 0 0
0 p −p 0 · · · 0 0
0 0 p −p · · · 0 0
. . .
0 0 0 0 · · · p −p
0 0 0 0 · · · 0 p
which has inverse
(Id −Q)
− 1
p
− 1 p
− 1 p
− 1 p
− 1 · · · p
− 1 p
− 1
0 p
− 1 p
− 1 p
− 1 · · · p
− 1 p
− 1
0 0 p
− 1 p
− 1 · · · p
− 1 p
− 1
0 0 0 0 · · · p
− 1 p
− 1
0 0 0 0 · · · 0 p
− 1
The expected number of trials to reach the unique absorbing state n from the
transient state 0 is then given by the sum of the entries in the first row (really the
0
th row), which is np
− 1 .
(2) Recall that we discussed the problem of finding the expected number of fouls
needed for a player to make some number n of free throws, assuming that the
player shoots two free throws each time (s)he (in the (W)NBA) is fouled. This
depends on the player’s free throw percentage p, which for Shaq is p = .528.
1
Explain how this problem can be viewed as a (finite, homogeneous, absorbing)
Markov chain with state space X = { 0 , 1 ,... , n}. Unlike in the previous problem,
there will be a positive probability of moving from state i to i + 2 ≤ n, which
makes the transition matrix P less sparse, and therefore harder to invert. Use
the general machinery to compute the expected number of fouls needed to make
n = 1, 2 , 3 free throws (as a function of p). Plot these three functions of p on the
same axes. This might be hard if you are not capable of using some computer
software to invert the matrices.
Solution. The player’s probabilities of making 0, 1 , 2 free throws on one foul are
given by
p 0 := (1 − p)
2
p 1 := 2 p(1 − p)
p 2 := p
2 ,
respectively. Note
p 0 + p 1 + p 2 = 1.
Here I view the states as the number of free throws made up to a given point in
time, so that the transition matrix is
P =
p 0 p 1 p 2 0 · · · 0 0
0 p 0 p 1 p 2 · · · 0 0
0 0 p 0 p 1 · · · 0 0
. . .
0 0 0 0 · · · p 1 p 2
0 0 0 0 · · · p 0 p 1 + p 2
0 0 0 0 · · · 0 1
and the matrix (Id −Q) is
(Id −Q) =
p 1 + p 2 −p 1 −p 2 0 · · · 0 0
0 p 1 + p 2 −p 1 −p 2 · · · 0 0
0 0 p 1 + p 2 −p 1 · · · 0 0
. . .
0 0 0 0 · · · p 1 + p 2 −p 1
0 0 0 0 · · · 0 p 1 + p 2
2 p − p
2 − 2 p + 2p
2 −p
2 0 · · · 0 0
0 2 p − p
2 − 2 p + 2p
2 −p
2 · · · 0 0
0 0 2 p − p
2 − 2 p + 2p
2 · · · 0 0
. . .
0 0 0 0 · · · 2 p − p
2 − 2 p + 2p
2
0 0 0 0 · · · 0 2 p − p
2
The case n = 1 is rather trivial: the matrix Id −Q is the 1 × 1 matrix with entry
p 1 + p 2 , so its inverse is just (p 1 + p 2 )
− 1
. The expected number of fouls in this case
For n = 4, the fundamental matrix is given by
(Id 4 −Q 4 )
− 1
p 1 + p 2 −p 1 −p 2 0
0 p 1 + p 2 −p 1 −p 2
0 0 p 1 + p 2 −p 1
0 0 0 p 1 + p 2
− 1
p 1 + p 2
p
3 1 + 2p^2 p
2 1 +^ p
2 2 p^1
(p 1 + p 2 )^4
p
3 1 + 2p^2 p
2 1 + 2p
2 2 p^1 +^ p
3 2
(p 1 + p 2 )^4
p
3 1 + 2p^2 p
2 1 + 2p
2 2 p^1
(p 1 + p 2 )^4
p 1 + p 2
p
3 1 + 2p^2 p
2 1 +^ p
2 2 p^1
(p 1 + p 2 )^4
p
3 1 + 2p^2 p
2 1 + 2p
2 2 p^1 +^ p
3 2
(p 1 + p 2 )^4
p 1 + p 2
p 1
(p 1 + p 2 )
2
p 1 + p 2
D 4 (p)
8 − 12 p + 6p
2 − p
3 8 − 16 p + 10p
2 − 2 p
3 8 − 16 p + 12p
2 − 3 p
3 8 − 16 p + 12p
2 − 4 p
3
0 8 − 12 p + 6p
2 − p
3 8 − 16 p + 10p
2 − 2 p
3 8 − 16 p + 12p
2 − 3 p
3
0 0 8 − 12 p + 6p
2 − p
3 8 − 16 p + 10p
2 − 2 p
3
0 0 0 8 − 12 p + 6p
2 − p
3
where
D 4 (p) = 16 p − 32 p
2
3 − 8 p
4
5 .
Adding the entries in the first row, we find:
E 4 =
32 − 60 p + 40p
2 − 10 p
3
16 p − 32 p^2 + 24p^3 − 8 p^4 + p^5
(3) In class we studied the expected number of coin flips needed to observe two heads
in a row: “HH”. One can of course also consider the expected number of coin
flips needed to observe some more elaborate pattern: “HHTH,” say. This can
be calculated in a variety of ways. Do Exercises 28 and 30 in Chapter 11.2 of
Grinstead and Snell (pp. 428-430) to get the general idea.
0.0 0.2 0.4 0.6 0.8 1.
2
4
6
8
10
Figure 1. Graphs of E 1 , E 2 , E 3 , E 4 in blue, purple, yellow, green, respectively.
