Utah State University
ECE 6010
Stochastic Processes
Homework #11
Due Friday Dec 10, 2004
These problems come from the Leon-Garcia text.
1. Let Mndenote the sequence of sample means from an i.i.d. random process Xn:
Mn=X1+x2+···+Xn
n
(a) Is Mna Markov process?
(b) If so, find the state transition p.m.f. fMn(x|Mn−1=y)
2. An urn initially contains five black balls and five white balls. The following experiment is repeated
indefinitely. A ball is drawn from the urn; if the ball is white it is put back in the urn, otherwise it is
left out. Let Xnbe the number of black balls remaining in the urn after ndraws from the urn.
(a) Is Xna Markov process? If so, find the appropriate transition probabilities.
(b) Do the transition probabilities depend on n?
3. Let Xnbe the Bernoulli i.i.d. process and let Ynbe
Yn=Xn+Xn−1.
(a) Show that Ynis not a Markov process.
(b) Now consider the vector process Zn= (Xn, Xn−1). Show that Znis a Markov process.
(c) Find the state transition diagram for Zn.
4. Show that the following autoregressive process is a Markov process:
Yn=rYn−1+Xn,
with Yn= 0, where Xnis an i.i.d. process.
5. Let Xnbe the Markov chain in problem 2.
(a) Find the one-step transition probability matrix Pfor Xn.
(b) Find the two-step transition probability matrix P2. Check your answer by computing p54(2) and
comparing it to the corresponding entry of P2
(c) What happens to Xnas n→ ∞? Use your answer to guess the limit of Pnas n→ ∞.
6. Two players play the following game. A fair coin is flipped; if the outcome is heads, player A pays
player B $1, and if th outcome is tails, player B pays player A $1. The game is continued until one of
the players goes broke. Suppose initially that player A has $1 and player B has $2, so a total of $3 is
up for grabs. Let Xnbe the number of dollars held by player A after nrounds.
1
