Download 3 Problems with Solutions on Stochastic Processes - Assignment | M 362M and more Assignments Stochastic Processes in PDF only on Docsity!
University of Texas at Austin
HW Assignment 6
Problem 6.1. Let {Xn}n∈N 0 be the simple symmetric random walk. Which of the following processes {Yn}n∈N 0 are Markov?
(1) Yn = (Xn)^2 , N ∈ N 0 (Hint: this is a simple combination of two results, one about one-to-one functions and the other about the absolute value of random walks from the notes.)
(2) Yn = Xn (mod 2) =
0 , if Xn is even, and 1 , if Xn is odd. (3) Let
Rn = Xn (mod 3) =
0 , if Xn is divisible by 3, 1 , if Xn − 1 is divisible by 3, 2 , if Xn − 2 is divisible by 3, be the remainder obtained when Xn is divided by 3, and let Yn = (Xn − Rn)/3 be the quotient, so that Yn ∈ Z and 3Yn ≤ Xn < 3(Yn + 1). (Hint: Play with small values of n: n = 0, 1 , 2 , 3 will be enough.)
Solution:
(1) We know that |Xn| is a Markov chain (see the “More complicated example” (Example 8.) in Lecture 8), and its state space S is N 0. The function f : S → { 0 , 1 , 2 , 4 , 9 ,... }, f (n) = n^2 is one-to-one and onto. Therefore, the process Yn = f (|Xn|) = (Xn)^2 is a Markov chain. (2) A simple random walk jumps from an odd number to an even one, and vice versa. Therefore,
Yn =
0 , n even, 1 , n odd, and it is clearly a Markov chain (it is not even random). (3) This is not a Markov chain. Let us consider the event A = {Y 2 = 0, Y 1 = 0}. The only way for this to happen is if X 1 = 1 and X 2 = 2 or X 1 = 1 and X 2 = 0, so that A = {X 1 = 1}. Also Y 3 = 1 if and only if X 3 = 3. Therefore P[Y 3 = 1|Y 2 = 0, Y 1 = 0] = P[X 3 = 3|X 1 = 1] = 1/ 4. On the other hand, Y 2 = 0 if and only if X 2 = 0 or X 2 = 2, so P[Y 2 = 0] = 3/4. Finally, Y 3 = 1 and Y 2 = 0 if and only if X 3 = 3 and so P[Y 3 = 1, Y 2 = 0] = 1/8. Therefore
P[Y 3 = 1|Y 2 = 0] = P[Y 3 = 1, Y 2 = 0]/P[Y 2 = 0] =
Therefore, Y is not a Markov chain.
Problem 6.2. A monkey is sitting in front of a typewritter in an effort to re-write the complete works of William Shakespeare. She has two states of mind “inspired” and “in writer’s block”. If the monkey is inspired, she types the letter a with probability 1/2 and the letter b with probability 1/2. If the monkey is in writer’s block, she will type b. After the monkey types a letter, her state of mind changes independently the previous state of mind, but depending on the letter typed as follows:
- into “inspired” with probability 1/3 and “in writer’s block” with probability 2/3, if the letter typed was a and,
- into “inspired” with probability 2/3 and “in writer’s block” with probability 1/3, if the letter typed was b. (1) What is the probability that the monkey types abbabaabab in the first 10 strokes, if she starts in the inspired state?
(2) Another monkey is sitting next to her, trying to do the same (re-write Shakespeare). He has no states of mind, and types a or b with equal probability each time, independently of what he typed before. A piece of paper with abbabaabab on it is found, but we don’t know who produced it. Which monkey is more likely to have done it, the she-monkey or the he-monkey?
Solution:
(1) One can model this situation by a Markov chain {Xn}n∈N 0 with states {IN, W B, a, b} where a or b always follow IN or W B, and vice versa. The letters typed also form a Markov chain on their own: P[Xn+2 = a|Xn = a] = P[Xn+2 = a|Xn = a, Xn+1 = IN ]P[Xn+1 = IN |Xn = a] = P[Xn+2 = a|Xn = a, Xn+1 = W B]P[Xn+1 = W B|Xn = a] = 12 × 13 + 0 × 23 = 16. Similarly, P[Xn+2 = b|Xn = a] = 12 × 13 + 1 × 23 = (^56) P[Xn+2 = a|Xn = b] = 12 × 23 + 0 × 13 = (^13) P[Xn+2 = b|Xn = b] = 12 × 23 + 1 × 13 = (^23) Since we start in IN , the first letter typed is a or b with equal probabilities. Therefore, the probability Ps of abbabaabab is Ps = 12 × 56 × 23 × 13 × 56 × 13 × 16 × 56 × 13 × 56 = 5
4
(2) The probability Ph that the he-monkey typed the letters is 2−^10. You can use your calculator to compute that P Phs ∼ 0 .98, so it is more likely that the she-monkey wrote it.
Problem 6.3. Use the Mathematica package Stochastics (to be covered in class on Thursday, 11/06) to simulate 500 games of tennis between Am´elie and Bj¨orn, where Am´elie beats Bj¨orn in a single point with p = 0.55 (you can set the game-length to 50, just to be sure that the game will end by then). The result will be a matrix with 500 rows and 50 columns. Find a way to convert the last column of this matrix into a list and count the number of wins each player made (command Count may be useful). How do the relative frequencies compare to the value p = 0.55? How does that relate to the question about playing Roger Federer in the notes? (Optional: Repeat for a variety (say 20) values of p between 0 and 1. Draw a graph of your results. For this part you will have to figure out how to do loops (try Do) or define functions and map them to lists. For plotting graphs of lists of points try ListPlot.)
Solution: In[28]:= << Stochastics`
In[29]:= nsim = 500; pathlength = 50; p = 0.55;
In[30]:= MyChain = TennisChain@pD;
In[31]:= Paths^ =^ SimulatePaths@nsim, pathlength, MyChainD;
In[32]:= TransposedPaths^ =^ Transpose@PathsD;
In[33]:= Results^ =^ TransposedPaths@@^50 DD;
In[34]:= AmelieWins^ =^ Count@Results,^8 "Amelie^ wins"<D Out[34]= 318
In[35]:= Prob^ =^ N@AmelieWins^ ^ nsimD Out[35]= 0.
One way of getting the last column of a matrix is by transposing it first and then accessing its last row. Alternatively, you can just write Paths[[All,50]]. The command Count is self-explanatory. The result
GameProbs. Here is what you get: In[107]:= TwoRows = 8 PointProbs, GameProbs< Out[107]= 88 0., 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95, 1.<, 8 0., 0., 0.002, 0.006, 0.034, 0.048, 0.114, 0.144, 0.276, 0.424, 0.464, 0.642, 0.688, 0.822, 0.896, 0.946, 0.984, 0.994, 0.996, 1., 1.<< In[108]:= Points^ =^ Transpose@TwoRowsD Out[108]= 88 0., 0.<,^8 0.05, 0.<,^8 0.1, 0.002<,^8 0.15, 0.006<,^8 0.2, 0.034<, 8 0.25, 0.048<, 8 0.3, 0.114<, 8 0.35, 0.144<, 8 0.4, 0.276<, 8 0.45, 0.424<, 8 0.5, 0.464<, 8 0.55, 0.642<, 8 0.6, 0.688<, 8 0.65, 0.822<, 8 0.7, 0.896<, 8 0.75, 0.946<, 8 0.8, 0.984<, 8 0.85, 0.994<, 8 0.9, 0.996<, 8 0.95, 1.<, 8 1., 1.<< In[110]:= ListPlot@PointsD
Out[110]=
0.2 0.4 0.6 0.8 1.
