Download Problem Set 6 - Stochastic Processes - 2007 | MATH 285 and more Assignments Stochastic Processes in PDF only on Docsity!
Problem Set 6
Troy Kravitz
June 4, 2007
Problem 1
A chemical solution initially contains N molecules of type A and an equal number of molecules of type B. A reversible reaction occurs between type A and B molecules in which a molecule of type A bonds with a molecule of type B to form a new compound AB. Suppose that in any small time interval of length h, any particular unbounded A molecule will react with any particular unbounded B molecule with probability αh + o(h), where α is a positive reaction rate. Suppose also that in any small time interval of length h, any particular AB molecule dissociates into its A and B constituents with probability βh + o(h), where β is a positive reaction rate. Let X(t) denote the number of AB molecules at time t. Model X as a birth-death process and specify its infinitesimal matrix Q.
Let i denote the number of AB molecules, λi denote the parameter governing the creation of AB molecules when there are already i such AB molecules, and let μi denote the parameter governing the dissolution of AB molecules when there are currently i such AB molecules.
Since when there are i AB molecules there are N − i ”free” A and B molecules each, and since each free A molecule can bind with any free B molecule, there are (N − i)^2 potential marriages of A and B molecules. Thus, λi = α(N − i)^2. Similarly, since each of the i combined molecules dissolves according to the exponential distribution with rate parameter μi, we have μi = βi.
The Q matrix is therefore
Q =
−λ 0 λ 0 0 0 ... 0 0 μ 1 −(μ 1 + λ 1 ) λ 1 0 ... 0 0 0 μ 2 −(μ 2 + λ 2 ) λ 2 ... 0 0 ... ... ... ... ... ... ... 0 0 0 0 ... μN −μN
−N 2 α N 2 α 0 0 ... 0 0 β −(β + (N − 1)^2 α) (N − 1)^2 α 0 ... 0 0 0 2 β −(2β + (N − 2)^2 α) (N − 2)^2 α ... 0 0 ... ... ... ... ... ... ... 0 0 0 0 ... N β −N β
The initial distribution is
[
]′
, an (N + 1) × 1 vector with unity as the first element (corresponding to no AB molecules) and zeros everywhere else.
Problem 2
A shopping center parking lot has twenty spaces. Cars arrive at the lot according to a Poisson process with rate λ > 0. If a car arrives at the lot and there is an empty space, it takes the space. If a car arrives and there are no empty spaces, the car moves on to another lot and never returns. Any parked car remains in its space for an exponentially distributed amount of time with mean (^) μ^1 where μ is a positive finite number. Assume that the times that different cars remain parked are independent and that these times are independent of the Poisson arrival process. Take as given that the number of cars in the lot at time t is a continuous time Markov chain. (a) Determine the infinitesimal transition probabilities for this Markov chain. (b) Write down the equations for a stationary distribution Π = (π 0 , ..., π 20 ) of this Markov chain. (c) Solve the equations in (b) for πi, i = 1, ..., 20 in terms of π 0. (d) Find an expression for the long run probability that the lot is full.
We are given that parked cars leave spots according to an exponential distribu- tion with mean (^) μ^1. This means the rate parameter is μ. Letting i denote the number of cars currently parked, we have λi = λ and μi = i · μ for i = 0, ..., 20.
Problem 3
Consider a CTMC with Q matrix (depicted in answers below). Is the chain irreducible? Find a stationary distribution for this chain. What is limt→∞ P 12 (t)?
First, we establish that the chain is irreducible. This can be seen most easily by considering the discrete-time skeleton of the chain (which I cannot reproduce in LaTeX). The discrete-time skeleton shows that you can get from any state to any other state in any positive amount of time. (To get from state 2 to state 3, we must first pass through state 1).
Next, we note that since the chain has a finite number of states, it does not explode. (This was given as one of the sufficient conditions to rule out explosion in class.)
Now, we seek a stationary distribution. In the context of continuous time Markov chains, a stationary distribution is a probability vector (the elements must be non-negative and sum to unity), Π, such that Π′Q = 0′. Letting Π′^ =
[
π 1 π 2 π 3
]
be our arbitrary probability vector, the matrix system Π′Q = 0′^ corresponds to
[ π 1 π 2 π 3
]
[
]
This system generates the following four conditions (below, we’ve included the requirement that Π is a probability vector:
- − 2 π 1 + 4π 2 + 2π 3 = 0
- π 1 − 4 π 2 + π 3 = 0
- π 1 − 3 π 3 = 0
- π 1 + π 2 + π 3 = 1
These conditions yield the stationary distribution Π′^ =
[ 3
5
1 5
1 5
]
. Since our irreducible, non-exploding Markov chain has a stationary distribution, we invoke Theorem 3.6.2 to show
lim t→∞ P 12 (t) = π 2 =
Problem 4
Problem 3.7.1: Consider a fleet of N buses. Each bus breaks down independently at rate μ, when it is sent to the depot for repair. The
repair shop can only repair one bus at a time and each bus takes an exponential time of parameter λ to repair. Find the equilibrium distribution of the number of buses in service.
We will follow the example for a simple biochemical reaction considered in class. Let i denote the number of buses in service at a given time. Since repaired buses are returned to service at constant rate λ regardless of how many buses are currently out-of-service, λi = λ. Since currently-operable buses break down at rate μ, the parameter governing the rate at which buses go out-of-service depends on the number of buses currently in service (as opposed to the previous case). Thus, we have μi = iμ.
As before, it is possible to reach any state from any other state in any positive amount of time (although some combinations are highly unlikely). Therefore, the Markov chain under consideration is irreducible. (Heuristically, this Markov chain is akin to the drunkard’s walk on a subset of Z.) Starting from any state, it is possible to move left or move right in any positive amount of time. That is, we can go from having i buses in service to having either i − 1 or i + 1 buses in service in any amount of time. If we repeat this argument it is apparent that we can go from any state to any other state in any positive amount of time.
To ground the explanation that follows, we briefly consider two degenerate cases. Suppose λ < μi ∀i; then buses are always breaking down faster than they can be repaired. In expectation then, there should be no buses in service. On the other hand, suppose λ > μi ∀i; then the broken buses are always being replaced at a quicker rate than they are breaking down. In expectation, the full fleet should be in operation.
Now suppose we have detailed balance. (We actually seek a stationary distri- bution; as such, we could re-create the steps used in problem (2) but, instead, we’ll seek detailed balance. If we satisfy detailed balance, then we know we have also found a stationary distribution.) Then πiQi,i+1 = πi+1Qi+1,i. In our case, Qi,i+1 = λ and Qi+1,i = μi+1 = (i + 1)μ. It follows that πi+1 = πi (^) μiλ+1. We can
As an aside, the expected number of buses in service is given by:
E[i] =
∑^ N
j=
jπj
∑^ N
j=
jπj
∑^ N
j=
j( λ μ
)j−^1 π 0 (j − 1)!
= π 0
∑^ N
j=
λ μ
)j−^1
(j − 1)!
∑N
j=1(^
λ μ ) j−1 1 j!
∑^ N
j=
λ μ
)j−^1
(j − 1)!
∑N
j=1(^
λ μ )
j− 1 1 (j−1)! 1 +
∑N
j=1(^
λ μ ) j−1 1 j!
