Download Final exam for stochastic course and more Exams Stochastic Processes in PDF only on Docsity!
Applied Stochastic Processes - Spring 2025
Final Exam
May 3, 2025
- You have 2 hours to complete this exam.
- Calculators and one standard cheat sheet are allowed.
- If you need additional space, you may write your answers on additional sheets of paper.
- “Justify/explain your answer” generally means providing one or two sentences of explanation. Usually, you just need to quote a result or theorem given in class or in the book.
- (10 points) Consider a CTMC {X(t), t ≥ 0 } and the associated embedded Markov chain {Xn, n = 1, 2 ,.. .}. For each of the following properties explain briefly whether or not you can deduce if the property holds in the CTMC from knowledge of whether or not the property holds in the EMC: irreducibility, aperiodicity, recurrence, positive recurrence.
- If the EMC is irreducible so is the CTMC. The converse also holds.
- A CTMC is aperiodic with probability one. Thus the EMC is irrelevant for this property.
- If the EMC is recurrent so is the CTMC. The converse also holds.
- If the EMC is positive recurrent, the CTMC could be null recurrent or positive recurrent. Thus the EMC does not help you determine positive recurrence.
- (42 points total, each part is worth 6 points) The M/M/∞ Queue. Consider a queueing system with Poisson arrivals, i.i.d. exponential services, and an infinite numbers of servers. Since there are an infinite number of servers, no waiting room is needed. Let the arrival rate be λ and let the service rate (at each server) be μ.
(a) Model the system as a CTMC. You can just draw the rate diagram or give the rate matrix. (b) Suppose that there are n customers in the system at time 0. What is the expected time until these particular n customers have all exited the system? (c) Write down the global balance equations that the steady-state probabilities must satisfy.
(a) The infinitesimal transition rates are qi,i+1 = λ for i ≥ 0 and qi,i− 1 = iμ for i ≥ 1. Thus, the process is a birth-death process with a constant birth rate and variable death rate. (b) Let Ti be the time the ith customer, of the original n, departs. By the memoryless property of exponential, and the fact that the minimum of exponential r.v.’s is also exponential, the Ti ∼ exp[μ(n − i + 1)]. Next, let R be the time until the last customer exits. Then we have
ER = E
" (^) n X
i=
Ti
X^ n
i=
ETi =
X^ n
i=
[μ(n − i + 1)]−^1 =
μ
X^ n
i=
i
(c) For state 0 we have λp 0 = μp 1. For n > 0 the equations are
(λ + nμ)pn = λpn− 1 + (n + 1)μpn+1.
Finally, we must have X∞
n=
pn = 1.
- (28 points total, each part worth 7 points) Suppose we are concerned with a maintenance policy for a machine with i.i.d. lifetimes Y 1 , Y 2 ,... with common c.d.f. F (·). The maintenance policy is to replace the machine with a new one, either when it fails, or when its current age is T. If the machine fails during operation (we call this an unplanned replacement) there is a one time cost Cf associated with this failure. The cost to do a planned replacement (at age T ) is Cp. Let Si be the time of the ith replacement, planned or unplanned.
(a) Argue that the sequence {Si, i ≥ 0 } is a renewal sequence and that {Z(t), t ≥ 0 }, where Z(t) is the total cost up time t, is a renewal-reward process. (b) Give an expression for the mean inter-renewal time τ , in terms of F and T. (c) Give an expression for the long-run cost rate, as a function of F , T , Cf and Cp. (d) Suppose now that the machine lifetimes are U (0, b). Use the result in (c) to calculate the value of T which minimizes the long-run cost rate.
(a) Note that the inter-replacement times are given by Xn = Yn ∧ T. Since the Yn are i.i.d. it is then clear that the Xn are also i.i.d. Further, the reward during the nth renewal epoch is Rn =
Cp if Xn = T Cf if Xn < T. Since the reward is a function of the inter-renewal time, the process is a renewal- reward process. (b)
τ = E[Yn ∧ T ] =
Z T
0
(1 − F (t)) dt.
(c) Note that the expected cost during a renewal epoch is
r = E[Rn] = Cf F (T ) + Cp[1 − F (T )] = Cp + (Cf − Cp)F (T ).
Then by the renewal-reward theorem, we have:
lim t→∞
Z(t) t
Cp + (Cf − Cp)F (T ) R (^) T 0 (1^ −^ F^ (t))^ dt^
(d) First we have F (T ) = T /b, assuming 0 ≤ T ≤ b. Next, Z (^) T
0
(1 − F (t)) dt =
Z T
0
(1 − t/b) dt = T (1 − T /(2b)).
Using the result in (c) the cost is
g(T ) =
Cp + (Cf − Cp)(t/b) T (1 − T /(2b))
Setting g′(T ) = 0 and solving the resulting quadratic, we obtain
T ∗^ = b Cp Cf − CP
"s 1 + 2(Cf − Cp) Cp
- (20 points total, 10 points each part) Let A(t) be the age process associated with a renewal process.
(a) Write down a renewal type equation for E[A(t)k], for k > 0. (b) Find limt→∞ E[A(t)k] and argue that the KRT is applicable.
(a) Conditioning, as usual, on the first renewal time we obtain:
E[A(t)k^ | X 1 = x] =
H(t − x) x < t tk^ x ≥ t
Setting H(t) = E[A(t)k] yields the renewal-type equation
H(t) =
Z ∞
t
tk^ dG(x) +
Z (^) t
0
H(t − x) dG(x).
(b) To check the KRT, note that
D(t) =
Z ∞
t
tk^ dG(x) = tk(1 − G(t)).
The function tk^ and tkG(t) are both clearly increasing in t, hence D(t) is directly Riemann integrable. Applying now the KRT:
t^ lim→∞ E[A(t)k]^ =^
τ
Z ∞
0
Z ∞
t
tk^ dG(x)dt
τ
Z ∞
0
Z (^) x
0
tk^ dtdG(x)
E(Xk+1) (k + 1)τ
Bonus Question: (Ego and the Renewal-Reward Theorem) What great Russian writer, who died in a train station, said “A man is like a fraction whose numerator is what he is and whose denominator is what he thinks of himself. The larger the denominator, the smaller the fraction.”? Count Lyev Nikolayevich Tolstoy
