Download Math 574: Optimal Control - HW 7 on Stochastic Dynamic Programming for Jump-Diffusions and more Assignments Mathematics in PDF only on Docsity!
Math 574 Applied Optimal Control – Hanson – Fall 2006
(Stochastic Processes and Control for Jump-Diffusions)
Homework 7 – Stochastic Control (Chapters 7 of Text; See also Chapter 0 for Preliminaries)
Homework due 08 December 2005 in class. Acknowledge consultation with others, even if none, else receive a grade discount.
- For the linear jump-diffusion dynamics
dX(t) = (μ 0 X(t) + β 0 U (t))dt + σ 0 dW (t) + ν 0 X(t)dP (t),
for 0 ≤ t ≤ tf and state X(0) = x 0 > 0 and the control process −∞ < U (t) < +∞ is unconstrained. The coefficients μ 0 6 = 0, β 0 6 = 0, σ 0 > 0, ν 0 6 = 0 and λ 0 > 0 are constants, where E[dP (t)] = λ 0 dt. The costs are quadratic, i.e.,
V [X, U ](X(t), t) =
∫ (^) tf
t
q 0 X^2 (s) + r 0 U 2 (s)
ds +
Sf X^2 (tf )
for q 0 > 0, r 0 > 0, and Sf > 0,
(a) Derive the PDE of Stochastic Dynamic Programming for the optimal expected value: v∗(x, t) = min u [E [V [X, U ](X(t), t) |X(t) = x, U (t) = u]] ,
using the Principle of Optimality; (b) Specify the final condition for v∗(x, t); (c) Formally find the optimal (unconstrained) control u∗(x, t) in terms of the shadow “cost” v∗ x(x, t);
- Derive the modifications necessary in the set of Riccati-like equations for the Linear- Quadratic Jump-Diffusion (LQJD, LQGP or JLQG) problem when the dynamics are scalar and linear (affine), i.e.,
dX(t) = f (X(t), U (t), t)dt + g(X(t), t)dW (t) + h(X(t), t)dP (t) ,
where E[dP (t)] = λ(t)dt , f (x, u, t) = f 0 , 0 (t) + f 1 , 1 (t)x + f 1 , 2 (t)u , g(x, t) = g 0 , 0 (t) + g 1 , 1 (t)x , h(x, t) = h 0 , 0 (t) + h 1 , 1 (t)x , the jump amplitude being independent of any mark process. The running and terminal costs for a maximum objective are quadratic,
C(x, u, t) = C 0 , 0 (t) + C 1 , 1 (t)x + C 1 , 2 (t)u + 0. 5 C 2 , 1 , 1 (t)x^2 + C 2 , 1 , 2 (t)xu +0. 5 C 2 , 2 , 2 (t)u^2 ,
where C 02 (t) < 0, and
S(x, t) = S 0 (t) + S 1 (t)x + 0. 5 ∗ S 2 (t)x^2 ,
where S 2 (t) < 0. If the objective is to maximize the expected total utility in the unconstrained control case, then find the coefficient functions v 0 (t), v 1 (t), v 2 (t), u 0 (t) and u 1 (t) in the solutions
v∗(x, t) = v 0 (t) + v 1 (t)x + 0. 5 v 2 (t)x^2
and u∗(x, t) = u 0 (t) + u 1 (t)x explicitly in terms of the dynamical and cost coefficient functions. Do not try to solve the Riccati equation system for {v 0 (t), v 1 (t), v 2 (t)}.
- Derive the Hamilton-Jacobi-Bellman PDE for the optimal stochastic control problem (a simplified jump-diffusion optimal portfolio problem), with stochastic dynamical sys- tem,
dX(t) = X(t)
μ 0 (t) + μ 1 (t)U 1 (t))dt + σ(t)dW (t) +
eQ^ − 1
dP (t)
− U 2 (t)dt ,
where t ∈ [0, tf ], X(0) = x 0 , E[dP (t)] = λ(t)dt, Var[W (t)] = E[W 2 (t)] = dt, Q is an IID uniformly distributed mark on [a, b], a < 0 < b, {μ 0 (t), μ 1 (t), σ(t), λ(t)} are specified time-dependent coefficients, X(t) ≥ 0 is the state, {U 1 (t), U 2 (t)} is the control set, 0 ≤ U 2 (t) ≤ K 2 X(t), K 2 > 0, and the optimal objective is
v∗(x, t) = max {u 1 , u 2 }
[
E
[∫ (^) tf
t
e−β(s^ −^ t)^
U 2 γ (s) γ
ds + e−β(tf^ −^ t)^
Xγ^ (tf ) γ
∣ C
]]
where C ≡ {X(t) = x, U 1 (t) = u 1 (t), U 2 (t) = u 2 (t)} is the conditioning set, β > 0 is a constant discount factor, γ ∈ (0, 1) is a constant utility power and the zero-state absorbing boundary condition is v∗(0+, t) = 0. Determine the modified HJBE when the discount factor is time-dependent, i.e., the instantaneous discount factor β̂ (t), replacing β(s − t) by the cumulative discount B(t, s) ≡
∫ (^) s t
β(r)dr, etc.
