Dynamic Programming Under Certainty 6
Marek Kapicka, Econ 204b
April 15, 2009
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Optimal Growth Model with Stochastic Programming - Prof. M. Kapicka, Study notes of Economics
The concepts of dynamic programming under uncertainty, focusing on the optimal growth model and stochastic dynamic programming. The existence and uniqueness of the fixed point, monotonicity, concavity, and differentiability of the value function, as well as the dynamics of the optimal growth model and the stability of the steady state. The document also introduces stochastic dynamic programming and markov shocks, and demonstrates how the bellman equation applies in this context.
Typology: Study notes
Pre 2010
1 / 19
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Optimal Growth Model with Stochastic Programming - Prof. M. Kapicka and more Study notes Economics in PDF only on Docsity!
Dynamic Programming Under Certainty 6
Marek Kapicka, Econ 204b
April 15, 2009
Today
I We will
- Look at the dynamics of the solution in the optimal growth
model
- Look at stochastic dynamic programming
6.1. Example: Optimal Growth
Optimal Policy Function
I g (k) satisfies:
I First order condition:
U
′
(f (k) − g (k)) = β v
′
(g (k))
I Envelope condition:
v
′
(k) = U
′
(f (k) − g (k))f
′
(k)
I g (k) is strictly increasing.
7.1. Dynamics in the Optimal Growth Model
I Is there a steady state?
I Is it unique?
I Is it stable? (Does the capital stock converge to the steady
state?)
7.1. Dynamics in the Optimal Growth Model
Stability of Steady State
I Since v is strictly concave,
[v
′
(k) − v
′
(
k)][k −
k] ≤ 0 all k,
k
with equality only if k =
k.
I Choose
k = g (k). Then
[v
′
(k) − v
′
(g (k))][k − g (k)] ≤ 0
with equality only in steady state.
7.1. Dynamics in the Optimal Growth Model
Stability of Steady State
I Using FOC and EC,
[U
′
(c)f
′
(k) − U
′
(c)
β
][k − g (k)] ≤ 0
[f
′
(k) −
β
][k − g (k)] ≤ 0
I Therefore,
g (k) > k ⇔ f
′
(k) >
β
⇔ k < k
ss
g (k) < k ⇔ f
′
(k) <
β
⇔ k > k
ss
Optimal capital stock is decreasing if k > k
ss
and increasing if
k < k
ss
. Hence the solution is stable.
7.1. Stochastic Dynamic Programming
I Two important cases:
- The stochastic process is i.i.d.
Pr(z˜ t
= z t
| z˜ t− 1
= z t− 1
, ..˜z 0
= z 0
) = Pr(z˜ t
= z t
)
- The stochastic process is Markov
Pr(z˜ t
= z t
| z˜ t− 1
= z t− 1
, ..z˜ 0
= z 0
) = Pr(z˜ t
= z t
| z˜ t− 1
= z t− 1
)
7.1. Stochastic Dynamic Programming
Markov Shocks
I Define, for any z
t
= {z j
t
j = 0
π (z t
|z t− 1
) = Pr(z˜ t
= z t
| z˜ t− 1
= z t− 1
Π(z
t
|z 0 ) = Pr(z˜
t
= z
t
| z˜ 0 = z 0
I Note that since the shocks are Markov,
Π(z
t
|z 0 ) = Pr(z˜
t
= z
t
| z˜ 0 = z 0
= Pr(z˜
t
= z
t
| z˜
t− 1
= z
t− 1
) Pr(z˜
t− 1
= z
t− 1
| z˜ 0
= z 0
= Pr(z˜ t = z t | z˜
t− 1
= z
t− 1
) Pr(z˜
t− 1
= z
t− 1
| z˜ 0 = z 0
= π (z t
|z t− 1
)Π(z
t− 1
|z 0
I Therefore
Π(z
t
|z 0
) = π (z t
|z t− 1
) π (z t− 1
|z t− 2
).. π (z 1
|z 0
7.2. Stochastic Dynamic Programming
Optimal Growth Problem
I We will show that the value function in the sequence problem
v
∗
(k, z) satisfies the Bellman Equation
I One can also show that under a certain boundedness
condition, the solution to the Bellman Equation v (k, z)
satisfies the sequence problem.
7.2. Stochastic Dynamic Programming
A solution to (SP) satisfies (FE)
v
∗
(k 0 , z 0
= max
{ 0 ≤k t+ 1 (z
t )≤z t f ((z
t− 1 ))}
∞
t= 0
∞
t= 0
z
t ∈Z
t
β
t
U[c t
(z
t
)]Π(z
t
|z 0
= max
{ 0 ≤k t+ 1 (z
t )≤z t fk t ((z
t− 1 ))}
∞
t= 0
{U[z 0 f (k 0 ) − k 1 (z 0
)]
- β
∞
t= 0
z
t ∈Z
t
β
t− 1
U[c t (z
t
)]Π(z
t
|z 0
= max
{ 0 ≤k 1 (z)≤z 0 f (k 0 )}
∞
t= 0
{U[z 0
f (k 0
) − k 1
(z 0
)]
- β max
{ 0 ≤k t+ 1 (z
t )≤z t f ((z
t− 1 ))}
∞
t= 1
∞
t= 1
z
t ∈Z
t
β
t− 1
U[c t (z
t
)]Π(z
t
|z 0
7.3. Stochastic Dynamic Programming
General Setup
I z can affect
I the correspondence Γ
I the objective function F
I The Sequence Problem
v
∗
(x 0 , z 0 ) = max
{k t+ 1 (z
t )}
∞
t= 0
∞
t= 0
z
t ∈Z
t
β
t
F [x t (z
t− 1
), x t+ 1 (z
t
), z t ]Π(z
t
|z 0
s.t. x t+ 1
(z
t
) ∈ Γ[x t
(z
t− 1
), z t
] x 0
, z 0
given
I The Bellman Equation
v (x, z) = max
y ∈Γ(x,z)
F (x, y , z) + β
z
′ ∈Z
v (y , z
′
) π (z
′
|z)
7.3. Stochastic Dynamic Programming
Existence and uniqueness of the solution to Bellman Equations
I For existence and uniqueness, we need the same assumptions
as before!
I The fact that z is discrete is critical here. If z is not discrete,
we need to make sure that
I The expectation operator maps continuous functions into
continuous functions
I Γ(x, z) is continuous in z
7.3. Stochastic Dynamic Programming
Properties of the value function
I For (strict) concavity in x and differentiability in x: the same
assumptions as before:
I for (strict) concavity in x
I F is jointly (strictly) concave in x, y for all z
I Γ is convex in x for all z
I For differentiability in x:
I all of the above
I F is differentiable in x for all y , z
I g (x, z) is in the interior of Γ(x, z)