Dynamic Programming: Stochastic Problems and Numerical Methods in Economics, Exams of Economics

Download Dynamic Programming: Stochastic Problems and Numerical Methods in Economics and more Exams Economics in PDF only on Docsity!

More on Dynamic Programming:

Stochastic Problems

and Numerical Methods

ECO S

Noah Williams

Princeton University

S500: More on Dynamic Programming 1

Stochastic LQ Problem

• Add additive, i.i.d. stochastic shocks to LQ problem.
• Solution displays certainty equivalence: same optimal policy as deterministic problem.
• Very useful since leads to practical solution methods for deterministic problems.
• More general problems will not display this. Uncertainty will matter for decision rules.

S500: More on Dynamic Programming 2

Numerical Dynamic Programming

• Very large topic of practical importance. Many efficient methods, but no all-purpose efficient one.
• For more detail: Judd (1998), Miranda and Fackler (2002).
• We’ll touch on simple method, which is relatively robust: discretizing state space.
• We focus on value function iteration. More efficient methods to update value function (policy iteration), others try to solve Euler equation directly.

S500: More on Dynamic Programming 3

Discrete State Dynamic Programming

• Require all state variables to reside on a finite space.
• Converts Bellman equation from general functional equation to a vectors and matrices.
• Illustration: consumption/saving problem with random income.
• Household either employed or not st ∈ {s 1 , s 2 ,... , sm}. Fixed wage rate w: labor income = wst.
• Employment status follows Markov chain with transition matrix P.
• Agent accumulates asset at ∈ A = {a 0 , a 1 ,... , an}.

S500: More on Dynamic Programming 4

Matlab Implementation

• Define parameters mu = 3; % risk aversion beta = 0.9; % subjective discount factor wage= 0.2; r=0.05 % wages and interest rates N = 2; prob=[.8,.2;.05,.95]; % # of wage states and transitions s=[0.2,1] % employment states
• Setup asset grid maxkap = ; % maximum value of capital grid minkap = -s(1)*wage/r; % borrowing constraint inckap = 0.01; % size of capital grid increments kap = minkap:inckap:maxkap; % state of assets nkap = length(kap); % number of grid points

S500: More on Dynamic Programming 5

• Initialize variables v = zeros(nkap,N); tv = zeros(N,nkap); decis = zeros(nkap,N); tdecis = zeros(N,nkap); cons = zeros(nkap,nkap,N); util = zeros(nkap,nkap,N); vint = zeros(nkap,nkap,N); test=10;
• Iterate on Bellman equation

S500: More on Dynamic Programming 7

Policy Functions: Assets

−1−1 −0.5 0 0.5 1 1.5 2 2.5 3

−0.

0

1

2

3 Policy function: Capital

asset of current period

asset of next period

Low s High s at=at+

S500: More on Dynamic Programming 8

Policy Functions: Consumption

−1^0 −0.5 0 0.5 1 1.5 2 2.5 3

0.45^ Policy function: Consumption

current asset

consumption

Low s High s