Dynamic Programming Under Certainty: Bellman Equation and Contraction Mapping Theorem, Study notes of Environmental Science

Download Dynamic Programming Under Certainty: Bellman Equation and Contraction Mapping Theorem and more Study notes Environmental Science in PDF only on Docsity!

Dynamic Programming Under Certainty 4

Marek Kapicka, Econ 204b

April 8, 2009

Review

I (^) Bellman Equation v ∗^ as a fixed point

v ∗^ = Tv ∗

of the operator

(Tv )(x ) = max y ∈Γ(x )

F (x, y ) + β v (y )

I (^) Contraction Mapping Theorem: If T maps bounded and continuous functions to itself and is a contraction, it has a unique fixed point

4.2. Contraction Mapping

Blackwell’s Sufficient Conditions for a Contraction

Theorem

Let S be a space of bounded functions on X , endowed with a sup norm. Let T : S → S. If i) T is monotone: If f (x ) ≤ g (x ) for all x ∈ X then Tf (x ) ≤ Tg (x ) for all x ∈ X. ii) T discounts: For some β ∈ (0, 1) and any a ∈ R+,

T (f + a)(x ) ≤ Tf (x ) + β a ∀x ∈ X ,

where (f + a)(x ) = f (x ) + a, then T is a contraction with modulus β.

4.3. Theorem of the Maximum

I (^) We want to make sure that an operator T maps continuous functions into continuous functions.

I (^) Assumptions: 2a. Γ is nonempty (i.e. Γ(x ) is nonempty for all x ∈ X ) 2b. Γ is compact valued (i.e. Γ(x ) is compact for all x ∈ X ) 2c. Γ is continuous (??)

4.3. Theorem of the Maximum

Theorem

Let X ∈ Rl^ and Y ∈ Rm. Define

h(x ) = max y ∈Γ(x )

f (x, y ), g (x ) = arg max y ∈Γ(x )

f (x, y )

Suppose that f : X × Y → R is continuous and Γ : X → Y is nonempty, compact valued and continuous. Then i) h : X → R is continuous and ii) g : X → Y is upper hemi-continuous and compact valued.

4.4. Bellman Equation Application

I (^) The Bellman Operator:

(Tv )(x ) = max y ∈Γ(x )

F (x, y ) + β v (y )

Theorem

Let S be the space of bounded and continuous functions with a sup norm. Suppose that: i) (A1): F (x, y ) is bounded and continuous ii) 0 < β < 1 iii) (A2): Γ is nonempty, compact valued and continuous. Then the Bellman operator T i) maps S onto itself, ii) has a unique fixed point v ∗^ ∈ S, iii) ‖T nv 0 − v ∗‖ ≤ β n‖v 0 − v ∗‖, iv) The optimal policy correspondence g (x ) is compact valued and u.h.c.

5.1. Corollaries to the contraction mapping theorem

Corollary

Let (S, ρ ) be a complete metric space. Let T : S → S be a contraction mapping that has a fixed point v ∈ S. Then

1. If S′^ is a closed subset of S and T (S′) ⊆ S′^ then v ∈ S′.
2. If in addition T (S′) ⊆ S ′′ then v ∈ S′′.

5.2 General Approach

I (^) to show that the fixed point has a given property, we will I (^) look at the conditions that guarantee that T maps a set of functions with that property onto itself I (^) If the set of functions with a given property is closed, then, by Corollary 1, the fixed point will preserve that property I (^) If the set of functions with a given property is not closed, but Corollary 2 holds, then fixed point will preserve the property. I (^) For differentiability, the approach fails: Corollaries 1 and 2 do not apply. I (^) Other tricks

5.3. Strict Monotonicity

Theorem

If in addition F (x, y ) is strictly increasing in x then the fixed point v ∗^ is strictly increasing. I (^) Idea of proof: Let S++^ be a set of bounded, continuous and strictly increasing functions. S+^ ⊆ S. But since S++^ is not closed, one must show that for any v increasing, Tv is strictly increasing. This is guaranteed by the assumption that F is strictly increasing.