Download Lecture Notes on Unconstrained Optimization | ECON XLIST and more Study notes Economics in PDF only on Docsity!
Math Camp Notes: Unconstrained Optimization
Economics is a science of optima. We maximize utility functions, minimize cost functions, and nd optimal allocations. In order to study optimization, we must rst de ne what a maxima and minima are. Let F : X → Y be a function, where X is open. Then
- We say x ∈ X is a local maximum of F on X if there is r > 0 such that f (x) ≥ f (y) for all y ∈ X ∩ B(x, r). If the inequality is strict, then we have a strict local maximum.
- We say x ∈ X is a local minimum of F on X if there is r > 0 such that f (x) ≤ f (y) for all y ∈ X ∩ B(x, r). If the inequality is strict, then we have a strict local minimum.
- We say x ∈ X is a global maximum of F on X if f (x) ≥ f (y) for all y ∈ X. If the inequality is strict, then we have a strict global maximum.
- We say x ∈ X is a global minimum of F on X if f (x) ≤ f (y) for all y ∈ X. If the inequality is strict, then we have a strict global minimum.
We seek conditions whereby we can tell if x∗^ ∈ X is a local maximum or minimum.
First Order Conditions
Let F : U → R^1 be a continuously dierentiable function de ned on U and open subset of Rn. Then if x∗^ is a local maximum or minimum of F in U , then
DF (x∗) = 0.
Notice that the converse is not true, that if DF (x∗) = 0 , then x∗^ is a local maximum. For example, consider the function f (x) = x^3 on R^1. Then Df = 3x^2 , which implies that when x = 0, Df (0) = 0. However, x = 0 is not a local maximum or minimum since any element greater than 0 in any -ball about 0 will be greater than f (0), and any element less than 0 in any -ball about 0 will be less than f (0). A condition which only goes in the ⇒ direction such as this is called a necessary condition. A condition which only goes in the ⇐ direction is called a su cient condition. Therefore, if a condition goes in both directions, we say it is a necessary and su cient condition. Note that our rst order condition for maxima or minima is a necessary condition, but not su cient.
Example:
Let f : R → R, f (x) = 2x^3 − 3 x^2. Then Df (x) = 6x^2 − 6 x = 6x(x − 1), which implies that the only candidates for a maximum or minimum are x = 0 and x = 1. Without further conditions, however, we cannot say whether these are actual maxima or minima.
Example:
Let F : R^2 → R, F (x, y) = x^3 − y^3 + 9xy. Then Df (x) = (3x^2 + 9y, − 3 y^2 + 9x), which implies that the only candidates for a maximum or minimum are when 3 x^2 + 9y = 0 and − 3 y^2 + 9x = 0. Solving the rst equation for y yields y = − 13 x^2. Plugging this into the other equation we have:
0 = − 3 y^2 + 9x = − 3
x^2
x^4 + 9x.
This equation can be re-written as:
x^4 + 9x = 27x − x^4 = x(27 − x^3 ),
which implies that x = 0 and x = 3 are possible solutions. Plugging the x solutions into the equation for y gives y = 0 and y = − 3 respectively. Therefore, the only possible optima are at (x, y) = (0, 0) and (x, y) = (3, −3). Without further conditions, however, we cannot say whether these are actual maxima or minima.
Second Order Conditions
The following theorem provides a su cient condition for nding local maxima or minima:
Let F : U → R be twice continuously dierentiable, where U is an open subset of Rn, and the rst order condition holds for some x∗^ ∈ U :
- If the Hessian matrix D^2 F (x∗) is a negative de nite matrix, then x∗^ is a strict local maximum of F.
- If the Hessian matrix D^2 F (x∗) is a positive de nite matrix, then x∗^ is a strict local minimum of F.
- If the Hessian matrix D^2 F (x∗) is an inde nite matrix, then x∗^ is neither a local maximum nor a local minimum of F.
Notice again, however, that this proof does not go both ways. For example, it is not true that all local minima have positive de nite Hessian matrices. For example, take the function f (x) = x^4 , which has a local minimum at x = 0, but its Hessian at x = 0 is D^2 f (x) = 0, which is not positive de nite. The following theorem provides weaker necessary conditions on the Hessian for a local maximum or minimum:
- Let F : U → R be twice continuously dierentiable, where U is an open subset of Rn, and x∗^ is a local maximum of F on U. Then DF (x∗) = 0, and D^2 f (x) is negative semide nite.
- Let F : U → R be twice continuously dierentiable, where U is an open subset of Rn, and x∗^ is a local minimum of F on U. Then DF (x∗) = 0 , and D^2 f (x) is positive semide nite.
According to the weaker necessary conditions, if we can nd x∗^ such that DF (x∗) = 0 and D^2 f (x) = 0 is either negative (or positive) semide nite, then that x∗^ is a candidate for a local maximum (or minimum). However, we cannot know for sure without further inspection.
Example:
Recall the function f : R → R, f (x) = 2x^3 − 3 x^2 has DF (x) = 0 when x = 0 or x = 1. We can calculate that D^2 F (x) = 12x − 6. When x = 0, then D^2 F (x) = − 6 which is negative de nite, so we can be sure that x = 0 is a local maximum. However, when x = 1, then D^2 F (x) = 6 which is positive de nite, so we can be sure that x = 1 is a local minimum.
Example:
Recall the function F : R^2 → R, F (x, y) = x^3 − y^3 + 9xy has DF (x, y) = 0 when (x, y) = (0, 0) or (x, y) = (3, −3). We can calculate that
D^2 F (x) =
6 x 9 9 − 6 y
When (x, y) = (0, 0), then
D^2 F (x) =
In this case, the rst order leading principal minor (the determinant of the matrix left after we delete the last row and column, or the determinant of the top left element) is 0 , and the second order principal minor (the determinant of the whole matrix) is − 81. Therefore, this matrix is inde nite, and (x, y) = (0, 0) is neither a maximum or minimum. When (x, y) = (3, −3), then D^2 F (x) =
In this case, the rst order leading principal minor is 18 , and the second order principal minor is 243. Therefore, this matrix is positive de nite, and (x, y) = (0, 0) is a strict local minimum.
