Download Newton's Method for Finding Minimum and Maximum: Univariate and Multivariate and more Study notes Agricultural engineering in PDF only on Docsity!
Lecture X
Finding the Minimum Using Newton’s Method
I. Finding the Univariate Minimum (Algorithm1.ma) A. A Sufficiently Complex Function
- It is obvious that finding the minimum of a simple univariate quadratic function is trivial given the rules we discussed in the preceding section. For example U x ( ) = 5 x^2 − 4 x + 2 has a minimum determined by its first derivative
U x x
x
x
In addition, straightforward transformations such as e^5 x^^^4 x^2
(^2) − +
offer little additional complexity
[ ]
U x x
U x x
x
- One possibility is the rational function
f x
x x x
2 .
f(x)
0
5
10
15
20
25
30
35
-5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10 11 12 13
However, these functions typically have bizarre discontinuities. For example, plotting the above function over a broader range indicates that
Professor Charles B. Moss
-208.
0
200
400
600
-19 -16 -13 -9 -6 -
(^0369121518)
If we restrict our attention to the range of x’s such that x is greater than -10 the problem becomes more tractable.
f x x
x x
x x x
x
x
2 2
B. Solving the Zero
- As I have previously stated, the trick to optimization is to find the zero of the gradient. Plotting the gradient of the rational function, we see g(x)
0
5
-5 -3 -1^1357911
- The Method of bisection
Professor Charles B. Moss
f ' ( ) x =
Thus, the Newton search points are
Search Point Function Value Derivative Step -8.0000 -130.5000 135.5000 0. -7.0369 -56.7315 41.6668 1. -5.6754 -23.9799 13.4022 1. -3.8861 -9.4998 4.7432 2. -1.8833 -3.2269 2.0272 1. -0.2914 -0.7503 1.1846 0. 0.3419 -0.0675 0.9800 0. 0.4108 -0.0007 0.9607 0. 0.4115 0.0000 0.9605 0.
Graphically, the search path can be depicted
Professor Charles B. Moss
Newton Search
- - -
0
50
100
150
-8 -6 -4 -2 0 2
Function Value Derivative
II. Finding the Multivariate Maximum A. The basic difference between univariate and multivariate optimization is the number of equations which we want to solve simultaneously for zero. In the multivariate case we want to solve ∇ (^) x f ( ) x = 0 where x is an n element vector, so we want to solve for n equations equal to zero. Appealing again to the second order Taylor series expansion
−
x x xx
xx x
f x f x f x x x
x x f x f x
2
2 1
which implies that
x t + x t ( xx f x t ) x f xt
− 1 =^ − ∇^2 ∇
1 ( ) ( ) B. A Simple Problem
- As a first problem, consider a Cobb-Douglas utility function with a budget constraint imposed max.^.^.^. x x x x x
st x x x x
1
2 2
3 3
4 4
1
1 +^2 2 +^3 3 +^4 =^100
By substitution, this problem becomes max.^.^ ( ).^. x x (^) 23 x (^) 34100 − 2 x (^) 2 − 3 x (^) 3 − x (^) 4 2 x 41 Starting with point x=(1,1,1), we have
