Download Newton's Method - Algorithm Design I - Lecture Notes | CSCE 145 and more Study notes Computer Science in PDF only on Docsity!
1 Newton’s Method
Newton’s method is a very simple and at the same time very powerful method for finding roots of functions. It doesn’t always work, because no method can be said to “always” work. There are pathological cases for which the process of Newton’s method fails to converge to a root even though a root does exist, and there are cases for which Newton’s method will try to find a root when the function in fact has no root. Nonetheless, it’s usually a pretty good method to try, because it works a lot of the time, and because after all once one knows the value of a putative root one can in fact test that value to see if in fact it’s a root. In order to apply Newton’s method to the finding of roots of some function f (x), we must be able to compute f (x 0 ) for any arbitrary x 0 we happen to feed to the function, and we need to be able to compute the value of the derivative f ′(x 0 ) for any arbitrary x 0. The basic idea is this. Given a function f (x), choose some arbitrary value x 0 that we believe to be somewhere close to a root of the function. Compute f (x 0 ) and f ′(x 0 ) at this point. Now, from elementary calculus we know that the value f ′(x 0 ) is the slope of the tangent line at x 0. Define x 1 to be the point at which the tangent crosses the x-axis, so that we have f (x 1 ) = 0. This gives us two points on the tangent line, namely (x 0 , f (x 0 )) and (x 1 , f^ (x 1 )) = (x 1 , f^ (x 1 )), so we have the equation
Δy Δx
= f ′(x 0 ) =
f (x 0 ) − f (x 1 ) x 0 − x 1
f (x 0 ) − 0 x 0 − x 1
We now simply solve the equation
f ′(x 0 ) =
f (x 0 ) x 0 − x 1
to get
x 1 = x 0 −
f (x 0 ) f ′(x 0 )
Now, what can we say about this? If we believe that f (x) is a reasonably well-behaved function, then the tangent is a reasonable approximation to the function itself, and therefore the intercept x 1 at which the tangent line has a zero is perhaps not very far from a point where f (x) has a zero.
What we do operationally for Newton’s method is to start with the func- tion f (x) and its derivative, choose a point x 0 , and then iterate
xn+1 = xn −
f (xn) f ′(xn)
in hopes that we will be getting closer and closer to a root of f (x). The computer program that does Newton’s method reads somewhat as follows
xOld = infinity xNew = somenumber while(abs(xOld - xNew) > epsilon) { xOld = xNew xNew = xOld - f(xOld)/fPrime(xOld) }
Notice that it takes a little sleight of hand to get this synchronized right, so that the “new” value of the previous iteration becomes the “old” value of the current iteration, and yet the test at the top of the loop starts out correctly. Notice also the termination condition. With the bisection method for finding roots, we knew that a root existed in the interval we were testing; the approximation to the root was the midpoint of that interval; and the interval was being cut in half with each iteration. Under those conditions, we could state precisely an upper bound for the difference between the approximation to the root and the actual root. With Newton’s method, in contrast, we don’t actually know that there’s a root, and we don’t actually know how far from that root we might be. All we really know is that the algorithm is supposed to be getting closer and closer to what Newton’s method will think is a root, so our termination condition has to be that if successive differences have gotten smaller than some tolerance ε, we might as well quit.
1.1 The Example of Square Roots
I had discussed a Newton method square root algorithm earlier this semester. If Newton’s Method is in fact Newton’s Method, then the general version of Newton’s method ought to behave the same as the previous square root
xNew xOld f (xOld) f ′(xOld) 0.75 2.0 5.0 4. -0.29166666666666674 0.75 1.5625 1. 1.5684523809523803 -0.29166666666666674 1.0850694444444444 -0. 0.4654406117285619 1.5684523809523803 3.4600428713151907 3. -0.841530602630985 0.4654406117285619 1.2166349630462578 0. 0.1733901559391644 -0.841530602630985 1.7081737551644687 -1. -2.7969749740686076 0.1733901559391644 1.0300641461766078 0. -1.2197229272382335 -2.7969749740686076 8.823069005566088 -5. -0.1999322995161441 -1.2197229272382335 2.487724019230605 -2. 2.4008803928468465 -0.1999322995161441 1.0399729243898133 -0.
What is happening, of course, is that there is no root to this function, but the process of the Newton method is perfectly happy to keep computing something forever.
1.4 Example
A more complicated function is
f (x) = x^4 − 10 x^3 + 35x^2 − 50 x + 24
which has roots at x = 1. 0 , 2. 0 , 3. 0 , 4 .0. Newton’s method will find these roots, but the different roots that are found depend on the starting point, and it’s not even the case that the root that is found is the root nearest to the starting point.
1.5 Example
Finally, let’s look at f (x) = x^2 + 3. Now, we know that it will have no roots, but if we apply Newton’s method, we get something somewhat worse. We have f ′(x) = 2x. If we start with x 0 = 1.0, then we find x 1 = − 1 .0. That’s not a problem. But then we get x 2 = 1.0, and so on forever.
