Download The Secant Method: An Approximation Technique for Solving Equations and more Lecture notes Economic Analysis in PDF only on Docsity!
Jim Lambers MAT 772 Fall Semester 2010- Lecture 4 Notes
These notes correspond to Sections 1.5 and 1.6 in the text.
The Secant Method
One drawback of Newton’s method is that it is necessary to evaluate 푓 ′(푥) at various points, which may not be practical for some choices of 푓. The secant method avoids this issue by using a finite difference to approximate the derivative. As a result, 푓 (푥) is approximated by a secant line through two points on the graph of 푓 , rather than a tangent line through one point on the graph. Since a secant line is defined using two points on the graph of 푓 (푥), as opposed to a tangent line that requires information at only one point on the graph, it is necessary to choose two initial iterates 푥 0 and 푥 1. Then, as in Newton’s method, the next iterate 푥 2 is then obtained by computing the 푥-value at which the secant line passing through the points (푥 0 , 푓 (푥 0 )) and (푥 1 , 푓 (푥 1 )) has a 푦-coordinate of zero. This yields the equation
푓 (푥 1 ) − 푓 (푥 0 ) 푥 1 − 푥 0
which has the solution
푥 2 = 푥 1 −
which can be rewritten as follows:
푥 2 = 푥 1 −
This leads to the following algorithm.
Algorithm (Secant Method) Let 푓 : ℝ → ℝ be a continuous function. The following algorithm computes an approximate solution 푥∗^ to the equation 푓 (푥) = 0.
Choose two initial guesses 푥 0 and 푥 1 for 푘 = 1, 2 , 3 ,... do if 푓 (푥푘) is sufficiently small then 푥∗^ = 푥푘 return 푥∗ end 푥푘+1 = 푥푘−^1 푓푓 (푥^ (푥푘푘 )^ −)−푓 푥(푥푘푘^ 푓−^ (푥 1 푘)−^1 ) if ∣푥푘+1 − 푥푘∣ is sufficiently small then 푥∗^ = 푥푘+ return 푥∗ end end
Like Newton’s method, it is necessary to choose the starting iterate 푥 0 to be reasonably close to the solution 푥∗. Convergence is not as rapid as that of Newton’s Method, since the secant-line approximation of 푓 is not as accurate as the tangent-line approximation employed by Newton’s method.
Example We will use the Secant Method to solve the equation 푓 (푥) = 0, where 푓 (푥) = 푥^2 − 2. This method requires that we choose two initial iterates 푥 0 and 푥 1 , and then compute subsequent iterates using the formula
푥푛+1 = 푥푛 −
We choose 푥 0 = 1 and 푥 1 = 1.5. Applying the above formula, we obtain
푥 2 = 1. 4 푥 3 = 1. 41379310344828 푥 4 = 1. 41421568627451.
As we can see, the iterates produced by the Secant Method are converging to the exact solution 푥∗^ =
2, but not as rapidly as those produced by Newton’s Method. □ We now prove that the Secant Method converges if 푥 0 is chosen sufficiently close to a solution 푥∗^ of 푓 (푥) = 0, if 푓 is continuously differentiable near 푥∗^ and 푓 ′(푥∗) = 훼 ∕= 0. Without loss of generality, we assume 훼 > 0. Then, by the continuity of 푓 ′, there exists an interval 퐼훿 = [푥∗−훿, 푥∗+훿] such that 3 훼 4
It follows from the Mean Value Theorem that
푥푘+1 − 푥∗^ = 푥푘 − 푥∗^ − 푓 (푥푘)
The Bisection Method
Suppose that 푓 (푥) is a continuous function that changes sign on the interval [푎, 푏]. Then, by the Intermediate Value Theorem, 푓 (푥) = 0 for some 푥 ∈ [푎, 푏]. How can we find the solution, knowing that it lies in this interval? The method of bisection attempts to reduce the size of the interval in which a solution is known to exist. Suppose that we evaluate 푓 (푚), where 푚 = (푎 + 푏)/2. If 푓 (푚) = 0, then we are done. Otherwise, 푓 must change sign on the interval [푎, 푚] or [푚, 푏], since 푓 (푎) and 푓 (푏) have different signs. Therefore, we can cut the size of our search space in half, and continue this process until the interval of interest is sufficiently small, in which case we must be close to a solution. The following algorithm implements this approach.
Algorithm (Bisection) Let 푓 be a continuous function on the interval [푎, 푏] that changes sign on (푎, 푏). The following algorithm computes an approximation 푝∗^ to a number 푝 in (푎, 푏) such that 푓 (푝) = 0.
for 푗 = 1, 2 ,... do 푝푗 = (푎 + 푏)/ 2 if 푓 (푝푗 ) = 0 or 푏 − 푎 is sufficiently small then 푝∗^ = 푝푗 return 푝∗ end if 푓 (푎)푓 (푝푗 ) < 0 then 푏 = 푝푗 else 푎 = 푝푗 end end
At the beginning, it is known that (푎, 푏) contains a solution. During each iteration, this algo- rithm updates the interval (푎, 푏) by checking whether 푓 changes sign in the first half (푎, 푝푗 ), or in the second half (푝푗 , 푏). Once the correct half is found, the interval (푎, 푏) is set equal to that half. Therefore, at the beginning of each iteration, it is known that the current interval (푎, 푏) contains a solution. The test 푓 (푎)푓 (푝푗 ) < 0 is used to determine whether 푓 changes sign in the interval (푎, 푝푗 ) or (푝푗 , 푏). This test is more efficient than checking whether 푓 (푎) is positive and 푓 (푝푗 ) is negative, or vice versa, since we do not care which value is positive and which is negative. We only care whether they have different signs, and if they do, then their product must be negative. In comparison to other methods, including some that we will discuss, bisection tends to converge rather slowly, but it is also guaranteed to converge. These qualities can be seen in the following result concerning the accuracy of bisection.
Theorem Let 푓 be continuous on [푎, 푏], and assume that 푓 (푎)푓 (푏) < 0. For each positive integer 푛, let 푝푛 be the 푛th iterate that is produced by the bisection algorithm. Then the sequence {푝푛}∞ 푛= converges to a number 푝 in (푎, 푏) such that 푓 (푝) = 0, and each iterate 푝푛 satisfies
2 푛^
It should be noted that because the 푛th iterate can lie anywhere within the interval (푎, 푏) that is used during the 푛th iteration, it is possible that the error bound given by this theorem may be quite conservative.
Example We seek a solution of the equation 푓 (푥) = 0, where
푓 (푥) = 푥^2 − 푥 − 1.
Because 푓 (1) = −1 and 푓 (2) = 1, and 푓 is continuous, we can use the Intermediate Value Theorem to conclude that 푓 (푥) = 0 has a solution in the interval (1, 2), since 푓 (푥) must assume every value between −1 and 1 in this interval. We use the method of bisection to find a solution. First, we compute the midpoint of the interval, which is (1 + 2)/2 = 1.5. Since 푓 (1.5) = − 0 .25, we see that 푓 (푥) changes sign between 푥 = 1.5 and 푥 = 2, so we can apply the Intermediate Value Theorem again to conclude that 푓 (푥) = 0 has a solution in the interval (1. 5 , 2). Continuing this process, we compute the midpoint of the interval (1. 5 , 2), which is (1.5 + 2)/2 = 1 .75. Since 푓 (1.75) = 0.3125, we see that 푓 (푥) changes sign between 푥 = 1.5 and 푥 = 1.75, so we conclude that there is a solution in the interval (1. 5 , 1 .75). The following table shows the outcome of several more iterations of this procedure. Each row shows the current interval (푎, 푏) in which we know that a solution exists, as well as the midpoint of the interval, given by (푎 + 푏)/2, and the value of 푓 at the midpoint. Note that from iteration to iteration, only one of 푎 or 푏 changes, and the endpoint that changes is always set equal to the midpoint.
푎 푏 푚 = (푎 + 푏)/ 2 푓 (푚) 1 2 1.5 − 0. 25 1.5 2 1.75 0. 1.5 1.75 1.625 0. 1.5 1.625 1.5625 − 0. 12109 1.5625 1.625 1.59375 − 0. 053711 1.59375 1.625 1.609375 − 0. 019287 1.609375 1.625 1.6171875 − 0. 0018921 1.6171875 1.625 1.62109325 0. 0068512 1.6171875 1.62109325 1.619140625 0. 0024757 1.6171875 1.619140625 1.6181640625 0. 00029087
if 푓 (푎) ⋅ 푓 (푐) < 0 then 푏 = 푐 else 푎 = 푐 end end
Example We use the Method of Regula Falsi (False Position) to solve 푓 (푥) = 0 where 푓 (푥) = 푥^2 −2. First, we must choose two initial guesses 푥 0 and 푥 1 such that 푓 (푥) changes sign between 푥 0 and 푥 1. Choosing 푥 0 = 1 and 푥 1 = 1.5, we see that 푓 (푥 0 ) = 푓 (1) = −1 and 푓 (푥 1 ) = 푓 (1.5) = 0.25, so these choices are suitable. Next, we use the Secant Method to compute the next iterate 푥 2 by determining the point at which the secant line passing through the points (푥 0 , 푓 (푥 0 )) and (푥 1 , 푓 (푥 1 )) intersects the line 푦 = 0. We have
푥 2 = 푥 0 −
Computing 푓 (푥 2 ), we obtain 푓 (1.4) = − 0. 04 < 0. Since 푓 (푥 2 ) < 0 and 푓 (푥 1 ) > 0, we can use the Intermediate Value Theorem to conclude that a solution exists in the interval (푥 2 , 푥 1 ). Therefore, we compute 푥 3 by determining where the secant line through the points (푥 1 , 푓 (푥 1 )) and 푓 (푥 2 , 푓 (푥 2 )) intersects the line 푦 = 0. Using the formula for the Secant Method, we obtain
Since 푓 (푥 3 ) < 0 and 푓 (푥 2 ) < 0, we do not know that a solution exists in the interval (푥 2 , 푥 3 ). However, we do know that a solution exists in the interval (푥 3 , 푥 1 ), because 푓 (푥 1 ) > 0. Therefore, instead of proceeding as in the Secant Method and using the Secant line determined by 푥 2 and 푥 3 to compute 푥 4 , we use the secant line determined by 푥 1 and 푥 3 to compute 푥 4. □
