Docsity
Docsity
Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity

Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips
Guidelines and tips
Sell on Docsity
Sell on Docsity
Docsity AI
Docsity AI
Log inSign up

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search for your university

Find the specific documents for your university's exams

Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents
download point icon20 Points

For each uploaded document

Answer questions
download point icon5 Points

For each given answer (max 1 per day)

All the ways to get free points
Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

Square Roots via Newton's Method, Lecture notes of Mathematics

Mathematics

2 Square roots. A classic algorithm that illustrates many of these concerns is “Newton's” method to compute square roots x =.

Typology: Lecture notes

2021/2022

Uploaded on 09/12/2022

techy
techy 🇺🇸

4.8

(9)

258 documents

1 / 3

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
Square Roots via Newton’s Method
S. G. Johnson, MIT Course 18.335
February 4, 2015
1 Overview
Numerical methods can be distinguished from other branches of analysis and computer science
by three characteristics:
•They work with arbitrary real numbers (and vector spaces/extensions thereof): the desired
results are not restricted to integers or exact rationals (although in practice we only ever
compute rational approximations of irrational results).
•Like in computer science (= math + time = math + money), we are concerned not only
with existence and correctness of the solutions (as in analysis), but with the time (and other
computational resources, e.g. memory) required to compute the result.
•We are also concerned with accuracy of the results, because in practice we only ever have
approximate answers:
–Some algorithms may be intrinsically approximate—like the Newton’s-method example
shown below, they converge towards the desired result but never reach it in a finite number
of steps. How fast they converge is a key question.
–Arithmetic with real numbers is approximate on a computer, because we approximate the
set Rof real numbers by the set Fof floating-point numbers, and the result of every
elementary operation (+,−,×,÷) is rounded to the nearest element of F. We need to
understand Fand how accumulation of these rounding errors affects different algorithms.
2 Square roots
A classic algorithm that illustrates many of these concerns is “Newton’s” method to compute square
roots x=√afor a > 0, i.e. to solve x2=a. The algorithm starts with some guess x1>0and
computes the sequence of improved guesses
xn+1 =1
2xn+a
xn.
The intuition is very simple: if xnis too big (>√a), then a/xnwill be too small (<√a), and
so their arithmetic mean xn+1 will be closer to √a. It turns out that this algorithm is very old,
dating at least to the ancient Babylonians circa 1000 BCE.1In modern times, this was seen to
1See e.g. Boyer, A History of Mathematics, ch. 3; the Babylonians used base 60 and a famous tablet (YBC 7289)
shows √2to about six decimal digits.
1
pf3

Partial preview of the text

Download Square Roots via Newton's Method and more Lecture notes Mathematics in PDF only on Docsity!

Square Roots via Newton’s Method

S. G. Johnson, MIT Course 18.

February 4, 2015

1 Overview

Numerical methods can be distinguished from other branches of analysis and computer science by three characteristics:

  • They work with arbitrary real numbers (and vector spaces/extensions thereof): the desired results are not restricted to integers or exact rationals (although in practice we only ever compute rational approximations of irrational results).
  • Like in computer science (= math + time = math + money), we are concerned not only with existence and correctness of the solutions (as in analysis), but with the time (and other computational resources, e.g. memory) required to compute the result.
  • We are also concerned with accuracy of the results, because in practice we only ever have approximate answers: - Some algorithms may be intrinsically approximate—like the Newton’s-method example shown below, they converge towards the desired result but never reach it in a finite number of steps. How fast they converge is a key question. - Arithmetic with real numbers is approximate on a computer, because we approximate the set R of real numbers by the set F of floating-point numbers, and the result of every elementary operation (+,−,×,÷) is rounded to the nearest element of F. We need to understand F and how accumulation of these rounding errors affects different algorithms.

2 Square roots

A classic algorithm that illustrates many of these concerns is “Newton’s” method to compute square roots x =

a for a > 0 , i.e. to solve x^2 = a. The algorithm starts with some guess x 1 > 0 and computes the sequence of improved guesses

xn+1 =

xn + a xn

The intuition is very simple: if xn is too big (>

a), then a/xn will be too small (<

a), and so their arithmetic mean xn+1 will be closer to

a. It turns out that this algorithm is very old, dating at least to the ancient Babylonians circa 1000 BCE.^1 In modern times, this was seen to (^1) See e.g. Boyer, A History of Mathematics, ch. 3; the Babylonians used base 60 and a famous tablet (YBC 7289) shows √ 2 to about six decimal digits.

be equivalent to Newton’s method to find a root of f (x) = x^2 − a. Recall that Newton’s method finds an approximate root of f (x) = 0 from a guess xn by approximating f (x) as its tangent line f (xn) + f ′(xn) (x − xn), leading to an improved guess xn+1 from the root of the tangent:

xn+1 = xn −

f (xn) f ′(xn)

and for f (x) = x^2 − a this yields the Babylonian formula above.

2.1 Convergence proof

A classic analysis text (Rudin, Principles of Mathematical Analysis) approaches the proof of con- vergence of this algorithm as follows: we prove that the sequence converges monotonically and is bounded, and hence it has a limit; we then easily see that the limit is

a. In particular:

  1. Suppose xn >

a, then it follows

a < xn+1 < xn:

(a) xn+1 − xn = 12 ( (^) xan − xn) = a−x^2 n 2 xn <^0. (b) x^2 n+1 − a = 14 (x^2 n + 2a + a

2 x^2 n^ )^ −^ a^ =^

1 4 (x

2 n −^2 a^ +^ a^2 x^2 n^ ) =^

1 4 (xn^ −^

a xn )

(^2) = (x^2 n−a)^2 4 x^2 n^ >^0 (regardless of whether xn >

a).

  1. A monotonic-decreasing sequence that is bounded below converges (Rudin theorem 3.14). If x 1 <

a, the second property above means that x 2 >

a; then for n > 2 it is monotonically decreasing and bounded below by

a.

  1. The limit x = limn→∞ xn satisfies x = 12 (x + ax ), which is easily solved to show that x^2 = a.

However, this proof by itself tells us nothing about how fast the sequence converges

2.2 Convergence example

Using the accompanying Julia notebook, we will apply this method to compute the most famous root of all,

  1. (Supposedly, the Greek who discovered that

2 is irrational was thrown off a cliff by his Pythagorean colleagues.). As a starting guess, we will use x 1 = 1, producing the following sequence when computed with about 60 digits of accuracy, where the correct digits are shown in boldface: 1 1. 1. 41 66666666666666666666666666666666666666666666666666666666675 1.41421 56862745098039215686274509803921568627450980392156862745 1.41421356237 46899106262955788901349101165596221157440445849057 1.41421356237309504880168 96235025302436149819257761974284982890 1.41421356237309504880168872420969807856967187537 72340015610125 1. Looking carefully, we see that the number of accurate digits approximately doubles on each iteration. This fantastic convergence rate means that we only need seven Newton iterations to obtain more than 60 accurate digits—the accuracy is quickly limited only by the precision of our floating-point numbers, a topic we will discuss in more detail later on.