Notes on Newton's Method

Supplementary Material for Honors Calculus
Originally prepared in the Fall Semester 1995

Revised for the Web: July 27, 2004

Proposition: Let $f$ be a function that is differentiable on an interval $I$ and assume further:

(a) The derivative $f^{\prime }$ is either positive throughout $I$ or negative throughout $I$.

(b) The derivative $f^{\prime }$ is either steadily increasing in $I$ or steadily decreasing in $I$.

(c) There are points $a,b$ in $I$ for which $f\left(a\right)$ and $f\left(b\right)$ are both non-zero with opposite sign.

Then:

1. There is one and only one point $z$ in $I$ for which $f\left(z\right)=0$.

2. If $x$ is on the convex side of the graph of $f$ in $I$, then so is $x^{\prime }=x-f\left(x\right)⁄f^{\prime }\left(x\right)$, and $x^{\prime }$ lies between $x$ and $z$.

3. Successive iterations of Newton's method beginning with a point $x$ on the convex side of the graph of $f$ in $I$ will converge to $z$.

4. Error control principle. If $c$ is any point in $I$ on the concave side of the graph of $f$ and $x$ is on the convex side, then the distance between $x$ and $z$ is at most the absolute value of $f\left(x\right)⁄f^{\prime }\left(c\right)$.

Proof: If $f^{\prime }$ is positive in $I$ one has $f\left(x_1\right)<f\left(x_2\right)$ whenever $x_1<x_2$ in $I$. If instead $f^{\prime }$ is negative in $I$, then one has $f\left(x_1\right)>f\left(x_2\right)$ for $x_1<x_2$. For this reason there is at most one root $z$ in $I$ with $f\left(z\right)=0$. The Intermediate Value Theorem for Continuous Functions guarantees that there is at least one root between $a$ and $b$.

We shall assume that $f^{\prime }$ is positive and increasing. One may reduce each of the other three cases to this case by reflecting either in the horizontal axis or in the vertical line $x=z$ or both. Under this assumption the convex side of the graph of $f$ is the right side. Suppose that $z<x$: then $0=f\left(z\right)<f\left(x\right)$. We apply the Mean Value Theorem to $f$ on the interval $\left[z,x\right]$ to conclude that there is a number $u$ with $z<u<x$ for which $$f\left(x\right)-f\left(z\right)=f^{\prime }\left(u\right)\left(x-z\right)\text{.}$$ Since $f\left(z\right)=0$ and $f^{\prime }\left(u\right)>0$, one obtains $$x-z=\frac{f\left(x\right)}{f^{\prime }\left(u\right)}\text{.}$$ Since $f^{\prime }$ is increasing, we find $f^{\prime }\left(u\right)<f^{\prime }\left(x\right)$, and, therefore, $f\left(x\right)⁄f^{\prime }\left(x\right)<f\left(x\right)⁄f^{\prime }\left(u\right)$. Consequently, $z<x^{\prime }<x$.

In view of (2) one has $$z<\dots <x_n<\dots <x_2<x_1\text{.}$$ Letting $$x_{*}={\inf }_{\left(n\ge 1\right)}\left\{x_n\right\}\text{,}$$ one has $$x_{*}={\lim }_{n\to \infty }{x}_n\text{,}$$ and, therefore, taking the limit as $n\to \infty$ on both sides of the relation $$x_{n+1}=x_n-\frac{f\left(x_n\right)}{f^{\prime }\left(x_n\right)}\text{,}$$ one finds that $f\left(x_{*}\right)=0$. Since by (1) there is only one root of $f$ in $I$, it follows that $x_{*}=z$.

In the proof of (2) we saw that for $x$ in $I$ on the right side of $z$ the distance from $x$ to $z$ is $f\left(x\right)⁄f^{\prime }\left(u\right)$, where $z<u<x$. Since $c$ is on the concave side of the graph of $f$, i.e., $c<z$, we find also $c<u$, hence, $f^{\prime }\left(c\right)<f^{\prime }\left(u\right)$. Consequently, $$\mathrm{the\; error}=x-z=\frac{f\left(x\right)}{f^{\prime }\left(u\right)}\le \frac{f\left(x\right)}{f^{\prime }\left(c\right)}\text{.}$$