Newton Raphson Method

The Newton-Raphson method (also known as Newton's method) is a way to quickly find a good approximation for the root of a real-valued function $f(x) = 0$. It uses the idea that a continuous and differentiable function can be approximated by a straight line tangent to it.

Suppose you need to find the root of a continuous, differentiable function $f(x)$, and you know the root you are looking for is near the point $x = x_0$. Then Newton's method tells us that a better approximation for the root is $$x_1 = x_0 - \frac{f(x_0)}{f'(x_0)}.$$ This process may be repeated as many times as necessary to get the desired accuracy. In general, for any $x$-value $x_n$, the next value is given by $$x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}.$$

Note: the term "near" is used loosely because it does not need a precise definition in this context. However, $x_0$ should be closer to the root you need than to any other root (if the function has multiple roots).

Here is a picture to demonstrate what Newton's method actually does:

We draw a tangent line to the graph of $f(x)$ at the point $x = x_n$. This line has slope $f'(x_n)$ and goes through the point $\big(x_n, f(x_n)\big)$. Therefore it has the equation $y = f'(x_n)(x - x_n) + f(x_n)$. Now, we find the root of this tangent line by setting $y = 0$ and $x=x_{n+1}$ for our new approximation. Solving this equation gives us our new approximation, which is $x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}$.

Find the root of the equation $x^2 - 4x - 7 = 0$ near $x = 5$ to the nearest thousandth.

---

We have our $x_0 = 5$. In order to use Newton's method, we also need to know the derivative of $f$. In this case, $f(x) = x^2 - 4x - 7$, and $f'(x) = 2x - 4$.

Using Newton's method, we get the following sequence of approximations:

$$\begin{aligned} x_1 &= 5 - \frac{5^2 - 4\times 5 - 7}{2\times 5 - 4} = 5 - \left(\frac{-2}{6}\right) = \frac{16}{3} \approx 5.33333\\ x_2 &= \frac{16}{3} - \frac{\left(\frac{16}{3}\right)^2 - 4\left(\frac{16}{3}\right) - 7}{2\left(\frac{16}{3}\right)-4} = \frac{16}{3} - \frac{\frac{1}{9}}{\frac{20}{3}} = \frac{16}{3} - \frac{1}{60} = \frac{319}{60} \approx 5.31667 \\ x_3 &= \frac{319}{60} - \frac{\left(\frac{319}{60}\right)^2 - 4\left(\frac{319}{60}\right) - 7}{2\left(\frac{319}{60}\right)-4} = \frac{319}{60} - \frac{\frac{1}{3600}}{\frac{398}{60}} \approx 5.31662. \end{aligned}$$

We can stop now, because the thousandth and ten-thousandth digits of $x_2$ and $x_3$ are the same. If we were to continue, they would remain the same because we have gotten sufficiently close to the root:

$$x_4 = 5.31662 - \frac{(5.3362)^2-4(5.3362)-7}{2(5.3362)-4} = 5.31662.$$

Our final answer is therefore 5.317. $_\square$

Newton's method may not work if there are points of inflection, local maxima or minima around $x_0$ or the root.

For example, suppose you need to find the root of $27x^3 - 3x + 1 = 0$ which is near $x = 0$.
The correct answer is $-0.44157265\ldots$ However, Newton's method will give you the following:

$$x_1 = \frac{1}{3}, x_2 = \frac{1}{6}, x_3 = 1, x_4 = 0.679, x_5 = 0.463, x_6 = 0.3035, x_7 = 0.114, x_8 = 0.473, \ldots.$$

This is very clearly not helpful. That's because the graph of the function around $x = 0$ looks like this:

As you can see, this graph has a local maximum, a local minimum and a point of inflection around $x = 0$. To see why Newton's method isn't helpful here, imagine choosing a point at random between $x = -0.19$ and $x = 0.19$ and drawing a tangent line to the function at that point. That tangent line will have a negative slope, and therefore will intersect the $y$-axis at a point that is farther away from the root.

In a situation like this, it will help to get an even closer starting point, where these critical points will not interfere.