1.1.7 Example: Square Roots by Newton's Method

Permalink copied!

1.1.7

Example: Square Roots by Newton's Method

Functions, as introduced above, are much like ordinary mathematical functions. They specify a value that is determined by one or more parameters. But there is an important difference between mathematical functions and computer functions. Computer functions must be effective.

As a case in point, consider the problem of computing square roots. We can define the square-root function as

\[ \sqrt{x}\ =\text{ the }y\text{ such that }y \geq 0\text{ and } y^2\ =\ x \]

This describes a perfectly legitimate mathematical function. We could use it to recognize whether one number is the square root of another, or to derive facts about square roots in general. On the other hand, the definition does not describe a computer function. Indeed, it tells us almost nothing about how to actually find the square root of a given number. It will not help matters to rephrase this definition in pseudo-JavaScript:

function sqrt(x) { return the y $\texttt{with}$ y >= 0 && square(y) === x; }

This only begs the question.

The contrast between mathematical function and computer function is a reflection of the general distinction between describing properties of things and describing how to do things, or, as it is sometimes referred to, the distinction between declarative knowledge and imperative knowledge. In mathematics we are usually concerned with declarative (what is) descriptions, whereas in computer science we are usually concerned with imperative (how to) descriptions.

[1]

How does one compute square roots? The most common way is to use Newton's method of successive approximations, which says that whenever we have a guess $y$ for the value of the square root of a number $x$, we can perform a simple manipulation to get a better guess (one closer to the actual square root) by averaging $y$ with $x/y$.

[2]

For example, we can compute the square root of 2 as follows. Suppose our initial guess is 1:

\[ \begin{array}{lll} \textrm{Guess} & \textrm{Quotient} & \textrm{Average}\\[1em] 1 & {\displaystyle \frac{2}{1} = 2} & {\displaystyle \frac{(2+1)}{2} = 1.5} \\[1em] 1.5 & {\displaystyle \frac{2}{1.5} = 1.3333} & {\displaystyle \frac{(1.3333+1.5)}{2} = 1.4167} \\[1em] 1.4167 & {\displaystyle \frac{2}{1.4167} = 1.4118} & {\displaystyle \frac{(1.4167+1.4118)}{2} = 1.4142} \\[1em] 1.4142 & \ldots & \ldots \end{array} \]

Continuing this process, we obtain better and better approximations to the square root.

Now let's formalize the process in terms of functions. We start with a value for the radicand (the number whose square root we are trying to compute) and a value for the guess. If the guess is good enough for our purposes, we are done; if not, we must repeat the process with an improved guess. We write this basic strategy as a function:

function sqrt_iter(guess, x) {
    return is_good_enough(guess, x)
           ? guess
           : sqrt_iter(improve(guess, x), x);
}

A guess is improved by averaging it with the quotient of the radicand and the old guess:

function improve(guess, x) {
    return average(guess, x / guess);
}

where

function average(x, y) {
    return (x + y) / 2;
}

We also have to say what we mean by good enough. The following will do for illustration, but it is not really a very good test. (See exercise 1.7.) The idea is to improve the answer until it is close enough so that its square differs from the radicand by less than a predetermined tolerance (here 0.001):

[3]

function is_good_enough(guess, x) {
    return abs(square(guess) - x) < 0.001;
}

Finally, we need a way to get started. For instance, we can always guess that the square root of any number is 1:

function sqrt(x) {
    return sqrt_iter(1, x);
}

If we type these declarations to the interpreter, we can use sqrt just as we can use any function:

sqrt(9);

3.00009155413138

sqrt(100 + 37);

11.704699917758145

sqrt(sqrt(2) + sqrt(3));

1.7739279023207892

square(sqrt(1000));

1000.000369924366

The sqrt program also illustrates that the simple functional language we have introduced so far is sufficient for writing any purely numerical program that one could write in, say, C or Pascal. This might seem surprising, since we have not included in our language any iterative (looping) constructs that direct the computer to do something over and over again. The function sqrt_iter, on the other hand, demonstrates how iteration can be accomplished using no special construct other than the ordinary ability to call a function.

[4]

Exercise 1.6

Alyssa P. Hacker doesn't like the syntax of conditional expressions, involving the characters ? and :. Why can't I just declare an ordinary conditional function whose application works just like conditional expressions? she asks. Alyssa's friend Eva Lu Ator claims this can indeed be done, and she declares a conditional function as follows:

[5]

function conditional(predicate, then_clause, else_clause) {		    
    return predicate ? then_clause : else_clause;
}

Eva demonstrates the program for Alyssa:

conditional(2 === 3, 0, 5);

conditional(1 === 1, 0, 5);

Delighted, Alyssa uses conditional to rewrite the square-root program:

function sqrt_iter(guess, x) {
    return conditional(is_good_enough(guess, x),
                       guess,
                       sqrt_iter(improve(guess, x),
                                 x));
}

What happens when Alyssa attempts to use this to compute square roots? Explain.

Any call of sqrt_iter leads immediately to an infinite loop. The reason for this is our applicative-order evaluation. The evaluation of the return expression of sqrt_iter needs to evaluate its arguments first, including the recursive call of sqrt_iter, regardless whether the predicate evaluates to true or false. The same of course happens with the recursive call, and thus the function conditional never actually gets applied.

Exercise 1.7

The is_good_enough test used in computing square roots will not be very effective for finding the square roots of very small numbers. Also, in real computers, arithmetic operations are almost always performed with limited precision. This makes our test inadequate for very large numbers. Explain these statements, with examples showing how the test fails for small and large numbers. An alternative strategy for implementing is_good_enough is to watch how guess changes from one iteration to the next and to stop when the change is a very small fraction of the guess. Design a square-root function that uses this kind of end test. Does this work better for small and large numbers?

The absolute tolerance of 0.001 is too large when computing the square root of a small value. For example, sqrt(0.0001) results in 0.03230844833048122 instead of the expected value 0.01, an error of over 200%.

On the other hand, for very large values, rounding errors might make the algorithm fail to ever get close enough to the square root, in which case it will not terminate.

The following program alleviates the problem by replacing an absolute tolerance with a relative tolerance.

const relative_tolerance = 0.0001;
function is_good_enough(guess, x) {
    return abs(square(guess) - x) < guess * relative_tolerance;
}

Exercise 1.8

Newton's method for cube roots is based on the fact that if $y$ is an approximation to the cube root of $x$, then a better approximation is given by the value

\[ \begin{array}{lll} \dfrac{x/y^{2}+2y} {3} \end{array} \]

Use this formula to implement a cube-root function analogous to the square-root function. (In section 1.3.4 we will see how to implement Newton's method in general as an abstraction of these square-root and cube-root functions.)

function is_good_enough(guess, x) {
    return abs(cube(guess) - x) < 0.001;
}
function div3(x, y) {
     return (x + y) / 3;
}
function improve(guess, x) {
    return div3(x / (guess * guess), 2 * guess);
}
function cube_root(guess, x) {
    return is_good_enough(guess, x)
           ? guess
           : cube_root(improve(guess, x), x);
}

[1]

Declarative and imperative descriptions are intimately related, as indeed are mathematics and computer science. For instance, to say that the answer produced by a program is correct is to make a declarative statement about the program. There is a large amount of research aimed at establishing techniques for proving that programs are correct, and much of the technical difficulty of this subject has to do with negotiating the transition between imperative statements (from which programs are constructed) and declarative statements (which can be used to deduce things). In a related vein, programming language designers have explored so-called very high-level languages, in which one actually programs in terms of declarative statements. The idea is to make interpreters sophisticated enough so that, given what is knowledge specified by the programmer, they can generate how to knowledge automatically. This cannot be done in general, but there are important areas where progress has been made. We shall revisit this idea in chapter 4.

[2]

This square-root algorithm is actually a special case of Newton's method, which is a general technique for finding roots of equations. The square-root algorithm itself was developed by Heron of Alexandria in the first century CE. We will see how to express the general Newton's method as a JavaScript function in section 1.3.4.

[3]

We will usually give predicates names starting with is_, to help us remember that they are predicates.

[4]

Readers who are worried about the efficiency issues involved in using function calls to implement iteration should note the remarks on tail recursion in section 1.2.1.

[5]

As a Lisp hacker from the original Structure and Interpretation of Computer Programs, Alyssa prefers a simpler, more uniform syntax.

< Previous

Next >

1.1.7

Example: Square Roots by Newton's Method