A Two-Point Newton Method suitable for non-convergent Cases and with Super-Quadratic Convergence

An iterative formula based on Newton Method alone is presented for the iterative solutions of equations that ensures convergence in cases where the traditional Newton Method may fail to converge to the desired root. In addition, the method has super quadratic convergence of order 2.414. Newton method is said to fail in certain cases leading to oscillation, divergence to increasingly large number or off-shooting away to another root further from the desired domain or off shooting to an invalid domain where the function may not be defined. In addition when the derivative at the iteration point is zero, Newton method stalls. In most of these cases, hybrids of several methods such as Newton, bisection and secant methods are suggested as substitute methods and Newton method is essentially blended with other methods or altogether abandoned. This paper argues that a solution is still possible in most of these cases by the application of Newton Method alone without resorting to other methods and with the same computational effort, two functional evaluations per iteration, like the traditional Newton method. In addition, the proposed modified formula based on Newton method has better convergence characteristics than the traditional Newton method.


Introduction
Iterative procedures for solutions of equations are routinely employed in many science and engineering problems. Starting with the classical Newton methods, a number of methods for finding roots of equations have come to exist each of which has its own advantages and limitations. The Newton method of root finding is based on the iterative formula: Newton's method displays a faster quadratic convergence near the root while it requires evaluation of the function and its derivative at each step of the iteration. However, when the derivative evaluated is zero, Newton method stalls. For low values of the derivative, the Newton iteration offshoots away from the current point of iteration and may possible converge to a root far away from the intended domain. For certain forms of equations, Newton method diverges or oscillates and fails to converge to the desired root. In addition, the convergence of Newton method can be slow near roots of multiplicity although modifications can be made to increase the rate of convergence [1].
Modification of the Newton method with higher order convergence have been proposed that require also evaluation of a function and its derivatives. An example of such methods is a third order convergence method by S. Weeraksoon and T.G. Fernando [2] that requires evaluation of one function and two first derivatives. A fourth order iterative method, accordintg to J.F. Traub [3] also requires evaluation of one function and two derivative evaluations. Sanchez and Barrero [4] gave a compositing of function evaluation at a point and its derivative to improve the convergence of Newton's method from 2 to 4. Recently other methods of fifth, sixth, seventh and higher order convergence have been proposed [5][6][7][8][9][10][11]. While higher order Newton methods ensure faster convergence, their stability for certain equation forms may have the same problem similar to that of the taditional Newton method.
The secant method does not require evaluation of derivatives. However, the rate of convergence is about 1.618 and the convergence may be a problem for some forms of equations and the secant method may fail to converge in those cases. Muller's method is an extension of the secant method to a quadratic polynomial [12]. It requires three functional evaluations to start with but continues with one function evaluation afterwards. The method does not require derivatives and the rate of convergence is about 1.84. However, Muller's method can converge to a complex root from an initial real number [13]. Muller's method also requires the three points to be distinct. If two of the points coincide, the method degenerates to secant method.
Hybrids of methods are also used to provide stability and ensure convergence to a desired root. For example Newton or Secant methods can be combined with bisection to bracket the root by a small interval so that a good initial guess is available for applying Newton method with quadratic convergence. Dekker's method [14] combines bisection method with that of secant method. The method starts by bracketing the root between two initial points that have functional values opposite in sign. The secant estimate of x is compared with the bisection of the interval and the one estimate resulting in small interval with the point of smaller magnitude functional value is chosen and the iteration continues. Brent's method [15] is a root finding algorithm that combines root bracketing, bisection and inverse quadratic interpolation. It is a modification of Dekker's method to avoid slow convergence when the difference between consecutive estimates of x is arbitrarily small. In such cases bisection is used for the next root estimate.
The Leap-frogging Newton method [16] uses the Newton method as an intermediate step for the estimation of the root followed by the application of the secant method using the initial point and the intermediate point found by Newton method. The method has cubic convergence and works in certain pathological cases where Newton method may fail. However, being dependent on Newton method for the intermediate step, the method may suffer from the same drawbacks of using the traditional Newton method mentioned above.

Method development
It will be shown that the iterative formula for the two point method will take the form; The method starts by selecting two points lying on a curve y= f(x), namely (x 0 , y 0 ) and (x 1 , y 1 ). A line connecting the two points is drawn from the point (x 0 , y 0 ) to the point (x 1 , y 1 ) as shown in Figure 1 below. A new variable m (the cotangent of the angle  1 between this line and the vertical) is defined so that: In general, for any other point (x, y) lying on the curve, the variable m is defined so that: Next, Newton Method of finding roots will be applied taking m as the independent variable and y as the dependent variable. Denoting that m r is the estimate of the root for which y (m) = 0 from the Newton method with the corresponding value of x r and applying Newton method gives; Substituting for m r and m 1 the equivalent expressions in terms of x and y values, namely, into Equation 2 above gives the following expression; .
The derivative dy/dm is evaluated from dy/dx = y' and dm/dx using the formula; The evaluation of the derivative dm/dx in turn gives; ( ) Therefore, dy/dm can now be written in terms of x, y and y' as follows; x y (x 0 , y 0 ) (x 1 , y 1 ) Figure 1 The graph of y = f(x) on which the angle  is defined between the two points.
Substituting the above expression for dy/dm in the equation, Solving for the root estimate x r and further rearranging results in; ( ) ( ) Continuing the iteration using the above formula, for the k th step of the iteration, the k+1 th estimate of the root will take the form; Denoting (y ky k-1 )/ (x k -x k-1 ) as (y/x) k will result in the expression;

Proof of super-quadratic convergence
Recalling the iteration formula of Equation 4 above: Defining the error at the k th iteration to be E k = X k -r where r is the root of the equation desired, the errors at the k-1 th and k th iteration are also defined similarly: The iteration formula in equation 4 above can now be rewritten in terms of the error terms as follows: Expanding the y k-1 , y k and y' k terms about the root r using Taylor Series expansion; Defining c = E k-1 and assuming that the convergence is of order n so that E k = E k-1 n and E k+1 = E k n , etc., both E k and E k+1 can now be written in terms of E k-1 = c; Substituting the right hand side of the above equation (Equation 7) in the iteration formula of Equation 5 above and simplifying one gets the following expression: Dividing all the expressions by c will leave the denominator with the dominant expression (y' r ) 2 since all the other expression in the denominator contain the error term c = E k-1 . Discarding the terms containing c in the denominator after division by c and further simplification will give:

( )
Equating the greatest error term in the right side, c 2n+1 , with that of will give the equation: Therefore, the iteration formula of Equation 1 near the root has a convergence of order 2.414 which is super quadratic.

Stability of the proposed method
The proposed method shows stability near points where application of the traditional Newton Method may result in oscillation, divergence or off-shooting away from the desired root or offshooting to a possibly invalid domain where the function y=f(x) may not be defined. This stability characteristics may be studied by examining the iteration formula again which takes the following form; Where the variable r is defined as; The estimate for the root at the k+1 th iteration, x k+1 , therefore, can be taken as the weighted sum of the x k-1 and x k values at the k-1 th and k th iteration respectively. The weighing factors are (1-1/r) for x k-1 and (1/r) for x k . When the method converges to the root the value of r approaches unity. This is found by examining the behavior of r near the root and noting that y k approaches 0 near the root, i.e.,

( ) ( )
The iterative formula will, near the root, approach the following expression; The iteration, therefore, moves away from x k-1 with a weighing factor approaching zero and gives weighing factor of 1 to x k .
Near points where the derivative of the function may approach zero, the r value approaches infinity and the root estimate weighs heavily in favor of x k-1 rather than x k . This can be seen from the following limit evaluation of r.
The iteration, therefore, moves away from x k with a weighing factor approaching zero and gives weighing factor of near to 1 to x k-1 Near points where the derivative of the function may approach zero, Newton method typically displays oscillation or off-shooting behavior. The proposed two-point modified Newton procedure however shows stability without displaying oscillation and off-shooting tendencies. For example, for the function y = sin (x), the derivative of y (i.e., dy/dx = cos(x)) is zero at x= /2. Starting Newton method near this point, say x = (/2 + 0.01), will result in off-shooting of the iteration to a different domain. As shown in Figure 2, for a starting point x= 1.58079633 where the function y= sin(x) has near zero derivative, the traditional Newton method off-shoots to a root further from the nearest root of x=0. The newton method as such converges to x=100.531. On the other hand, the proposed two point Newton method shows stability and converges to the nearest root of x=0.
Another example of the stability of the proposed method is where Newton method diverges instead converging to the desired root. Figure 3 and Figure 4 show for the proposed method and the traditional Newton method for the function y = x 1/3 which is often used to illustrate the pathological condition with respect to application of Newton's method which leads to an oscillating divergence to infinity. By contrast the proposed method displays an oscillating convergence to the desired root albeit with slower rate of convergence of order 1.0.

Application Examples
Equations used to test efficiency of root finding methods are used here to evaluate the number of iterations required to reach to a specified level of convergence (Table 1). The stopping criterion used for the iteration process is given by: The rate of convergence towards the root x = r for each step of the iteration is evaluated using the formula: Table 1 below shows comparison of the proposed two-point Newton method with those of Newton and secant methods for a number of equations used to test efficiency of root finding methods elsewhere. A super-quadratic convergence with which the proposed method converges to the root is mostly evident with a C k value being close to 2.414 during most of the iterations. It can also be seen from Table 1 that less number of iterations are required to reach convergence for the proposed method than those required for Newton and secant methods. For example, for equations with multiple roots such as , shown in Table 1, Newton method displays linear convergence while the proposed method converges super-quadratically with an order of 2.414. Similar order of convergence is also observed for the equation: .

Examples where the proposed method works while Newton method fails
The advantage of the use the proposed Two-Point Newton method is shown for cases where the Newton method and in several cases also the secant method fail to converge to the root. Table 2 Table 2. For example, in the case of y = Log(x), Newton method starting with x=3 fails with the second iteration because the estimated x value is a negative number whose logarithm is undefined. For the case y= arctan(x) Newton method always diverges to increasingly large number for any starting x value while the proposed two-point Newton method converges super qudratically near the root. For y= x 1/3 , Newton method also diverges whereby each iteration gives an estimate of the root which is twice the previous value and with alternate signs leading to oscillating divergence. On the other hand application of the proposed two-point modification of Newton method leads to oscillating convergence at somehow reduced rate of convergence (of order 1 as explained above).
For the equation , both Newton and secant methods continue to diverge to increasingly large x values while the proposed two-point Newton method shows stability of convergence. For the polynomials of different degrees cited in Table 2, Newton and secant methods display oscillation for the starting values shown in the Table 2 while the proposed Two-Point Newton method displays stability and super-quadratic convergence for iteration near the root.

Conclusion
A numerical procedure of root finding using two-point modification of Newton method has been presented. It is proved that the method has a super-quadratic convergence of order about 2.414. The method is based on application of Newton iteration formula by taking as the independent variable the cotangent of the angle between the line connecting the two successive points of iteration with the vertical and as the dependent variable the given function y = f(x). The resulting iteration formula for root estimation is shown to be the weighted sum of the estimates of the two previous iterations with a weighing factor that penalizes the iteration point having undesirable characteristics such as a near zero derivative. For example, near a point where the derivative is zero, the weighing factor for that point will be near zero effectively moving the iteration away from that undesirable point.
Application examples have been given that demonstrated that the proposed method requires fewer number of iterations for convergence to a root than the traditional Newton and secant methods. The method offers particular advantage for cases where the traditional Newton method and its variants of various order convergence may not converge. A number of examples are given where the proposed method converges to a root in a stable manner without oscillation, divergence or offshooting from the desired domain of the root whereas the Newton method, and in several of the cases also the secant method, display undesirable behaviors which prevent convergence to the desired root.
In terms of computational effort, the proposed method requires one function evaluation and one derivative evaluation at each step of the iteration except for the first step where two functional evaluations are required. The proposed method, therefore, requires essentially the same number of functional evaluations as the traditional Newton method while offering a super-quadratic convergence of order 2.414.