MODIFICATIONS OF THE CONTINUATION METHOD FOR THe . SOLUTION OF SYSTEMS OF NONLINEAR EQUATIONS

Modifications are proposed to the Davidenko-Broyden algorithm for the solution of a system of nonlinear equations. The aim of the modifications is to reduce the overall number of function evaluations by limiting the number of function evaluations for any one subproblem. To do this alterations are made to the strategy used in determining the subproblems to be solved. The modifications are compared with other methods for a wide range of test problems, and are shown to significantly reduce the number of function evaluations for the difficult problems. For the easier problems the modified method is equivalent to the Davidenko-Broyden algorithm.


i. INTRODUCTION
This paper considers the solution of a system of n nonlinear equations with n unknowns _x--(Xl,X2,...,Xn)T, i.e. the system f(x) O An essential requirement of numerical techniques for solving such a system is the existence of an initial approximation to the solution.In general the closer the initial approximation is to the solution the easier this solution is to obtain.
Davidenko [3] approached the problem of providing a good approximation to the solution of the above system by transforming the original problem into a sequence of subproblems.Although the solution of each of the subproblems involves the solution of a nonlinear equation system the aim of the Davidenko method is to provide a sufficiently good approximation to each to ensure the overall difficulty of the problem is reduced.Davidenko proposes a specific embedding or continuation which has the form g(_x,O) f(x) Of where f = f(o 0 is a scaler parameter and x the initial approximation to the solution.The variation of the parameter 0 provides the subproblems.In particular when 0 O g(x,O) f(_x) gives the original system and when 0 I then x is a solution of g(x,) 0 this provides the initial subproblem.We can now specify a sequence of sub- problems by taking values of 0 then the subproblems will be g(_x,Oi) 0 i 1,2,3, ,m The 8. should be chosen sufficiently close to one another so that the solution of one subproblem provides a good initial approximation for the next.
The major problem with this method is to make an effective choice of the 8. values which reduces the difficulty of the problem.Broyden[2] has studied 1 the combination of his quasi Newton method[l] with the method of Davidenko, the   quasi Newton method being used as the subproblem solver.In this paper Broyden suggested a specific method for choosing the 8 i values.Broyden chose 8 o I, 81 .99 and 82 .98 and fitted a quadratic approximation through the solution points of the latest three subproblems.Broyden obtains an estimate of the 8 value required for the next subproblem by extrapolating for 8 using the current quadratic approximation, the extent of the extrapolation being limited according to the local nonlinearity of the problem.
Broyden's [2] numerical studies showed that a significant improvement was obtained for harder problems by using this modified Davidenko technique with the quasi Newton method.However examination of the Davidenko-Broyden algorithm shows that for a number of problems a comparatively large proportion of the function evaluations arise from the solution of one or two subproblems indicating a poor choice of 8.The next section proposes modifications to the Davidenko-Broyden algorithm with the aim of reducing the number of function evaluations.

MODIFICATION OF THE CONTINUATION METHOD
The aim of the modification is to prevent the number of function evaluations for solving any subproblem becoming excessive.The Davidenko-Broyden continuation method, as discussed in the previous section, solves the sequence of subprob lems g(x, Si) 0 i l,...,m If in solving equation (2.1) for some 8r+I the number of function (2.1) evaluations reaches a value MAXK without convergence to the solution, then it suggests that the step from O r to Or+I is too large and the solution of (X, Sr+ l) 0 (2.2) should be terminated.We must now choose a revised 8 for which convergence may be obtained within MAXK iterations without wasting the effort expended in part solving (2.2).
The strategy proposed here is to compute a new value, 8*, from x*, the solution vector at the point of terminating the solution of (2.2).The sub- problem g(_x,8*) --0 is then solved, x_* being used as the initial approximation.
If x* is the solution vector of (2.2) after MAXK function evaluations, then as this is not the exact solution we can write g(__x*, 8) ! ( We wish to calculate a 8 such that the error vector is as small as possible.
That is, choose 8 so as to minimise n i-- i--i derivative 2S/82 2f_oT.f_o> O, so that S is a minimum when 8 has The second the value 8* (g.f__*)/(g.f_o).Or, by writing f__* g* + 8r+if_o where g* g(x*,Sr+l) this occurs then 8" is set equal to 8r+l' i.e. we continue with the solution of (2.2). 8" is also set equal 8r+l if the following condition on the norm of f(x*) is not satisfied.
If the condition (2.5) is satisfied then prior to the solving of the sub- problem g(x,8*) 0 a new initial approximation to the inverse Jacobian used by Broyden quasi Newton method is computed.In other words the computed value of 8" is only used if the solution vector x* is a better approximation to the solution of the nonlinear equations f(x) 0 than x the solution of the previous subproblem.If condition (2.5) is not satisfied or 8" is out of range then we continue with the solution of (2.2) with x__* as the starting vector for another MAXK function evaluation.
Once a solution of g(x, *) 0 has been found we proceed in the normal way and compute the next 8 from 8 8 and * and the solutions of the r-l' r corresponding subproblems.
In solving the text problems in the next section the value for MAXK is chosen to be 25.This is an arbitrary choice and there may be some other optimal choice for this parameter, probably problem dependent.However Swift and Lindfield [6] performed experiments with different MAXK values for single nonlinear equations and found that within a range the value of MAXK was not critical.

RESULTS AND DISCUSSION
In this section the proposed modification is compared with the unmodified algorithm of Broydenl] and the continuation method of Broyden [2].Also considered is the algorithm of Powell [5].This is a hybrid method using approximations to the Jacobian and inverse Jacobian to estimate a Newton step and solution is a linear combination of these two steps.This algorithm is included as it is currently in the Mark 5 release of the Numerical Algorithms Group (NAG) Library.
The test equations are now presented.Test problems I) 3) are those of Broyden [2], problems 2) and 3) being particularly difficult.The algebraic equations 4) and 5) again prove difficult to solve; problem 6) is the test example appearing in the NAG specification of the Powell algorithm [5].
Equations as in 2), initial values x I --i.0, also uses t I and the results correspond to the method of Broyden [2] where an initial approximation to the inverse Jacobian is computed before the start of each subproblem.The NAG routine of Powell is denoted by P and the proposed modified Davidenko-Broyden algorithm by MDB.
Comparison of the first two columns in Table I shows that the unmodified version of Broyden's algorithm and the NAG routine of Powell are very similar in their performance.Powell's algorithm being superior for problems 6), 7) and 8), but inferior for problem i).However, out of the eight test problems the NAG routine fails to produce solutions in five of them.The failures marked F in column P are not due to exceeding the maximum number of function evaluations but due to some other failure in the NAG routine.
The Davidenko-Broyden algorithm produces solutions to all but one of the problems.It fails for problem 7) but is very satisfactory for problem 8), which is the same problem but with different starting values.The DB algorithm tends to take more function evaluations than the Broyden algorithm for the easier problems, but is much better for .thedifficult problems.
The modified Davidenko-Broyden algorithm gives converged results for all the test problems.For the difficult problems 2), 4), 7) and 8), where the DB algorithm takes a large number of function evaluations, the modified method significantly reduces the number of function evaluations.For the easier problems it performs the sae as the DB algorithm.
An examination of the sequence of e values used by the Davidenko-Broyden algorithm in solving the four difficult problems, shows that only three sub- problems are solved for e values of 0.99, 0.98 and 0.O.In all cases the majority of function evaluations arise from the subproblem with O, as shown in Table II.
The modified method however, stops the solution of the subproblem with 8 0 after MAXK function evaluations and calculates a 8" close to zero and solves the subproblems g(x,8*) 0 as is shown in Table III for problems 4) and 8).
The value of 30 function evaluations appearing after the first attempt to solve the subproblem with 8 O, comprises 25 evaluations (MAXK) from the solving of the subproblem and (n+l) function evaluations from the computation of the new initial approximation to the inverse Jacobian before solving the sub- problem.
For problem 4) the 8" computed after the termination of the solution of g(x,O) 0 is out of range and so the solution of the subproblem with O 0 is continued after computing a new initial approximation to the inverse Jacobian.
This is sufficient for convergence to be attained after another 13 function evaluations.
It can be seen therefore that the modifications to the Davidenko-Broyden algorithm have the desired effect of spreading out the function evaluations among the subproblems and in reducing the total number of function evaluations.

TABLE I .
Number of function evaluations for solutions of the test problems.

TABLE III .
Sequence of subproblems for modifiedDavidenko-Broyden