We revisit the necessary and sufficient conditions for linear and high order of convergence of fixed point and Newton’s methods in the complex plane. Schröder’s processes of the first and second kind are revisited and extended. Examples and numerical experiments are included.
Natural Sciences and Engineering Research Council of CanadaInstitut des Sciences Mathématiques1. Introduction
In this paper, we revisit fixed point and Newton’s methods to find a simple solution of a nonlinear equation in the complex plane. This paper is an adapted version of [1] for complex valued functions. We present only proofs of theorems we have to modify compared to the real case. We present sufficient and necessary conditions for the convergence of fixed point and Newton’s methods. Based on these conditions we show how to obtain direct processes to recursively increase the order of convergence. For the fixed point method, we present a generalization of Schröder’s method of the first kind. Two methods are also presented to increase the order of convergence of the Newton’s method. One of them coincide with the Schröder’s process of the second kind which has several forms in the literature. The link between the two Schröder’s processes can be found in [2]. As for the real case, we can combine methods to obtain, for example, the super-Halley process of order 3 and other possible higher order generalizations of this process. We refer to [1] for details about this subject.
The plan of the paper is as follows. In Section 2, we recall Taylor’s expansions for analytic functions and the error term for truncated expansions. In Section 3 we consider the fixed point method and its necessary and sufficient conditions for convergence. These results lead to a generalization of the Schröder’s process of the first kind. Section 4 is devoted to Newton’s method. Based on the necessary and sufficient conditions, we propose two ways to increase the order of convergence of the Newton’s method. Examples and numerical experiments are included in Section 5.
2. Analytic Function
Since we are working with complex numbers, we will be dealing with analytic functions. Supposing g(z) is an analytic function and α is in its domain, we can write(1)gkz=∑j=0∞gk+jαj!z-αj,for any k=0,1,…. Then, for q=1,2,… we have(2)gkz=∑j=0q-1gk+jαj!z-αj+wgk,qzz-αq,where wgk,qz is the analytic function:(3)wgk,qz=∑j=0∞gk+q+jαq+j!z-αj.Moreover, the series for gkz and wgk,qz have the same radius of convergence for any k, and(4)wgk,qjα=j!q+j!gk+q+jαfor j=0,1,2,….
3. Fixed Point Method
A fixed point method use an iteration function (IF) which is an analytic function mapping its domain of definition into itself. Using an IF Φz and an initial value z0, we are interested by the convergence of the sequence zk+1=Φ(zk)k=0+∞. It is well known that if the sequence zk+1=Φ(zk)k=0+∞ converges, it converges to a fixed point of Φz.
Let Φz be an IF, p be a positive integer, and zk+1=Φzkk=0+∞ be such that the following limit exists: (5)limk→+∞zk+1-αzk-αp=Kpα;Φ.Let us observe that for p1<p<p2 we have (6)limk→+∞zk+1-αzk-αp1=0,limk→+∞zk+1-αzk-αp2=∞.We say that the convergence of the sequence to α is of (integer) order p if and only if Kpα;Φ≠0, and Kpα;Φ is called the asymptotic constant. We also say that Φz is of order p. If the limit Kpα;Φ exists but is zero, we can say that Φz is of order at least p.
From a numerical point of view, since α is not known, it is useful to define the ratio:(7)K~pα,k=zk+1-zk+2zk-zk+1p.
Following [3], it can be shown that(8)limk→+∞K~pα,k=Kpα;Φ,limk→+∞lnK~1α,k+1lnK~1α,k=p.
We say that α is a root of fz of multiplicity q if and only if fjα=0 for j=0,…,q-1, and fqα≠0. Moreover, α is a root of fz of multiplicity q if and only if there exists an analytic function wf,qz such that wf,qα=fqα/q!≠0 and fz=wf,qzz-αq.
We will use the big O notation gz=Ofz and the small o notation gz=ofz, around z=α, respectively, when c≠0 and c=0, when(9)limz→αgzfz=c.
For α a root of multiplicity q of fz, it is equivalent to write gz=Ofz or gz=Oz-αq. Observe also that if α is a simple root of fz, then α is a root of multiplicity q of fqz. Hence gz=Ofqz is equivalent to gz=O(z-α)q.
The first result concerns the necessary and sufficient conditions for achieving linear convergence.
Theorem 1.
Let Φz be an IF, and let Φ1z stand for its first derivative. Observe that although the first derivative is usually denoted by Φ′z, one will write Φ1z to maintain uniformity throughout the text.
If Φ1α<1, then there exists a neighborhood of α such that for any z0 in that neighborhood the sequence zk+1=Φzkk=0+∞ converges to α.
If there exists a neighborhood of α such that for any z0 in that neighborhood the sequence zk+1=Φzkk=0+∞ converges to α, and zk≠α for all k, then Φ1α≤1.
For any sequence zk+1=Φzkk=0+∞ which converges to α, the limit K1α;Φ exists and K1α;Φ=Φ1α.
Proof.
(i) By continuity, there is a disk Dρα=α∈C∣z-α<ρ such that wΦ,1z≤1+Φ1α/2=L<1. Then if zk∈Dρα, we have(10)zk+1-α=Φzk-Φα=wΦ,1zkzk-α≤Lzk-α≤zk-α<ρ,and zk+1∈Dρ(α). Moreover(11)zk-α≤Lkz0-α,and the sequence zk+1=Φzkk=0+∞ converges to α because 0≤L<1.
(ii) If Φ1α>1, there exists a disk Dρα, with ρ>0, such that wΦ,1z≥1+Φ1α/2=L>1. Let us suppose that the sequence zk+1=Φzkk=0+∞ is such that zk≠α for all k. If zk and zk+1∈Dρα, then we have (12)zk+1-α=Φzk-Φα=wΦ,1zkzk-α≥Lzk-α.Let 0<ϵ<ρ, and suppose zk,zk+1,…,zk+l are in Dϵα⊂Dρα. Because (13)zk+l-α≥Llzk-αeventually Ll+1zk-α≥ϵ and zk+l∉Dϵ(α). Then the infinite sequence cannot converge to α.
(iii) For any sequence zk+1=Φ(zk)k=0+∞ which converges to α we have(14)limk→+∞zk+1-αzk-α=limk→+∞wΦ,1zk=Φ1α.
For higher order convergence we have the following result about necessary and sufficient conditions.
Theorem 2.
Let p be an integer ≥2 and let Φz be an analytic function such that Φα=α. The IF Φz is of order p if and only if Φjα=0 for j=1,…,p-1, and Φpα≠0. Moreover, the asymptotic constant is given by (15)Kpα;Φ=limk→+∞zk+1-αzk-αp=Φpαp!.
Proof.
(i) The (local) convergence is given by part (i) of Theorem 1. Moreover we have(16)zk+1-α=Φzk-Φα=wΦ,pzkzk-αp,and hence(17)limk→+∞zk+1-αzk-αp=limk→+∞wΦ,pzk=Φpαp!=Kpα;Φ.
(ii) If the IF is of order p≥2, assume that Φjα=0 for j=1,2,…,l-1 with l<p. We have(18)zk+1-α=Φzk-Φα=wΦ,lzkzk-αl,where(19)wΦ,lα=limk→+∞wΦ,lzk=Φlαl!.But(20)wΦ,lzk=zk+1-αzk-αl=zk+1-αzk-αpzk-αp-l,and hence(21)wΦ,lα=limk→+∞wΦ,lzk=Kpα;Φlimk→+∞zk-αp-l=0ifl<p,Kpα;Φifl=p.So Φlα=0.
It follows that, for an analytic IF and p>2, the limit Kpα;Φ exists if and only if Klα;Φ=0 for l=1,…,p-1.
As a consequence, for an analytic IF Φz we can say that (a) Φz is of order p if and only if Φz=α+Oz-αp, or, equivalently, if Φα=α and Φ1z=Oz-αp-1, and (b) if α is a simple root of fz, then Φz is of order p if and only if Φz=α+Ofpz, or, equivalently, if Φα=α and Φ1z=Ofp-1z.
Schröder’s process of the first kind is a systematic and recursive way to construct an IF of arbitrary order p to find a simple zero α of fz. The IF has to fulfill at least the sufficient condition of Theorem 2. Let us present a generalization of this process.
Theorem 3 (see [<xref ref-type="bibr" rid="B7">1</xref>]).
Let α be a simple root of fz, and let c0z be an analytic function such that c0α=α. Let Φpz be the IF defined by the finite series:(22)Φpz=∑l=0p-1clzflz,where clz are such that (23)clz=-1l1f1zddzcl-1zfor l=1,2,… Then Φpz is of order p, and its asymptotic constant is (24)Kpα,Φp=Φpαp!=1pcp-11αf1αp-1=-cpαf1αp.
For c0z=z in (22), we recover the Schröder’s process of the first kind of order p [4–7], which is also associated with Chebyshev and Euler [8–10]. The first term c0z could be seen as a preconditioning to decrease the asymptotic constant of the method, but its choice is not obvious.
4. Newton’s Iteration Function
Considering c0z=z and p=2 in (22), we obtain (25)Φ2z=z-fzf1zwhich is Newton’s IF of order 2 to solve fz=0. The sufficiency and the necessity of the condition for high-order convergence of the Newton’s method are presented in the next result.
Theorem 4.
Let p≥2 and let Ψz be an analytic function such that Ψα=0 and Ψ1α≠0. The Newton iteration NΨz=z-Ψz/Ψ1z is of order p if and only if Ψjα=0 for j=2,…,p-1, and Ψpα≠0. Moreover, the asymptotic constant is(26)Kpα;NΨ=p-1p!ΨpαΨ1α.
Proof.
(i) If Ψjα=0 for j=2,…,p-1, and Ψpα≠0 we have(27)zk+1-α=zk-α-ΨzkΨ1zk=zk-αΨ1zk-ΨzkΨ1zk.But(28)Ψ1zk=Ψ1α+wΨ1,p-1zkzk-αp-1,Ψzk=Ψ1αzk-α+wΨ,pzkzk-αp.It follows that(29)zk+1-α=wΨ1,p-1zk-wΨ,pzkΨ1zkzk-αp,so(30)limk→+∞zk+1-αzk-αp=limk→+∞wΨ1,p-1zk-wΨ,pzkΨ1zk=Ψpα/p-1!-Ψpα/p!Ψ1α=p-1p!ΨpαΨ1α.
(ii) Conversely, if NΨz is of order p we have NΨjα=0 for j=1,…,p-1, and NΨpα≠0. Hence α is a root of multiplicity p-1 of NΨ1z and we can write (31)NΨ1z=wNΨ1,p-1zz-αp-1.We also have(32)Ψz=wΨ,1zz-α.But(33)NΨ1z=ΨzΨ2zΨ1z2,so we obtain(34)Ψ2z=NΨ1zΨ1z2Ψz=wNΨ1,p-1zwΨ,1zΨ1z2z-αp-2,where(35)limz→αwNΨ1,p-1zwΨ,1zΨ1z2=NΨpαΨ1αp-1!≠0.It follows that α is a root of multiplicity p-2 of Ψ2z. Hence Ψjα=0 for j=2,…,p-1, and Ψpα≠0.
We can look for a recursive method to construct a function Ψpz which will satisfy the conditions of Theorem 4. A consequence will be that NΨpz will be of order p, and NΨpz=α+Ofpz. A first method has been presented in [11, 12]. The technique can also be based on Taylor’s expansion as indicated in [13].
Theorem 5 (see [<xref ref-type="bibr" rid="B10">11</xref>]).
Let fz be analytic such that fα=0 and f1α≠0. If Fpz is defined by (36)F2z=fz,Fpz=Fp-1zFp-11z1/p-1forp≥3,then Fpα=0, Fp1α≠0, Fplα=0 for l=2,…,p-1. It follows that NFpz is of order at least p.
Let us observe that in this theorem it seems that the method depends on a choice of a branch for the p-1th root function. In fact the Newton iterative function does not depend on this choice because we have(37)NFpz=z-Fp-1z/Fp-11z1-1/p-1Fp-1zFp-12z/Fp-11z2=z-Fp-1z/Fp-11z1-1/p-11-Fp-1z/Fp-11z1.In fact the next theorem shows that a branch for the p-1th root function is not necessary.
Theorem 6 (see [<xref ref-type="bibr" rid="B9">12</xref>]).
Let Fpz be given by (36); one can also write (38)NFpz=z-fzf1z-1/p-1fzQp1z/Qpz=z-fzQpz/Qp+1z,where(39)Q2z=1,Qpz=f1zQp-1z-1/p-2fzQp-11zforp≥3.
Unfortunately, there exist no general formulae for NFpz and its asymptotic constant Kpα;NFp exists. However, the asymptotic constant can be numerically estimated with (7).
A second method to construct a function Ψpz which will satisfy the conditions of Theorem 4 is given in the next theorem.
Theorem 7 (see [<xref ref-type="bibr" rid="B7">1</xref>]).
Let α be a simple root of fz. Let Ψpz be defined by(40)Ψpz=∑l=0p-1dlzflz,where d0z and d1z are two analytic functions such that (41)d0α=0d01α+d1αf1α≠0,(42)dlz=-1lf1z×dl-11z+1l-1f1zdl-21z+l-1dl-1zf1z1for l=2,3,…. Then (43)NΨpz=z-ΨpzΨp1zis of order p, with(44)Ψppα=-p!dpαf1αp,Kpα;NΨp=-p-1dpαf1αpd01α+d1αf1α.
Let us observe that if we set Ψpz=Φpz-z with Φpz given by (22), then Ψpz verifies the assumptions of Theorem 7.
Remark 8.
For a given pair of d0z and d1z in Theorem 7, the linearity of expression (42) with respect to d0z and d1z for computing dlz’s allows us to decompose the computation for Ψpz in two computations, one for the pair d0z and d1z=0 and the other for the pair d0z=0 and d1z, and then add the two Ψpz’s hence obtained.
5. Examples
Let us consider the problem of finding the 3rd roots of unity: (45)αk=e2k-1πi/3fork=0,1,2,for which we have α3=1. Hence we would like to solve(46)fz=0,for(47)fz=z3-1.As examples of the preceding results, we present methods of orders 2 and 3 obtained from Theorems 3, 5, and 7. For each method, we consider also presenting the basins of attraction of the roots.
The drawing process for the basins of attraction follows Varona [14]. Typically for the upcoming figures, in squares 2.5,2.52, we assign a color to each attraction basin of each root. That is, we color a point depending on whether within a fixed number of iteration (here 25) we lie with a certain precision (here 10-3) of a given root. If after 25 iterations we do not lie within 10-3 of any given root we assign to the point a very dark shade of purple. The more there are dark shades of purple, the more the points have failed to achieve the required precision within the predetermined number of iteration.
5.1. Examples for Theorem <xref ref-type="statement" rid="thm3.3">3</xref>
We start with iterative methods of order 2. From Theorem 3, we first want c0α=α. We observe that the simplest such function is c0z=z. Such a choice has the advantage that derivative of higher order than 2 of this function c0z will be 0, thus simplifying further computation. This is in fact the choice of function c0z which leads to Newton’s method and Chebyshev family of iterative methods. We observe however that it is generally possible to consider different choices of functions, although most might be numerically convenient as we will illustrate here. We need c0α=α, in such we can also look at c0z=zaz where aα=1. In the examples that follow we will look at such functions az.
In Table 1, we have considered 3 functions of this kind. We have developed explicit expressions for fz=z3-1. Figure 1 presents different graphs for the basins of attraction for these methods. We observe that some of them have a lot of purple points.
Method of order 2 based on Theorem 3.
c0z=zaz
Φ2z
K2α;Φ2
zz3mm∈Z
3m+1-3m-2z3z3m-23
-3m+13m-22α2
zexpz3-1
1+5z3-3z63z2expz3-1
-132α2
zcosz3-1
2z3+13z2cosz3-1+zz3-1sinz3-1
112α2
Basins of attraction for methods of order 2 of Table 1.
For m=0, c0z=z and its asymptotic constant is α2
For m=1, c0z=z4 and its asymptotic constant is -2α2
For c0z=zez3-1, the asymptotic constant is -13/2α2
For c0z=zcosz3-1, the asymptotic constant is 11/2α2
Now let us consider method of order 3 with c0(z)=z3m+1 with m∈Z. In this case we obtain(48)Φ3z=z3m-2183m-23m-5z3-23m+13m-5+3m+13m-2z-3,and its asymptotic constant is (49)K3α;Φ3=3m+13m-23m-56α.Examples of basins of attraction are given in Figure 2 for m=0,1,2. The smallest asymptotic constant is for m=1.
Methods of order 3 for computing the cubic root with c0(z)=z3m+1 for M=0,1,2.
m=0 and its asymptotic constant is 5/3α (Chebyshev’s method)
m=1 and its asymptotic constant is -4/3α
m=2 and its asymptotic constant is 14/3α
5.2. Examples for Theorem <xref ref-type="statement" rid="thm4.2">5</xref>
Gerlach’s process described in Theorems 5 and 6 leads to Newton’s method for p=2 and Halley’s method for p=3. For our problem we have (50)NF2z=z-z3-13z2,NF3z=z-z3-1/3z21-1/21-z3-1/3z21=zz3+22z3+1.These methods are well known standard methods. For comparison, their basins of attraction are given in Figure 3.
First two methods for computing the third root with Theorem 5.
NF2z is Newton’s method
NF3z is Halley’s method
5.3. Examples for Theorem <xref ref-type="statement" rid="thm4.4">7</xref>
To illustrate Theorem 7, we set d0z=0 and d1z=zk for k∈Z, and let us consider methods of orders 2 and 3 to solve z3-1=0. Table 2 presents the quantities Ψpz, NΨpz, dpz, and Kpα;Ψp for p=2,3 for this example.
We observe that the asymptotic constant of the method of order 2 for k=-1 is zero; it means that this method is of an order of convergence higher than 2, and in fact it corresponds to Halley’s method which is of order 3. We observe that methods of order 3 for the values of k=-1 and k=2 both correspond to Halley’s method for our specific problem. Examples of basins of attraction are given in Figure 4 for methods of order 2 and in Figure 5 for methods of order 3 using values of k=-2,-1,0,1,2,3.
Methods of order 2 to illustrate Theorem 7.
k=-2 and d1(z)=z-2
k=-1 and d1(z)=z-1
k=0 and d1(z)=1
k=1 and d1(z)=z
k=2 and d1(z)=z2
k=3 and d1(z)=z3
Methods of order 3 to illustrate Theorem 7.
k=-2 and d1(z)=z-2
k=-1 and d1(z)=z-1
k=0 and d1(z)=1
k=1 and d1(z)=z
k=2 and d1(z)=z2
k=3 and d1(z)=z3
6. Concluding Remarks
In this paper we have presented fixed point and Newton’s methods to compute a simple root of a nonlinear analytic function in the complex plane. We have pointed out that the usual sufficient conditions for convergence are also necessary. Based on these conditions for high-order convergence, we have revisited and extended both Schröder’s methods of the first and second kind. Numerical examples are given to illustrate the basins of attraction when we compute the third roots of unity. It might be interesting to study the relationship, if there is any between the asymptotic constant and the basin of attraction for such methods.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work has been financially supported by an individual discovery grant from NSERC (Natural Sciences and Engineering Research Council of Canada) and a grant from ISM (Institut des Sciences Mathématiques).
DubeauF.GnangC.Fixed point and Newton's methods for solving a nonlinear equation: from linear to high-order convergenceDubeauF.Polynomial and rational approximations and the link between Schröder's processes of the first and second kindDubeauF.On comparisons of chebyshev-halley iteration functions based on their asymptotic constantsSchröderE.Ueber unendlich viele algorithmen zur auflösung der gleichungenSchröderE.StewartG. W.On Infinitely Many Algorithms for Solving EquationsTraubJ. F.HouseholderA. S.BodewigE.On types of convergence and on the behavior of approximations in the neighborhood of a multiple root of an equationShubM.SmaleS.Computational complexity. On the geometry of polynomials and a theory of costPetkovićM.HercegD.On rediscovered iteration methods for solving equationsGerlachJ.Accelerated convergence in Newton's methodFordW. F.PennlineJ. A.Accelerated convergence in Newton's methodDubeauF.On the modified Newton's method for multiple rootVaronaJ. L.Graphic and numerical comparison between iterative methods