CONJUGATE GRADIENT ALGORITHM AND FRACTALS

This work is an extension of the survey on Cayley's problem in case where the conjugate gradient method is used. We show that for certain values of parameters, this method produces beautiful fractal structures.


Introduction
It is well known that the basins of attraction of different roots have fractal boundaries when Newton's method is used to determine the complex roots of polynomials.The simplest example was given by Cayley in 1879 [2]; it is the case for the equation (1.1) The works of Cayley have been taken then by the French mathematician Julia [6] to found a theory on sets that will carry his name.The survey of this type of sets has been thrown back there after 1975 by another French mathematician, Mandelbrot [7], with the help of computers and melting; that is what is known by the fractal geometry.In this work we are going to give another approach precisely to Cayley's problem; we are going to study the case where an algorithm of the generalized conjugate gradient (GCG) is used and one is going to show that the qualitative and geometric aspects of the basins of attraction for each solution of (1.1) depend on certain parameters, sometimes, driving to beautiful fractal structures.We consider the nonlinear equation where f is complex function with variable z.To solve (1.2), we can use the conjugate gradient method (CG) defined by a recurrent sequence of form and the search direction d k is defined by The case where f is linear, parameters (λ k ,β k ) are well defined and permit a convergence of the method (1.3)-(1.4) in few iterations.In the nonlinear case, the choice of these parameters is not obvious.The well-know formulas for β k are Fletcher-Reeves (FR) [4,5] and Plolak-Ribière (PR) [9], and they are given by ) where z denotes conjugate of z, Re(z) = (1/2)(z + z), and |z| 2 = z • z.The choice of step length λ k is delicate because it is obtained by solving the equation f (z k + λ k d k ) = 0, which is absurd.Therefore we prefer to take λ k as approximal solution of problem it is called an inexact line search [8], and several methods are proposed among them: Wolfe line search (W-L.S.) and given by: if one puts θ(λ) ≡ | f (z k + λd k )| 2 and θ (λ) ≡ (d/dλ)θ(λ), the step length λ k > 0 satisfies the following conditions: Note that in the case where f would be nonlinear, results of convergence (and counter example) are rare.It has been established in [9,10] the convergence of the two versions FR and PR with a chosen step length after solving exactly the problem (1.7).In [1] Al-Baali proved a global convergence of FR method with strong Wolfe line searches, that is, (1.9)

Notations and definitions
We denote by C the complex plan, R a real line, and M nonempty open subset of C n (n = 1,2,...), where C = C ∪ {∞}.
Let us consider the complex mapping where g : M → M is holomorphic function and β is complex parameter.
ω is attracting if (2.4) Definition 2.2.The basin of attraction Ꮽ(ω) of an attractive fixed point ω ∈ M, associated with a function g, is defined by Let ω ∈ M be an attractive fixed point of g, then the disk of convergence Ᏸ(z,β) associated with the map (2.1) is defined by In the literature [3], if g(z) = z 2 + β and M = C, the closure of Ᏸ(0,β) is not other than a Mandelbrot set and ∂Ꮽ(ω) is the Julia set, (∂Ꮽ denotes a boundary of Ꮽ).Definition 2.4.Let Ꮿ zI ,zS be a cone in C generated by the complex z I , and z S is defined by Ꮿ zI ,zS = νz I + ξz S ; ν > 0, ξ > 0 . (2.7)

Generalized conjugate gradient method (GCG)
Instead of choosing the reals (λ k ,β k ) at every iteration of the CG method, we fix the real or complex parameters, and the sequences (3.1) may be generalized in the following form: where (z k ,w k ,λ,β) ∈ C 4 , w 0 = 0, and thus (3.1) is called generalized conjugate gradient method (GCG).
For i = 1,2, we have then and thus as a condition to satisfy for convergence of GCG, however if λ > 0 (the case of a linear search), it is necessary that with z I = δ − iΔ and z S = δ + iΔ, where But, as we will see further the condition (3.9) is still not true.

Application to Cayley's problem
Cayley asked about the problem of characterization of the basins of attraction of zeroes for the cubic complex polynomial The three roots of the equation f (z) = 0 are Let us begin to analyze an equivalent problem that consists in minimizing the square of the following residual: To solve the problem (4.3), return to solve the following nonlinear equation: Apply the GC method on this last equation, we obtain the following sequence in C: with β 0 = 0, and for k > 0, we choose between β FR k and β PR k in accordance with (1.5) or (1.6), and λ k is obtained by line search.Results of numerical test (Figure 4.1) confirm the superiority of the PR method on FR method [7].Note that we can not apply the analysis for Section 3 to (4.5), because the complex function f • f is not holomorphic.On the other hand, | f | 2 is locally convex and the theory developed in [1,4,8,9] assures the local convergence.Now the residual is used only to do a line search, the sequence (4.5) will take the following form: In Figure 4.2, we show the basins of solutions of z 3 − 1 = 0 with the process (4.6) and β ≡ β k chosen making use FR or PR and Wolfe line search.If we choose β according to PR or FR and for any λ ≡ λ k > 0 arbitrary small (Figure 4.3), we remark that we can obtain a local convergence for (4.6) method.
The local convergence is possible with different line searches and varying β in the range (0,1) (Figure 4.4).
It is necessary to note that contrarily to the method (4.5),where there is coexistence of three basins Ꮽ(z 1 ), Ꮽ(z 2 ), and Ꮽ(z 3 ), in Figures 4.2    Take β = β x + i • β y , according to bifurcation diagram in the parametric plane (β x ,β y ), from GCG method, with or without line search, one can detect convergence areas for the three roots z 1 ,z 2 , and z 3 in a disk Ᏸ(z 0 ,β) centered in (0,0) and of radius strictly lower than 1 (Figures 4.8, 4.9), which confirms Proposition 3.1.
In Figure 4.12, we explore one of fractal structure's features that is the self-similarity.