A Derivative-Free Liu–Storey Method for Solving Large-Scale Nonlinear Systems of Equations

In this paper, a descent Liu–Storey conjugate gradient method is extended to solve large-scale nonlinear systems of equations. Based on certain assumptions, the global convergence property is obtained with a nonmonotone line search. The proposed method is suitable to solve large-scale problems for the low-storage requirement. Numerical experiment results show that the new method is practically effective.


Introduction
Consider the following nonlinear system of equations: where (F: R n ⟶ R n ) has continuous partial derivatives. In the past few decades, Newton's methods have been widely used to solve problem (1) for their fast convergence speeds, see [1][2][3][4][5][6][7][8][9][10]. Newton's methods determine the search direction by solving a linear system of equations, in which the coefficient matrix is the Jacobian matrix of F or an approximation of it. It is expensive to compute the coefficient matrix and solve the linear system of equations at each iteration. erefore, Newton's methods are not suitable for solving a large-scale problem in which the Jacobian matrix of F is unavailable or needs massive storage space. To overcome this shortcoming of Newton's methods, in this paper, we attempt to develop a numerical algorithm based on a descent Liu-Storey conjugate gradient method and a nonmonotone line research. Recently, La Cruz and Raydan [11] designed the Spectral Algorithm for Nonlinear Equations (SANE) to solve (1) and analyzed the convergence property based on a modified Grippo-Lampariello-Lucidi (GLL) [12] nonmonotone line search. However, at each iteration, SANE needs to compute a directional derivative or an approximation of it. To improve the efficiency of the SANE algorithm, La Cruz et al. [13] proposed the derivative-free SANE (DF-SANE) algorithm by using a derivative-free nonmonotone line search method. Numerical experiment results show that the DF-SANE is efficient for solving large-scale problems. Similar to the DF-SANE method, Cheng and Li [14] developed the N-DF-SANE method for solving large-scale nonlinear systems of equations by combining the spectral residual method and the Zhang-Hager [15] nonmonotone line search. e global convergence of the method was established based on some mild conditions.
If the continuously differentiable mapping F: R n ⟶ R n is the gradient of f: R n ⟶ R, then (1) is the first-order necessary optimality condition of the optimization problem min f(x), x ∈ R n , (2) namely, we can get the solution of equation (1) by solving optimization problem (2). erefore, it is reasonable to extend the numerical algorithms for solving (2) to solve large-scale nonlinear systems of equations. In this paper, we develop a derivative-free method for solving (1) based on a descent nonlinear conjugate gradient method.
Recently, Li and Feng proposed a modified Liu-Storey conjugate gradient (MLS) method [24]. A good property of the MLS method is that, with any line search method, it can always generate descent search directions for objective functions. When the strong Wolfe line search is used, Li and Feng proved that their method is convergent globally. e numerical algorithm for solving nonlinear systems of equations has been widely studied in the last decades.
ere are some numerical algorithms for nonlinear systems based on conjugate gradient methods [29][30][31][32][33][34][35]. Yuan and Zhang [29] proposed a numerical method for large-scale nonlinear systems of equations based on a three-term Polak-Ribière-Polyak conjugate gradient method and the hyperplane projection strategy given by Solodov and Svaiter [8]. Abubakar and Kumam [32] improved this three-term method by combining the idea of Yan et al. [36]. Yuan et al. [30] also extended the Hestenes-Stiefel conjugate gradient method to solve large-scale nonlinear equations and establish its global convergence. Bala and Kumam [31] proposed a numerical method for nonlinear systems by combining a descent Dai-Liao conjugate gradient and hyperplane projection method. Yu [34] extended the PRP method to solve large-scale nonlinear systems of equations based on the Grippo-Lampariello-Lucidi (GLL) [12] and Li-Fukushima (LF) [5] nonmonotone line search.
Inspired by the CG-type methods for the nonlinear system of equations, in this paper, we attempt to extend the MLS conjugate gradient method to solve problem (1) for the sufficient descent property, global convergence, and excellent numerical performance. To guarantee the global convergence of the new method, we determine the step length by a nonmonotone line search method which is a modification of the line search in [14,15].
Our paper is organized as follows: in the second section, we introduce the algorithm. In Section 3, we analyze the global convergence of the proposed method. Some numerical experiment results are reported and analyzed in the last section.
roughout this paper, we denote the Euclidean norm of vectors by ‖ · ‖, and J(x) means the Jacobian matrix of F at x.

Algorithm
In the remainder of this paper, we let Based on the analysis in Section 1, we now extend the MLS method [24] to solve (1), and the steps of the algorithm are stated as follows.
Step 3: nonmonotone line search step: If then set x k+1 � x k + α k d k , and go to Step 4.
then set x k+1 � x k − α k d k , and go to Step 4. Else, set (α k � ρ α k ), and go to Step 3.
e line search is the modification of the nonmonotone line search of Cheng and Li [14]. Define e following lemma shows that if τ k ∈ [0, 1], the parameter C k lies between f k and A k . Lemma 1. Suppose that f(x k ) C k and A k are infinite sequences generated by Algorithm 1. en, (f(x k ) ≤ C k ≤ A k ) for all k ≥ 0, and We omit the proof of Lemma 1 since it is similar to Lemma 2.2 in [14]. It is clear that Step 3 will be finished in a finite number of iterations for k � 0 since C 0 � f(x 0 ) and f is continuously differentiable. Suppose that the line search step finishes in a finite number of iterations for index k. en, we get from the line search conditions that 2 Mathematical Problems in Engineering From the definition of Q k and C k , we have is means condition (7) or (8) will be satisfied when α k is small enough. erefore, the algorithm is well defined. e formula for direction d k is based on the MLS method proposed by Li and Feng [24]. From the definition of d k , we have that if F k ≠ 0 and F T k−1 d k−1 ≠ 0 for some k ≥ 1, then where the inequality follows from an upper bound for the second term of the third equation, which is obtained from Obviously, inequality (14) also holds for k � 0. erefore, the search direction d k will always satisfy is means that the scalar β MLS k in (6) is well defined as long as (15). Moreover, Algorithm 1 does not need to calculate the Jacobian matrix of F and occupy low-memory space; therefore, it is suitable for large-scale problems. For convenience, we call this algorithm as the DF-MLS method.

Convergence Property
In this section, we devote to analyze the global convergence of our method under the following assumptions.

Lemma 2. Suppose that x k is an infinite sequence generated by the DF-MLS method and Assumption 1 holds. en,
Proof. Based on the line search step of the algorithm, we have Combining with (11), we have e above inequality draws the conclusion.

Lemma 3.
Suppose that x k is an infinite sequence generated by the DF-MLS method and Assumption 1 holds. en, Proof. We get from the third step of the algorithm that Combining this with (9), we have By (4), we have Since 0 < τ min ≤ τ k ≤ τ max < 1 and Q 0 � 1, we get from (9) that is together with (24) implies which gives (21). is completes the proof.

Theorem 1. Suppose that x k is an infinite sequence generated by the DF-MLS method and Assumption 1 holds. en,
where J(x * ) is the Jacobian matrix of F(x) at x * .
Proof. Let x * be a limit point of x k and K be an infinite sequence of indices such that If lim k∈K α k ≠ 0, then there exists an infinite subsequence K 1 ⊆K such that α k is bounded away from zero for any k ∈ K 1 . erefore, in this case, we get from (38) that is implies ‖F(x * )‖ � 0, and (35) holds since F is continuous and lim k∈K 1 x k � x * . Next, we analyze the case when Assume that (35) does not hold; then, there exists an infinite subsequence K 2 ⊆ K such that ‖F k ‖ is bounded away from zero for any k ∈ K 2 . From the third step of the DF-MLS method, the step length ρ − 1 α k will satisfy neither (7) nor (8) when k ∈ K 2 is large enough. is means We get from the conclusion of Lemma 1 that C k ≥ f(x k ) ≥ 0. Combining this with (41), we have It follows from Lemma 2 that Combining this with (18) and (28), we obtain where By the mean value theorem and (3), there exists a parameter ξ k ∈ (0, 1) such that namely, Combining this with (21), (29), and (40) and taking limits in (48), we obtain Using (42) and proceeding in a similar way, we can get From the last two inequalities, we get (36) which completes the proof.

Corollary 1. Suppose that x k is an infinite sequence generated by the DF-MLS method and Assumption 1 holds. If
x * is a limit point of x k and y T J x * y ≠ 0, ∀y ∈ R n , y ≠ 0, then F x * � 0. Proof. We draw the conclusion straightforwardly by using eorem 1.

Mathematical Problems in Engineering
If the Jacobian matrix of F(x) is positive definite for all x ∈ R n , we say that the mapping F is strictly monotone, namely, the Jacobian matrix J(x) satisfies If mapping F(x) is strictly monotone and F(x) � 0 has a solution, its solution must be unique. See Chapter 5 of [37]. Proof. We draw the conclusion straightforwardly by using eorem 1.

Numerical Results
In this section, we report the numerical experimental results and analyze the efficiency of our method. We compare the numerical performance of our method with some other methods such as the DF-SANE method [13], the N-DF-SANE method [14], and the DF-SCGNE method [34]. We used the aforementioned methods to find the stationary points of the test problems by using their gradients only. We tested 112 unconstrained optimization problems in the CUTEr library [38] with dimensions varying from 50 to 10,000. We ran two versions of some problems for which the dimensions can be chosen randomly.
We downloaded the DF-SANE code from Professor Raydan's home page at http://kuainasi.ciens.ucv.ve/ mraydan/mraydan_pub.html. ese four methods were coded in Fortran 77 and carried out on a personal computer with 2.8 GHz CPU processor and 2 GB RAM and Linux operating system. We terminated the program when where e a � 10 − 5 and e r � 10 − 4 . Table 1 lists the detailed experimental results, which include the problem names (Prob), dimensions (Dim), the total number of iterations (Iter), function evaluations (Nfun), and the CPU time (Time) in seconds, respectively. e symbol "-" means that the number of iterations exceeds 1000, or the line search iterates more than 100 times. In other words, this means that the method failed to solve that problem.
We compared the numerical performance of the methods by using the profiles designed by Dolan and Moré [39]. Figures 1 and 2 describe the numerical performance of the methods relative to the CPU time (in seconds), the total number of iterations, and function value evaluations, respectively. e curves in the figures have the following meanings.
For each method, the corresponding curve in Figure 1 plots the fraction P of problems for which the method is within a factor τ of the smallest total number of iterations. We observe from Figure 1 that DF-MLS solves 60.71% (68 out of 112) of the test problems with the smallest total number of iterations.
is may be because the DF-MLS method can generate more efficiently than other methods. We observe from Figures 1 and 2 that the performance of DF-MLS is better than that of N-DF-SANE and DF-SCGNE. However, the DF-MLS method requires more function evaluations and CPU time than DF-SANE. is might be because that the DF-MLS method needs an additional function evaluation to determine the initial step length α k 0 at each iteration. Figure 3 compares the performance of these four methods relative to the number of function evaluations. How to improve the efficiency of the DF-MLS method will be an interesting topic for us in the future.

Conclusion
Many practical applications give rise to nonlinear systems of equations. In the past few decades, Newton's methods [1][2][3][4][5][6][7][8][9][10] have been widely used to solve this problem for their fast convergence speeds. However, Newton's methods are not suitable for solving a large-scale problem in which the Jacobian matrix of the objective function is unavailable or needs massive storage space. To overcome this shortcoming of Newton's methods, some derivative-free numerical algorithms have been developed such as the SANE, DF-SANE, and some CG-type methods [11,13,[29][30][31][32][33][34].
Inspired by the CG-type methods, we design a derivative-free method for a large-scale nonlinear system of equations based on the MLS [24] conjugate gradient method. e line search in the proposed method is a modification of the Zhang-Hager [15] and Cheng-Li [14] nonmonotone line searches. We established the global convergence based on some mild conditions. e proposed method is suitable to solve large-scale problems because it does not need the Jacobian matrix of the objective function.
In the numerical experiment, we tested 112 unconstrained optimization problems in the CUTEr library [38] and compared the numerical performance of our method with that of DF-SANE [13], N-DF-SANE [14], and DF-SCGNE [34]. We used the methods to find the stationary points of the test problems by using their gradients only. e numerical results show that the proposed method is efficient and has good numerical stability.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that there are no conflicts of interest regarding the publication of this paper.