A Modified Conjugate Gradient Method for Solving Large-Scale Nonlinear Equations

Solving nonlinear equations is an important problem which appears in various models of science and engineering such as computer vision, computational geometry, signal processing, computational chemistry, and robotics. More specifically, the subproblems in the generalized proximal algorithms with Bergman distances is a monotone nonlinear equations [1], and l1-norm regularized optimization problems can be reformulated as monotone nonlinear equations [2]. Due to its wide applications, the studies in the numerical methods for solving the monotone nonlinear equations have received much attention [3–10]. In this paper, we are interested in the numerical methods for solving monotone nonlinear equations with convex constraints:


Introduction
Solving nonlinear equations is an important problem which appears in various models of science and engineering such as computer vision, computational geometry, signal processing, computational chemistry, and robotics. More specifically, the subproblems in the generalized proximal algorithms with Bergman distances is a monotone nonlinear equations [1], and l 1 -norm regularized optimization problems can be reformulated as monotone nonlinear equations [2]. Due to its wide applications, the studies in the numerical methods for solving the monotone nonlinear equations have received much attention [3][4][5][6][7][8][9][10]. In this paper, we are interested in the numerical methods for solving monotone nonlinear equations with convex constraints: where F: R n ⟶ R n is a continuous function and S is a nonempty, closed, and convex set. e monotonicity of the mapping F means that (F(x) − F(y)) T (x − y) ≥ 0, ∀x, y ∈ R n .
e methods for solving monotone nonlinear equations (1) are closely relevant to the methods for solving the following optimization problems: min x∈R n f(x). (3) Notice that if f(x) is strictly convex, then ∇f(x) is strictly monotone which means (∇f(x)− ∇f(y)) T (x− y)≥0. It is well known that the strictly convex function exists a unique solution x * , satisfying ∇f(x * )�0. To sum up, if there is a convex function f(x) satisfying ∇f(x)�F(x), then solving the optimization problems (3) is equivalent to solving monotone nonlinear equations (1). So, a natural idea to solve monotone nonlinear equations (1) is to use the existing efficient methods for solving optimization problems (3). ere are many methods for solving optimization problems (3), such as the Newton method, quasi-Newton method, trust region method, and conjugate gradient method. Among these methods, the conjugate gradient method is a very effective method for solving optimization problems (3) due to their simplicity and low storage. A conjugate gradient method generates a sequence of iterates: where α k is the step length and direction d k is defined by where β k is a parameter and g k is the gradient of the objective function f(x). e choice of β k determines different conjugate gradient methods [11][12][13][14][15][16][17]. We are interested in the PRP conjugate gradient method in which the parameter β k is defined by where y k− 1 � g k − g k− 1 . Based on the idea of [18,19], Zhang et al. [20] proposed a new modified nonlinear PRP method in which β k is defined by where z k− 1 � max t‖d k− 1 ‖, ‖g k− 1 ‖ 2 , t > 0 and η > (1/4) are two constants. ere is a mistake about the definition of β NPRP k . By this definition, we will not be able to prove Lemma 1 in [20]. It should be ere are many conjugate gradient methods for solving nonlinear equations (1). Zhang and Zhou [4] proposed a spectral gradient method by combining the modified spectral gradient and projection method, which can be applied to solve nonsmooth equations. Xiao and Zhou [10] extended the CG-DESCENT method to solve large-scale nonlinear monotone equations and extended this method to decode a sparse signal in compressive sensing. Dai and Zhu [21] proposed a derivative-free method for solving large-scale nonlinear monotone equations and proposed a new line search for the derivative-free method. Other related works can be found [3, 5-8, 10, 22-30]. In this paper, we combined the projection method [3], the modified nonlinear PRP conjugate gradient method for unconstrained optimization [20] and the iterative method [10] and proposed a modified nonlinear conjugate gradient method for solving large-scale nonlinear monotone equations with convex constrains. is paper is organized as follows. In Section 2, we propose a modified nonlinear PRP method for solving monotone nonlinear equations with convex constraints. Under reasonable conditions, we prove its global convergence. In Section 3, we make some improvement to the proposed method and give the convergence theorem of the improved method. In Section 4, we do some numerical experiments to test the proposed methods. e results show that our methods are efficient and promising. Furthermore, we use the proposed methods to solve practical problems in compressed sensing.

A Modified Nonlinear PRP Method
In this section, we develop a modified nonlinear PRP method for solving the nonlinear equations with convex constraints. Based on the modified nonlinear PRP method [20], we now introduce our method for solving (1). Inspired by (8), we define d k as where where z k− 1 � max t‖d k− 1 ‖, ‖ F k− 1 ‖ 2 , t > 0 and η > (1/4) are two constants. e lemma below shows a good property of d k . e steps of the method are given in Algorithm 1.

Lemma 1.
Let d k be generated by Algorithm 1. If z k− 1 ≠ 0, then there exists a constant c > 0 such that Proof. For k � 0, we have For k ≥ 1, we obtain Denote en, we obtain Let c � − (1 − (1/4η)); then, inequality (11) is satisfied. Next, we establish the global convergence of the proposed method. Without specification, we always suppose that the solution set of equation (1) is nonempty and the following assumption holds.

Assumption 1
(i) e mapping F is Lipchitz continuous, and it means that the mapping F satisfies (ii) e projection operator P S [·] is nonexpansive, i.e., Lemma 2. Suppose that Assumption 1 holds and x * is a solution of (1), and the sequence x k and x k+1 are generated by Algorithm 1. en, the sequence x k , x k+1 , and F k are bounded.
Proof. We first show that x k is bounded. From the monotonicity of function F, we have It is easy to see that e last inequality implies It obviously that the sequences x k is bounded, i.e., there is a constant M > 0 such that At last, we prove that the sequence x k+1 is bounded. From (2) and (23) and Algorithm 1, we obtain So, the following inequality holds: Mathematical Problems in Engineering It implies that the sequence x k+1 is bounded.
□ Lemma 3. Suppose that Assumption 1 holds, and the sequence x k and F k is generated by Algorithm 1. en, there exists a constant c > 1 such that Proof. We first prove the right side of inequality (26). For k � 0, from (9), we have For k ≥ 1, by the definition of β NPRP k and (21), we obtain By the definition of d k (9) and the last inequality, we obtain Let c � 1 + (2LM(t + 2ηLM)/t 2 ); then, we have ‖d k ‖ ≤ c‖F k ‖. Now, we turn to prove the left side of the inequality. It follows from (11) that erefore, we have □ Lemma 4. Suppose Assumption 1 holds; then, the step length t k satisfies Proof. If the Algorithm 1 terminates in a finite number of steps, then there is a k ∈ R such that x k is a solution of equation (1) and ‖F k ‖ � 0. From now on, we assume that ‖F k ‖ ≠ 0, for any k. It is easy to see that d k ≠ 0 from (11). If t k ≠ ξ, by the line search process, we know that ρ − 1 t k does not satisfy Algorithm 1, that is, It follows from (11) and Assumption 1 that From the last inequality and Lemma 3, we obtain Hence, it holds that Proof. Noticed that Initial. Given a small constant ε > 0 and constants t > 0, ξ > 0, η > (1/4), σ, ρ ∈ (0, 1). Choose an initial point x 0 ∈ S. Let k � 0.
Step 3. Let t k � max ξρ i : , and P S [·] is a projection operator, defined by P S [x] � arg min ‖y − x‖: y ∈ S , ∀x∈R n .
Step 5. Let k � k + 1 and go to Step 1. ALGORITHM 1: Modified NPRP method. 4 Mathematical Problems in Engineering Since the function F(x) is continuous, and the sequence x k+1 is bounded, so the sequence ‖F(x k+1 )‖ is bounded.
at is, for all k ≥ 0, there exists a positive constant B > 0, such that ‖F(x k+1 )‖ ≤ B. en, we obtain So, we have Proof. Suppose that (41) does not hold; then, there exists ϵ > 0 such that, for any k ≥ 0, From (26) and the last inequality, it is easy to see From (41) and (42), we obtain e last inequality yields a contradiction with (37), so (41) is satisfied.

An Improvement
In this section, we make some improvement to the modified nonlinear PRP method proposed in Section 2. In Algorithm 1, we take the step length α k � (F( x k+1 ) T (x k − x k+1 ))/‖F(x k+1 )‖ 2 . Is there a better choice for α k ? is is our purpose to improve Algorithm 1. Under the condition of ensuring the convergence of the algorithm and the related good properties and results, we improve Algorithm 1 in order to get better numerical results.
From Algorithm 1, to make the inequality ‖x k+1 − x * ‖ ≤ ‖x k − x * ‖ hold, we only need to satisfy By solving the last inequality, we have It is easy to see that α k � (F( x k+1 ) T (x k − x k+1 ))/ ‖F(x k+1 )‖ 2 is the minimum point of the function ϕ(α). is is the reason why Algorithm 1 takes Under reasonable conditions, we hope to get a large step length than Algorithm 1. So, we obtain (47) Based on the above arguments, we propose an improved algorithm of Algorithm 1. In the improved algorithm, we make the step length: Similar to the proof of eorem 1, we have the following results.

Theorem 2. Suppose that Assumption 1 holds. e sequence x k is generated by Algorithm 2; then, we have
The iterative process of the improved method is stated as follows.

Numerical Results
In this section, we do some numerical experiments to test the performance of the proposed methods. We implemented our methods in MATLAB R2020b and run the codes on a personal computer with 2.3 GHz CPU and 16 GB RAM.
We first solve Problems 1 and 2.
Problem 1 (see [8]). e mapping F is taken as Problem 2 (see [4]). e mapping F is taken as and S � R n + .
e stopping criterion of the algorithm is set to ‖F(x k )‖ < 10 − 5 or the number of iteration reach to 500. e latter case means that the method is a failure for the test problems. We test both problems with the dimensions of variables n � 1000, 10000, 100000, 1000000, and 100000. Start from different initial points, and we list all results in Tables 1 and 2. We compared the performance of the proposed methods with the classical Newton method and an efficient algorithm CGD [10] in the total number of iterations as well as the computational time. e meaning of each column is given below.  Tables 1 and 2 show that our methods performs very well both in the number of iterations and CPU time. e IMNPRP performs best among these methods. It is worth noting that the number of iterations does not increase significantly as n increases. Hence, the proposed method is very suitable for solving large-scale problems. Because of the lack of memory, the dimension of the problems solved by the Newton method is no more than 100,000. Initial. Given a small constant ε > 0 and constants t > 0, ξ > 0, η > (1/4), σ, ρ ∈ (0, 1). θ ∈ [1,2]. Choose an initial point x 0 ∈ S. Let k � 0.
Step 3. Let t k � max ξρ i : i � 0, 1, 2, . . . satisfying Step 4. Compute Step 5. Let k � k + 1 and go to Step 1. ALGORITHM 2: e improved modified nonlinear PRP method. Mathematical Problems in Engineering e following example is a signal reconstruction problem from compressed sensing.
Problem 3 (see [10]). Consider a typical compressive sensing scenario, where we aim to reconstruct a length-n sparse signal form m observations (m ≪ n). In this test, the mea- where ω is the Gaussian noise distributed as N(0, σ 2 I) with σ 2 � 10 − 4 . e random A is the Gaussian matrix which is  0  200  400  600  800  1000  1200  1400  1600  1800  2000   0  200  400  600  800  1000  1200  1400  1600  1800  2000   0  200  400  600  800  1000  1200  1400  1600  1800  2000   0  200  400  600  800  1000  1200  1400  1600  1800  2000   0  50  100  150  200  250  300  350  400  450 where the value τ is forced to decrease as the measure in [31]. e iterative process starts at the measurement image, i.e., x 0 � A T b, and terminates when the relative change between successive iterates falls below 10 − 4 , i.e., where f k denotes the function value at iteration x k . By the discussion in [10], we know that the l 1 -norm problem can transformed a monotone nonlinear equation. Hence, it can be solved by Algorithms 1 and 2. Due to the storage limitations of the PC, we test a small size signal with n � 2 11 and m � 2 9 , and the original contains 2 6 randomly nonzero elements. e quality of restoration is measured by the mean of squared error (MSE) to the original signal x, that is, where x is the restored signal. We take the parameters ξ � 10, σ � 10 − 4 , and ρ � 0.5 in CGD, MNPRP, and IMNPRP. In order to test the effectiveness of the proposed methods, we compare the proposed methods with the CGD method [10] and the solver SGCS which is specially designed to solve monotone equations for recovering a large sparse signal in compressive sensing. e results are listed in Figures 1 and 2.
It can be seen from Figures 1 and 2 that all methods have recovered the original sparse signal almost exactly. Among these methods, the IMNPRP method performs best.

Conclusions
In this paper, a modified conjugate gradient method and its improved method are proposed for solving the large-scale nonlinear equations. Under some assumptions, global convergence of the proposed methods are established.

Mathematical Problems in Engineering
Numerical results show that the proposed methods are very efficient and competitive.

Data Availability
All data generated or analysed during this study are included within the article.

Conflicts of Interest
e authors declare that there are no conflicts of interest regarding the publication of this paper.