A New Conjugate Gradient Projection Method for Convex Constrained Nonlinear Equations

)e conjugate gradient projection method is one of the most effective methods for solving large-scale monotone nonlinear equations with convex constraints. In this paper, a new conjugate parameter is designed to generate the search direction, and an adaptive line search strategy is improved to yield the step size, and then, a new conjugate gradient projection method is proposed for large-scale monotone nonlinear equations with convex constraints. Under mild conditions, the proposed method is proved to be globally convergent. A large number of numerical experiments for the presented method and its comparisons are executed, which indicates that the presented method is very promising. Finally, the proposed method is applied to deal with the recovery of sparse signals.


Introduction
Solving a system of nonlinear equations can be transformed as an optimization problem, which is widely applied in many fields of sciences and engineering, for instance, the economic equilibrium problem [1], the neural networks problem [2], the financial problem [3,4], the chemical equilibrium system [5], and the compressed sensing problem [6,7].
In this paper, the following system of constrained monotone nonlinear equations is considered: where C is a nonempty closed convex set of R n and F: R n ⟶ R n is a monotone mapping, namely, (F(x) − F(y)) T (x − y) ≥ 0, ∀x, y ∈ R n .
In this work, we concentrate on studying CGPM for large-scale nonlinear equations with convex constraints. We aim to establish a more efficient CGPM by improving the search direction and the line search rule and use the proposed CGPM to deal with the problem of sparse signals recovery. e contributions of this article are listed as follows: (i) To guarantee the search direction satisfying the sufficient descent condition and trust region property independent of any line searches, a new conjugate parameter is proposed; (ii) Based on classic line searches for nonlinear equations, an adaptive line search is improved to seek suitable step size easily; (iii) Under general assumptions, the convergence analysis of the proposed algorithm is proved; (iv) e reported numerical experiments show that our method is promising for solving large-scale nonlinear constrained equations and handling the problem of recovering sparse signals. e remainder of this paper is organized as follows. In Section 2, a new search direction and an adaptive line search are proposed, and the corresponding algorithm is given. e global convergence is studied in Section 3. In Section 4, the numerical experiments for the proposed algorithm and its comparisons are performed, and the corresponding results are reported. Application of the proposed algorithm in compressed sensing is introduced in Section 5. In Section 6, a conclusion for this work is given.

A New CGPM Algorithm
e CGPM has been attracting extensive attention, since it not only inherits the advantages of the conjugate gradient method (CGM) with a simple algorithm structure, rapid convergence, and low storage requirements but also uses no any jacobian information of the equation in practice. To the best of our knowledge, the computation cost of CGPM mainly exists in the process of generating the search direction and computing the step size. erefore, in the following part, we design our search direction by the CGM and give an improved inexact line search to yield the step size.

e New Search Direction Yielded by CGM.
It is wellknown that the search direction of the classical CGM is generated by which is decided by the conjugate parameter β k . Usually, a different β k leads to a different search direction.
In the recent years, many scholars have made efforts to extend the CGM to solve the large-scale nonlinear monotone equations system. For example, based on the hybrid conjugate parameter β JHJ k in [34], Sun and Liu [28] proposed a modified conjugate parameter as follows: where y k− 1 � F k − F k− 1 and extended it to solve problem (1).
Recently, Tsegay et al. [35] gave a new CGM with sufficient descent property, that is, Inspired by [28,35], we propose a new hybrid formula as follows: where ] > (1/2). Clearly, the parameter β LJJ k satisfies It is interesting that the search direction with β LJJ k have better theoretical properties, that is, it satisfies the sufficient descent condition and trust region property, automatically.

An Improved Adaptive Line Search Strategy.
For an efficient CGPM, choosing an inexpensive line search is a key technique. To this end, many researchers try to exploit an inexact line search strategy to obtain step size with minimal cost. Zhang and Zhou [18] adopt an Armijo-type line search procedure, that is, the step size α k � max βρ i | i � 0, 1, 2, · · · , such that where β is an initial guess for the step size, ρ ∈ (0, 1), and σ is a positive constant. Li and Li [31] obtained the step size α k � max βρ i | i � 0, 1, 2, · · · by the following line search: which was originally proposed by Solodov and Svaiter [36]. Guo and Wan [26] proposed an adaptive line search, i.e., the step size α k � max βρ i | i � 0, 1, 2, · · · satisfied the following inequality: Obviously, when x k is far from the solutions of problem (1) and the ‖F(x k + α k d k )‖ is too large, it follows from (9) that the step size α k becomes small, which increases the computation cost. A similar case can appear for the line search (8) when x k is close to the solution set of problem (1) and ‖d k ‖ is too large. However, it is worth noting that the line search (10) can overcome the previously mentioned weaknesses and take advantage of line searches (8) and (9).
Here, μ is a disturbance factor, which can adjust the size of the right side of the line search (11) and further reduce the computation cost.
Remark 1. In fact, for a given μ, ‖F(x k + α k d k )‖ is too large if x k is far away from the solution set, namely, ((‖F(x k + α k d k )‖)/(max ‖F(x k + α k d k )‖, μ )) � 1, and then, the new line search (11) is similar to (8) in performance. Otherwise, when x k is close to the solution set, ‖F(x k + α k d k )‖ approaches 0 and so c k approaches ‖F(x k + α k d k )‖, and then, the new line search (11) comes back to (9).

Algorithm.
Based on the new search direction with β LJJ k (6) and adaptive line search (11), we describe our algorithm in detail for solving problem (1) as follows (Algorithm 1).

Convergence Property
In order to obtain some important properties and convergence property of Algorithm 1, the following basic assumptions are necessary.
Assumption H: (H1) e solution set of system (1), denoted by S, is nonempty, and the mapping F is monotone on R n . (H2) e mapping F is L-Lipschitz continuous on R n , i.e., there exists a constant L > 0 such that e well-known nonexpansive property of the projection operator [37] is reviewed in the following lemma.
Lemma 1 (see [37]). Let Ω ⊆ R n be a nonempty closed convex set. en, erefore, the projection operator P is L-Lipschitz continuous on R n . e following lemma shows that the search direction yielded by equation in step 2 in Algorithm 1 satisfies the sufficient descent condition and possesses some important properties.

Lemma 2. Suppose that Assumption H holds, then the search direction generated by equation in step 2 in Algorithm 1 satisfies the sufficient descent condition,
and τ‖F k ‖ ≤ ‖d k ‖ ≤ τ‖F k ‖, for some positive constants τ and τ.
Proof. For k � 0, it is easy to know that F T 0 d 0 � − ‖F 0 ‖ 2 and Lemma 2 holds. To proceed, we consider the case k ≥ 1.
Otherwise, multiplying both sides of equation in step 2 in Algorithm 1 by F T k , from (6) and (7), we have which shows that the sufficient descent property (14) holds by taking τ ≔ (1 − (1/2])). Again, according to the following relation holds: and then, ‖d k ‖ ≥ τ‖F k ‖.
On the other hand, it follows from (7) and equation in step 2 in Algorithm 1 that and the proof is completed. e next lemma not only indicates that the line search strategy (11) is well-defined but also provides a lower bound for step size α k .

□
Proof (i) Suppose that for any nonnegative integer i, (11) does not hold at the k 0 -th iterate,then Complexity 3 From the continuity of F and ρ ∈ (0, 1), let i ⟶ ∞, and it is clear that which contradicts (14). e proof is completed. (ii) For the second part, it is clear that if α k � β, then (19) holds. If α k ≠ β, α k is computed by the backtracking process in Algorithm 1. Let α k ≔ ρ − 1 α k , and then, α k does not satisfy (11), namely, where z k � x k + α k d k . It follows from (12), (14), and (22) that en, which completes the proof. e following lemma is necessary to analyze the global convergence of Algorithm 1. □ Lemma 4. Suppose that Assumptions H holds, let x k and z k be generated by Algorithm 1, and let x * be any given solution for system (1), i.e., x * ∈ S. en, the sequence ‖x k − x * ‖ is convergent, and sequences x k and z k are both bounded. Furthermore, it holds that Proof. In view of the definition of z k and (11), we know that On the other hand, taking Assumption (H1) and x * ∈ S into consideration, we have According to equation in step 4 in Algorithm 1, Assumption (H1), Lemma 1,and (26) and (27), it follows that which shows that the inequalities 0 ≤ ‖x k+1 − x * ‖ ≤ ‖x k − x * ‖ hold, that is, the sequence ‖x k − x * ‖ is monotone nonincreasing and bounded below. Hence, ‖x k − x * ‖ is convergent. Furthermore, the boundedness of x k is obtained. By Lemma 2, it holds that d k is bounded and so is z k . Without the loss of generality, there exists a constant M > 0 such that Hence, the following relation holds: Step 0: given x 0 ∈ C, the constants ε > 0, σ, ρ ∈ (0, 1), μ ≥ 1, ] > (1/2) and an initial stepsize β. Set k ≔ 0 Step 1: if a given terminate criterion is satisfied, e.g., ‖F(x k )‖ ≤ ε, terminate. Otherwise, go to Step 2 Step 2: and yield the conjugate parameter β LJJ k by (6) Step 3: compute the step size α k using the adaptive line search strategy (11) and set z k � x k + α k d k Step 4: compute x k+1 by Set k ≔ k + 1, and return to Step 1 is together with (28) implies that us, this further implies lim k⟶∞ α k ‖d k ‖ � lim k⟶∞ ‖x k − z k ‖ � 0, and the proof is completed.
Next, based on Assumption H and Lemmas 1-4, the global convergence of the proposed algorithm is established. □ Theorem 1. Suppose that Assumption H holds and the sequences x k be generated by the Algorithm 1; then, Furthermore, the whole sequence x k converges to a solution of system (1).
Proof. First, by contradiction, suppose that relation (31) is not true; then, there exists a constant ε > 0 such that Again, the following inequality comes directly from Lemma 2: is together with (25) shows that In addition, from Lemmas 2 and 3 (ii) and the boundedness of ‖d k ‖ , the following relation holds: which contradicts (34). erefore, (31) is true.
Second, (31) shows that there exists an infinite index set K such that lim k∈K ‖F k ‖ � 0. Again, the x k is bounded and C is a closed set, so without the loss of generality, suppose that lim k∈K x k � x∈ C. It follows from the continuity of F that which shows that x∈ S. Finally, noticing that x * ≔ x from Lemma 4, it follows that the sequence ‖x k − x‖ is convergent, namely, which implies that the whole sequence x k converges to x∈ S, and the proof is completed.

Numerical Experiments
In this section, the numerical performances of Algorithm 1 (LJJ CGPM) for solving convex constrained nonlinear equations are tested and reported by the following two subsections.

Experimental Setup.
In order to illustrate the effectiveness of the LJJ CGPM, we compare it with two recent CGPMs. Specifically, eight large-scale examples are solved by the LJJ CGPM method, PDY method [22], and ATTCGP method [23] in the same calculating environment. All codes were written in Matlab R2014a and run on a DELL with 4 GB RAM memory and Windows 10 operating system. For the LJJ CGPM, we use (6) and (11)  For all methods, the computation will be terminated when one of the following criteria are satisfied: where "Itr" refers to the total number of iterations. Define and the tested functions are listed as follows.
In addition, in order to show the numerical performance clearly, we adopt the profiles introduced by Dolan and Morè [38] to compare the performance on Tcpu, NF, and Itr, respectively. A brief explanation of the performance figures is as follows. Denote the whole set of n p test problems by P and the set of solvers by S. Let t p,s be the Tcpu (or the NF or the Itr) required to solve problem p ∈ P by solver s ∈ S, and the comparison results between different solvers are based on the performance ratio defined from and the performance profile for each solver s is defined by where size A means the number of elements in the set A. en, ρ s (τ) is the probability for solver s ∈ S that a performance ratio r p,s is within a factor τ ∈ R n . ρ s is the (cumulative) distribution function for the performance ratio.
Clearly, the top curved shape of the method is a winner. For details about the performance profile, see [38].

Numerical Testing Reports.
e specific results of the numerical tests are displayed in Tables 1-8. eir corresponding performance profiles are plotted in Figures 1-3 in terms of Tcpu, NF, and Itr, respectively. From Tables 1-8 and Figures 1-3, the following results are obtained.
(i) e LJJ CGPM, PDY, and ATTCGP can solve all test problems completely, which shows that abovementioned three methods are effective. (ii) From Figures 1-3, the LJJ CGPM is the top performer among the three algorithms, and this implies that the LJJ CGPM is superior to the PDY and ATTCGP totally, at least for this set of numerical experiments.
Reasonably, the advantages of the LJJ CGPM are attributed to the choice of techniques (6) and (11) for the conjugate parameter of search direction and the adaptive line search strategy. At the first glance, the parameter β LJJ k (6) and line search strategy (11) are a bit complicated; however, the LJJ CGPM is very effective for solving constrained equations. Furthermore, numerical results indicate that the proposed method is competitive to similar methods for large-scale problems.

Application in Compressed Sensing
In this section, the LJJ CGPM is applied to a typical compressed sensing scenario and compared with the PDY method [22] in terms of the mean of squared error (MSE), iterative number, and CPU time. e parameters for the LJJ CGPM are set as follows: ρ � 0.55, β � 10, σ � 10 − 4 , and μ � 1.25, ] � 2.6. Also, the parameters of the PDY method are taken from Section 4 in [22].

Compressed Sensing.
Compressed sensing, also called compressed sampling or sparse sampling, is a typical signal sampling technique. In electronic engineering, especially in signal processing, compressed sensing is often used to acquire and reconstruct sparse or compressible signals.
In this section, the attention is focused on recovering the unknown vector x 0 ∈ R n from an incomplete or disturbed observation where b ∈ R k is the observation data, A ∈ R k×n (k ≪ n) is a linear mapping, and e ∈ R k is an error term. In fact, the sparse result of x 0 can be obtained by solving the following convex unconstrained optimization problem: where parameter τ > 0, ‖ · ‖ 1 and ‖ · ‖ 2 mean the l 1 − norm and l 2 − norm, respectively. ere are many algorithms to solve problem (47). One of them is the gradient projection method 6 Complexity for sparse reconstruction proposed by Figueiredo et al. [39]. e first step of this method is to convert (47) into a convex quadratic programming problem.
For any vector x ∈ R n , it can be decomposed into the following two parts: where u i � max x i , 0 and v i � max − x i , 0 for all i � 1, 2, · · · , n. In this way, the l 1 − norm of a vector can be expressed as ‖ · ‖ 1 � e T n u + e T n v, where e T n � (1, 1, · · · , 1) ∈ R n . erefore, problem (47) can be expressed as the following convex quadratic programming problem with nonnegative constraints min u,v where u ≥ 0, v ≥ 0. Furthermore, it is easy to get the following standard form: where Obviously, H is a semipositive definite matrix. So, problem (50) is a convex quadratic programming problem. In [40], it has been proven to be equivalent to where the function F is vector valued, and the "min" is explained as componentwise minimum. From [40]

Numerical Results.
In these experiments, the main goal is to recover a one-dimensional sparse signal from its limited measurements with Gaussian noise, that is, to reconstruct a n− length sparse signal from k(k ≪ n) observations. e Gaussian noise distributed with N(0, 10 − 4 ) is used in the test. n � 2 12 and k � 2 10 are taken for equation (46). e quality of restoration is measured by the MSE, namely, where x * is the restored signal and x 0 is the original signal. Besides, the original signal x 0 contains 2 4 nonzero elements randomly. e random matrix A is given by the command rand (n, k) in Matlab, and the observed data b is obtained by b � Ax 0 + e, where e is the Gaussian noise. f(x) � τ‖x‖ 1 + (1/2)‖Ax − b‖ 2 2 denotes the merit function, and the value τ is obtained by the same continuation technique for the abovementioned two algorithms (LJJ CGPM and PDY method). e iterative process starts at the measurement signal, that is, x 0 � A T b, and terminates when the relative change between successive iterative falls below 10 − 5 , that is,  Figure 4 gives the original signal x 0 , the measurement b, and the signal x * reconstructed by the LJJ CGPM and PDY method. It can be seen from Figure 4 that the LJJ CGPM and PDY method almost completely recover the disturbed signal. Moreover, to visualize the performance of the two algorithms, Figures 5 and 6 are plotted, which describe the curves of MSE and objective function values over iterations and CPU time (seconds), respectively. As can be seen from Figures 5 and 6, the LJJ CGPM has advantages over the PDY method in the MSE and objective function values, indeed, and the red curves (LJJ CGPM) drop faster than the blue curves (PDY method). Overall, simple tests show that the LJJ CGPM can effectively decode sparse signals in compressed sensing.

Conclusions
In this work, based on the research studies for the CGMs and some common inexact line searches, a modified conjugate parameter and an adaptive line search strategy are proposed, and then, an effective CGPM is presented for monotone nonlinear equations. Besides, the search direction of our proposed algorithm possesses the properties of sufficient descent and trust region independent of any line search techniques. Furthermore, the presented method inherits the advantages of the CGM with a simple iterative form, rapid convergence, being derivative-free, and low memory requirements. Under some suitable conditions, the global convergence of the proposed method is established, and its numerical experiments are conducted and reported, which show that our method is very effective in dealing with largescale monotone nonlinear equations with convex constraints and the ℓ 1 − norm regularization problem in compressive sensing.
Regrettably, the parameter μ in a new adaptive line search strategy is fixed. Along with the existing line search rule, our future work is to develop the line search (11) by a dynamic adjustment technology instead of the constant μ. Moreover, summarizing the abovementioned observations and from the fact that the search direction is yielded by the CGM, how to improve the line search rule by combining with the frequently-used inexact line searches for the CGM deserves to be studied.

Data Availability
No data were used to support this study.

Conflicts of Interest
e authors declare that they have no conflicts of interest.