Two Improved Conjugate Gradient Methods with Application in Compressive Sensing and Motion Control

To solve the monotone equations with convex constraints, a novel multiparameterized conjugate gradient method (MPCGM) is designed and analyzed. This kind of conjugate gradient method is derivative-free and can be viewed as a modiﬁed version of the famous Fletcher–Reeves (FR) conjugate gradient method. Under approximate conditions, we show that the proposed method has global convergence property. Furthermore, we generalize the MPCGM to solve unconstrained optimization problem and oﬀer another novel conjugate gradient method (NCGM), which satisﬁes the suﬃcient descent property without any line search. Global convergence of the NCGM is also proved. Finally, we report some numerical results to show the eﬃciency of two novel methods. Speciﬁcally, their practical applications in compressive sensing and motion control of robot manipulator are also investigated.


Introduction
Let R, R n , and R m×n be the sets of real numbers, n dimensional real column vectors, and m × n dimensional real matrices, respectively. is paper is concerned with the following two active subjects in numerical analysis.
(i) Monotone equations with convex constraints: finding a vector x * ∈ X such that where F: R n ⟶ R n is a continuous nonlinear mapping (not necessarily smooth) and X ⊆ R n is a nonempty convex set. (ii) Unconstrained optimization problem: finding a vector x * ∈ R n such that where f: R n ⟶ R is a continuously differentiable function whose gradient is denoted by g(x).
Problems (1) and (2) are interchangeable in some sense. In fact, setting f(x) � (1/2)‖F(x)‖ 2 , problem (1) with X � R n can be transformed into problem (2). Similarly, the necessary condition of problem (2), i.e., g(x) * � 0, is a special case of problem (1). erefore, the design of numerical methods for the two problems often inspires each other and gives each other inspiration. For example, the first conjugate gradient method was developed by Hestenes and Stiefel to solve the system of linear equations [1], and then this method was generalized to solve the unconstrained optimization problem by Fletcher and Reeves [2].
Problems (1) and (2) appear frequently in many areas of applied mathematics and play important roles in many applications, such as compressive sensing, image processing, control theory, and motion control of robot manipulator [3][4][5][6][7][8]. For example, in the numerical solution theory of partial differential equations, the finite difference schemes of elliptic equations can be transformed into the following Sylvester equations: where A ∈ R p×m , B ∈ R n×q , and C ∈ R p×q are given matrices and X ∈ R m×n is the unknown matrix. en, using the Kronecker product ⊗ and the vectorization operator vec(·), we can transform the above Sylvester equations into a linear system of equations as follows [9]: which is a special case of problem (1) with x :� vec(X), Due to the numerous applications in diverse scientific areas, problems (1) and (2) have been extensively studied during the past few decades and many numerical methods have been proposed. e numerical methods for problem (1) can be roughly divided into two categories: the iterative methods for smooth case and the iterative methods for nonsmooth case. More specifically, the methods in the first category need to assume that the mapping F(x) is smooth, which includes the Newton method, quasi-Newton method, Levenberg-Marquardt method, and their variants [10][11][12][13]. e methods in this category often need to solve a linear system of equations at each iteration, which indicates that they are not suitable to solve largescale problem (1). e methods in the second category remove this restriction. For example, based on the spectral gradient method for unconstrained optimization problem, Cruz et al. [14,15], Zhang and Zhou [16], and Liu and Duan [17] have successively proposed some spectral gradient projection methods or spectral residual methods for solving problem (1) with X � R n . Motivated by the studies in [14][15][16], Cheng [18] extended the Polak-Ribiére-Polyak (PRP) method to solve problem (1) with X � R n . Other similar methods include the twoterm PRP-based method [19], the CG − DESCENT method [3], the Hestenes-Stiefel projection method [20], and the hybrid conjugate gradient projection method [21]. After careful analysis and comparison, we find that the above methods mainly consist of the following three steps at each iteration: (i) a sufficient descent direction is first generated, along which a step size is obtained by Armijolike line search; (ii) a temporal iterate z k is generated, and then a hyperplane is defined, which strictly separates the current iterate x k and the solution set X * of problem (1); (iii) the next iterate x k+1 is defined by the projection of x k onto the hyperplane H k . On the other hand, the conjugate gradient method is one of the most efficient solvers for large-scale problem (2), whose iteration sequence {x k } is generated by where α k > 0 is the step size and d k is the search direction defined by in which β k is the so-called conjugate gradient formula which is the main difference in conjugate gradient methods. Since 1952, many conjugate gradient methods have been offered, such as the Hestenes-Stiefel (HS) method [1], the Fletcher-Reeves (FR) conjugate gradient method [2], the Polak-Ribiére-Polyak (PRP) conjugate gradient method [22], the Liu-Storey (LS) conjugate gradient method [23], and the Dai-Yuan (DY) conjugate gradient method [24]. During the last two decades, many conjugate gradient methods with sufficient descent property were proposed, and the first one is that proposed by Shi and Shen [25], which has not aroused continuing concern. Lately, the CG − DESCENT method designed by Hager and Zhang [26] is another one with sufficient descent property, which has inspired to benefit much research and design in this direction, and many efficient conjugate gradient methods have been developed, such as the modified FR in [27], the modified PRP in [28], and the descent memory gradient method in [29], in which the modified FR in [27] accomplished a theoretical breakthrough of great significance.
In this paper, based on the Fletcher-Reeves (FR) conjugate gradient method, we firstly propose a multiparameterized conjugate gradient method (MPCGM) for problem (1), which is derivative-free and thus only needs to compute the value of mapping F(x) at each iteration. en, the method is generalized to solve problem (2), and a novel conjugate gradient method (NCGM) is obtained. Both methods' convergence property is analyzed under traditional conditions and their practical application in compressive sensing and motion control of robot manipulator is investigated. e remainder of this paper is organized as follows. In Section 2, we describe the MPCGM for nonsmooth problem (1). Moreover, the proof of its global convergence is also presented. In Section 3, we generalize MPCGM to solve nonconvex problem (2) and analyze the convergence property of the generalized method. In Section 4, some numerical results and comparisons are presented, and finally a brief conclusion is drawn in Section 5. Before ending this section, it is worth pointing out the main contributions of this paper as below.
(i) A multiparameterized conjugate gradient method is proposed for nonsmooth problem (1), which is used to solve compressive sensing. (ii) A novel conjugate gradient method is proposed for nonconvex problem (2), which is used to solve motion control of robot manipulator. (iii) Global convergence property of two novel methods is proved under mild conditions.

Multiparameterized Conjugate Gradient Method
Projection operator P Ω [x] is defined as a mapping from the n dimensional Euclidean space R n to a nonempty closed convex subset Ω ⊆ R n : which satisfies the following property [30].

Lemma 1.
Let Ω be a closed convex subset of R n . For any x, y ∈ R n , we have Assumption 1 (1) e solution set of problem (1), denoted by X * , is nonempty.
(3) e mapping F(x) is Lipschitz continuous on X, i.e., there exists a constant L > 0 such that Based on the research in [27,29], we present a multiparameterized conjugate gradient method for nonsmooth problem (1) as follows.
Step 2: compute d k by where θ k and β k are two parameters defined by Step 3: compute a temporal iterate z k � x k + α k d k , where α k � βρ m k with m k being the smallest nonnegative integer m such that Step 4: if z k ∈ X and ‖F(z k )‖ < ε, then stop; otherwise, compute the new iterate x k+1 by where Set k � k + 1 and go to Step 1.

Remark 1.
Parameter β k is obtained by replacing the denominator ‖F(x k− 1 )‖ 2 of β k in the classical FR conjugate gradient method by ‖d k− 1 ‖ 2 , and parameter θ k makes the generated direction d k satisfy sufficient descent property, which is proved in the next lemma.
Proof. If k � 0, from (13), it holds that If k ≥ 1, from (13) again, we have erefore, for all k ≥ 0, inequality (18) always holds. is completes the proof. □ Remark 2. By Cauchy-Schwarz inequality, it holds that Remark 3. Parameter ξ k in Step 4 of MPCGM is well defined, which is analyzed as follows.
(i) For v � 0: if ‖F(z k )‖ � 0, from the line search (15), we have ‖d k ‖ � 0, which together with (21) implies Mathematical Problems in Engineering . is together with the line search (15) gives i.e., e following lemma indicates that the Armijo-type line search (15) is well defined.

Lemma 3.
For each k ≥ 0, there exists a nonnegative integer m k satisfying inequality (15).
Proof. If the Armijo line search (15) is executed, then ‖F(x k )‖ ≥ ε > 0. Assume that there exists an integer k 0 ≥ 0 such that inequality (15) does not hold for any nonnegative integer m, i.e., Setting m ⟶ +∞ and taking limits on both sides of the above inequality, we get is together with inequality (18) gives en, for any fixed k ≥ 0, the step size α k is bigger than a positive number, i.e., there exists c k > 0, such that Furthermore, we can deduce that Proof. If α k ≠ β, then according to principle of the Armijo line search (15), the positive number α k ′ � α k /ρ does not satisfy the following inequality: So, by (15) and (18), we get from which we get inequality (26). is completes the proof.

Lemma 5. Let {x k } and {z k } be two sequences generated by MPCGM. en {x k } and {z k } are both bounded, and
Proof. From inequality (15), we have Choose x * ∈ X * ; from the monotonicity of F(x), we get which together with (10) and (16) implies where the last inequality follows from (31). erefore, the sequence ‖x k − x * ‖ is decreasing and convergent, and thus the sequence {x k } is bounded. From (33), we have en, Proof. We prove (36) by using reduction to absurdity. Suppose that (36) is not true. en, there is a constant ε 0 > 0 such that By (21), we have Combining this with (30), it holds that On the other hand, by the boundedness of {x k }, there exists a constant M 1 > 0 such that Furthermore, by (39) and the continuity of F(x), there exists M 2 > 0, such that en, by the definition of search direction d k defined by (13), we have is together with (26) implies that which contradicts (39). erefore, conclusion (36) holds and the proof is completed.

Novel Conjugate Gradient Method
In this section, we will generalized MPCGM to solve problem (2) and prove its global convergence. Firstly, we make the following standard assumption.
x 0 ∈ R n in an initial point. (3) e gradient g(x) is assumed to be Lipschitz continuous on R n , i.e., there exists a constant L > 0 such that
Step 2: compute d k by where θ k and β k are two parameters defined by Mathematical Problems in Engineering Determine the step size α k � ρ m k with m k being the smallest nonnegative integer m such that Step 3: set x k+1 � x k + α k d k and k � k + 1; go to Step 1.
Similar to Lemma 2, it holds that where C � min{1, c}. From inequality (48), it is easy to prove that the Armijo line search (47) is well defined. Moreover, from the Cauchy-Schwarz inequality and (48), it holds that e next theorem indicates that NCGM is globally convergent.

Theorem 2. If Assumption 2 holds and NCGM generates an infinite sequence {x k }, we have
Proof. First, we prove that there exists a constant c 1 > 0 such that the following inequality holds for all k: e proof of (51) is divided into the following two cases.
Case (I): if α k � 1, then from (49), we have Case (II): if α k < 1, then by the Armijo line search condition, ρ −1 α k does not satisfy inequality (47). at is, By the mean-value theorem of the continuous function, there exists a constant t k ∈ (0, 1) such that Substituting the last inequality into the left-hand side of (52), we get Setting c 1 � min C 2 , ((1 − δ)ρC 2 )/L , we can get inequality (51). From (47), (48), and Assumption 2, it is easy to deduce that Substituting (51) into the left-hand side of (55), we can derive the famous Zoutendijk condition Suppose that conclusion (50) is not true, so there is a constant ε 0 > 0 such that By the definition of d k , we have where M 2 > 0 is the upper bound of f(x) in the level set L 0 . From this inequality and (56), we get which contradicts (56). erefore, conclusion (50) holds. e proof is completed.

Numerical Results
In this section, to show the efficiency of MPCGM and NCGM, we apply them to solve problems (1) and (2). Furthermore, we compare the performance of MPCGM with the spectral gradient projection method in [31] (SGPM) and the conjugate gradient method in [3] (CGM). All codes were written in MATLAB R2014a, and run on a notebook 6 Mathematical Problems in Engineering computer with Intel Core 2 CPU 2.10 GHZ and RAM 2.00 GM.

Numerical Test of MPCGM.
We consider two synthesized problems and one practical problem, which are drawn from [3,32,33].
It is easy to prove that the above two mappings are monotone. e parameters in the three tested methods for Problem 1 and Problem 2 are set as follows: In the experiment, we use the following termination condition: In MPCGM, we have introduced two new parameters c and v. Now, we conduct some sensitivity tests on the two parameters to determine their optimal choices. Here, we use the tentative method and analyze the fluctuation of the number of iterations with respect to different values of c and v. Specifically, we set c or v as abscissa and we set the number of iterations as ordinate. e numerical results are graphically shown in Figure 1, from which we can see that for Problem 1, larger values of c can accelerate the convergence of MPCGM, and for Problem 2, the positive values of v can also accelerate the convergence of MPCGM. erefore, the advantage of incorporating the parameters c and v into MPCGM is verified. In the following, we set c � 1.7 and v � 0.07. Now, we give more numerical results about Problem 1 and Problem 2 with the number of variables n � 1000, 2000, 5000, 10000, 20000, 50000, 100000, 1000000, and the initial point is set as x 0 � (1, 1, . . ., 1). e numerical results are reported in Tables 1 and 2, which contain the dimension of the problem (Dim), the number of iterations (Iter), the CPU time required in seconds (Time), and the final norm of equations (Fn) when the termination condition is satisfied. It is well known that when a set is a polyhedral, that is, all the constraint functions defining the set are linear, then computing the projection on it reduces to solving a quadratic problem. Here, we use the quadratic program solver quadprog.m from the MATLAB optimization toolbox to perform the projection operator. e numerical results in Tables 1 and 2 verify that the gradient methods perform well on the large-scale constrained monotone equations. For Problem 1, the performance of CGM and MPCGM is obviously better than that of SGPM, and the performance of MPCGM is obviously better than that of CGM.
at is, MPCGM performs the best among the three tested methods. As the dimension increased, the advantage on the required CPU time of MPCGM becomes prominent gradually. For Problem 2, there seems to be not much difference among the performance of the three tested methods, and MPCGM still performs a little better than the other two methods because it is the fastest for most scenarios. In a word, the numerical experiments show that the proposed method provides an efficient tool to solve nonlinear constrained equations.
where A ∈ R m×n (m ≪ n) is a linear operator, b ∈ R m is an observation, is the unknown vector, ‖x‖ 1 � n i�1 |x i | is the ℓ 1norm of x, and parameter μ > 0 is used to trade off both terms of the objective function of (64). Following the procedure of Figueiredo et al. [33], we can set x � u − v, u ≥ 0, v ≥ 0, where u ∈ R n , v ∈ R n , and u i � (x i ) + , v i � (−x i ) + for all i � 1, 2, . . ., n with (·) + � max{0, ·}. en, CS can be rewritten as at is, where z � u v , Mathematical Problems in Engineering en, Xiao et al. [3] further transformed the above optimization problem as the constrained nonlinear equations: e following relative error (RelErr) to the original signal x is used to measure the quality of restoration: where x * is the restored signal. In the experiment, our goal is to reconstruct a length-n sparse signal from m observation.

Conclusion
In this paper, we have proposed a multiparameterized conjugate gradient method for nonlinear equations with convex constraints. Under the condition that the underlying mapping is monotone and Lipschitz continuous, we have established its global convergence. Furthermore, we have generalized this method to solve unconstrained optimization and get a new conjugate gradient method, whose global convergence is analyzed under mild conditions. Preliminary numerical results are reported which indicate that the proposed methods perform better than some well-developed methods.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that they have no conflicts of interest.