The Smoothing FR Conjugate Gradient Method for Solving a Kind of Nonsmooth Optimization Problem with l1-Norm

We study the method for solving a kind of nonsmooth optimization problems with l1-norm, which is widely used in the problem of compressed sensing, image processing, and some related optimization problems with wide application background in engineering technology. Transformated by the absolute value equations, this kind of nonsmooth optimization problem is rewritten as a general unconstrained optimization problem, and the transformedproblem is solved by a smoothing FR conjugate gradientmethod. Finally, the numerical experiments show the effectiveness of the given smoothing FR conjugate gradient method.


Introduction
In the last few years, the study work of finding sparsest solutions to undermined system of equations has been extensively done.Finding the sparsest solution of an undermined system of equations is equivalent to solving the  0 -norm regularized minimization problem as the following: where  = { |  = ,  ∈   },  ∈  × ( ≪ ),  ∈   , and ‖ ⋅ ‖ 0 denotes 0-norm.From [1][2][3][4], we know that the above problem is difficult to solve in a straight way.In order to solve the  0 -norm problem effectively, an approximation model is to replace the  0 -norm by the  1 -norm to solve the Basis Pursuit problem, such as in [5,6]: Therefore the convex envelope of ‖‖ 0 is ‖‖ 1 , where ‖‖ 1 = ∑  =1 |  | is the  1 -norm of .And this problem is also NP-hard problem.When  contains some noise in practice application, the above problem is rewritten as the following nonsmooth optimization problem with  1 -norm: where  ∈  × ( ≪ ),  ∈   ,  > 0, ‖ ⋅ ‖ 2 denotes 2-norm, and ‖⋅‖ 1 denotes 1-norm.For the minimization of the  1 -norm having a good recovery, (3) is widely used in the sense compression, image processing, and other related fields in engineering technology, one can see [7,8] and the references therein.For some  > 0, (1/2)‖ − ‖ 2 2 + ‖‖ 1 is a convex function but not a differentiable function.Recently, many scholars have studied the method for solving (3).For instance, gradient projection for sparse reconstruction was proposed by Figueiredo et al. in [9], a two-step iterative shrinkage thresholding (IST) method was proposed by Bioucas-Dias and Figueiredo in [10], a fast IST algorithm was presented by Beck and Teboulle in [11], SPGL1 (A solver for large-scale sparse reconstruction) was proposed by Ewout and Friedlander in [12], who consider a least-squares problem with  1 -norm constraint and use a spectral gradient projection method, and ADM method was proposed by Yang and Zhang in [13].Problem (3) was formulated to a convex quadratic problem in [14].Among all the references mentioned above, there is no use of the relationship between the linear complementarity problems and the absolute value equations to solve (3).They do not use the structure of the absolute value equation to propose new method to solve (3).They also do not translate the original problem into an absolute value equation problem and use the effective methods of absolute value equations.Just recently, only in [15], a smoothing gradient method is given for solving (3) based on the absolute value equations.Therefore, in this paper, we study how to use this new transformation to solve (3) by the useful FR conjugate method.
As we all know, the transformed linear complementarity problem is rewritten as the absolute value equation problem mainly based on the equivalence between the linear complementarity problem and the absolute value equation problem, such as [16][17][18].The absolute value equation can be solved easily now.On the other hand, the conjugate gradient method is suitable for solving large-scare optimization problems and has sample structure and global convergence [19][20][21][22][23][24][25].In addition, the smoothing methods are used to solve the related nonsmooth optimization problems, such as [26][27][28] and the references therein.Therefore, based on the above analysis, we present a new smoothing FR conjugate gradient method to solve (3); this is also our motivation to write this paper.The global convergence analysis of the given method is also presented.Finally, some computational results show that the smoothing FR conjugate gradient method is efficient in practice.
The remainder of this paper is organized as follows: In Section 2, the preliminaries are proposed, which include the description of how the linear complementarity problem is transformed into the absolute value equation problem.In Section 3, we present the smoothing FR conjugate gradient method and give its convergence analysis.Finally, in Section 4, we give some numerical results of the given method, which show the effectiveness of the given method.

Preliminaries
Firstly, we give the transformation form of (3).For any vector  ∈   , it can be formulated to where , V ∈   ,   = (  ) + , and V  = (−  ) + for all  = 1, . . ., .By the definition of ‖‖ 1 = ∑  =1 |  |, we get ‖‖ 1 =     +    V, where   = [1, 1, . . ., 1]  .Therefore, as in [13][14][15], problem (3) can be rewritten as min The above problem can be transformed to min 1 2    +   , where Since  is a positive semidefinite matrix, problem (3) can be transformed into convex optimization problem.Then problem ( 6) can be transformed into a linear variable inequality problem, which is to find  ∈  2 , such that Given that the feasible region of  is a special structure (such as nonnegative orthant), ( 8) can be rewritten as the linear complementary problem, which is used to find  ∈  2 , such that Now, we give some results about the absolute value equations and the linear complementarity problems as follows; one can see sources such as [16,29,30].The absolute value equations have the form  − || = , where  ∈  × ,  ∈   .The linear complementarity problems have the form 0 ≤  ⊥  +  ≥ 0, where  ∈  × ,  ∈   .

Proposition 1. (i) Conversely, if 1 is not an eigenvalue of 𝐻, then the linear complementary problem can be reduced to the following equation:
(ii) The absolute value equations is equivalent to the bimultilinear program: (iii) And the absolute value equations is equivalent to the generalized linear complementary problem: Proposition 2. Equation ( 9) can be transformed into the following absolute value equation problem, which is defined as Proof.Based on Proposition 1 and ( 9), we know that Then by ( 14), we have To satisfy the last equation of ( 15) for all  ∈   , denote Substituting ( 16) in ( 15), we get Due to the above (i) in Proposition 1, (9) can be reduced to the absolute value equations: and substituting form (17) in (18), we get the following absolute value equation problem, which has the form Thus, we get (13).Then, problem (3) can be transformed into the following problem:

The Smoothing FR Conjugate Gradient Method
In this section, we give the smoothing FR conjugate gradient method for solving (20).Firstly, we give the definition of smoothing function and the smoothing approximation function of the absolute value function; one can see [15,26,27].
There are so many smoothing functions; for example, Chen and Mangasarian introduced a class of smooth approximations of the function () + .Let  :  →  + be a density function satisfying Then Then  is a smoothing function of () + .
In this paper, we use the smoothing approximation function of the absolute value function as And ( 24) is the smoothing approximation function of (20).
Now, we give the smoothing FR conjugate gradient method for solving (20).
Figures 3 and 4 plotted the evolution of the objective function versus iteration number ( = 100,  = 110) when solving Example 2 with Algorithm 4 and smoothing gradient method.Figures 5 and 6 plotted the evolution of the objective function versus iteration number ( = 200,  = 210) when solving Example 2 with Algorithm 4 and smoothing gradient method.By comparison, we know that the number of iterations of the Algorithm 4 is less than that of the smoothing gradient method in [15].
(43)    and smoothing gradient method in [15].By comparison, Algorithm 4 is more effective than the smoothing gradient method.

Conclusion
Compared with the GPSR method and other methods in [4,9,[13][14][15], the smoothing FR conjugate gradient method is simple and needs small storage.The establishment and continuous improvement of the smoothing method for (3) provide a very useful tool to meet the challenges of many practical problems.For example, Figure 15 shows that the smoothing FR conjugate gradient method works well, and it provides an efficient approach to denoise sparse signals.Compare with the smoothing gradient method in [15], the smoothing FR conjugate gradient method is significantly faster than the smoothing gradient method, especially in large-scale iterations.We have also shown that, under weak conditions, the smoothing FR conjugate gradient method converges globally.

Figure 2 :
Figure 2: Numerical results for solving Example 1 with smoothing gradient method.

Figures 7 and 8 ,
Figures 7 and 8, respectively, show the results while  = 100,  = 110, the change in the number of iterations of the objective function when solving Example 3 with Algorithm 4 and smoothing gradient method.Figures 9 and 10, respectively, show the results while  = 150,  = 160, the change in the number of iterations of the objective function when solving Example 3 with Algorithm 4 and smoothing gradient method.By comparison, the objective function variants are faster in Algorithm 4 than that of the smoothing gradient method in [15].
Figures 7 and 8, respectively, show the results while  = 100,  = 110, the change in the number of iterations of the objective function when solving Example 3 with Algorithm 4 and smoothing gradient method.Figures 9 and 10, respectively, show the results while  = 150,  = 160, the change in the number of iterations of the objective function when solving Example 3 with Algorithm 4 and smoothing gradient method.By comparison, the objective function variants are faster in Algorithm 4 than that of the smoothing gradient method in [15].

Example 4 .
Consider the following optimization problem:

Figures 11 and 12
Figures11 and 12, respectively, show the objective function plotted against iteration number while  = 100,  = 110, when solving Example 4 with Algorithm 4 and smoothing gradient method.Figures13 and 14, respectively, show the objective function plotted against iteration number while  = 150,  = 160, when solving Example 4 with Algorithm 4 and smoothing gradient method in[15].By comparison, Algorithm 4 is more effective than the smoothing gradient method.

Figure 15 :
Figure 15: Numerical results for solving Example 5 with Algorithm 4.