Two New Conjugate Gradient Methods for Unconstrained Optimization

+e conjugate gradient method is very effective in solving large-scale unconstrained optimal problems. In this paper, on the basis of the conjugate parameter of the conjugate descent (CD) method and the second inequality in the strong Wolfe line search, two new conjugate parameters are devised. Using the strong Wolfe line search to obtain the step lengths, two modified conjugate gradient methods are proposed for general unconstrained optimization. Under the standard assumptions, the two presented methods are proved to be sufficient descent and globally convergent. Finally, preliminary numerical results are reported to show that the proposed methods are promising.


Introduction
e conjugate gradient method (CGM for short) plays an important role in obtaining the numerical solution of the optimal control problem for nonlinear dynamic systems and other mathematical models [1,2]. In this paper, we study the nonlinear CGM for the following unconstrained optimization problem: where f : R n ⟶ R is smooth and its gradient g(x) � ∇f(x) is available. e iterates of the classic CGMs for solving problem (1) are generated by x k+1 � x k + α k d k . First, the step length α k > 0 is usually yielded by a suitable inexact line search along the search direction d k , such as the Wolfe line search or the strong Wolfe line search where ‖·‖ stands for the Euclidean norm. A great number of CGMs with good convergence properties and effective numerical performance are deuterogenic by the six methods above, see e.g., [9][10][11][12][13][14][15]. As we know, Jiang et al. [12] studied a CGM work called JMJ method, where the parameter β k is correspondingly specified by , otherwise.
In this paper, we focus our attention on the ideas of the JMJ and CD methods as well as the strong Wolfe line search. In particular, by the second inequality of the strong Wolfe line search, it follows that |g T us, based on the formula β JMJ k , making full use of the characteristics of the JMJ and CD methods, and to ensure that our proposed methods possess nice convergent properties and to improve the numerical performance, two new formulas for β k are constructed in this paper. e first one is generated by replacing the term ‖g k ‖/‖d k− 1 ‖ in β JMJ k with the CD formula β CD k , namely, On the contrary, replacing the denominator in β LMYCD1 k with ‖g k− 1 ‖ 2 , the second one is presented with From (7) and (8), it is not difficult to know that the former formula β LMYCD1 k reduces to the DY formula when the exact line search is used, and accordingly, in the same condition, the later one β LMYCD2 k reduces to the FR formula. e rest of the paper is organized as follows. In Section 2, two modified methods and the sufficient descent properties are presented. Global convergence properties of the proposed methods are analyzed in Section 3. Some numerical results are reported in Section 4. Finally, we draw a conclusion in Section 5.

Methods and Sufficient Descent Properties
In this section, we first describe the details of the two proposed methods, which for convenience are called the LMYCD1 and LMYCD2 methods in Algorithm 1 and Algorithm 2, respectively.

Sufficient Descent Condition.
If there exists a constant c > 0 such that g T k d k ≤ − c‖g k ‖ 2 , ∀k ≥ 1, then, we say that the search direction d k of the method satisfies the sufficient descent condition, which is often used to analyze the convergence properties of CGMs for this kind of problem (1) under inexact line search, see e.g., [10][11][12][13][14]. e next lemmas show that the search directions yielded by the two proposed methods always satisfy the sufficient descent condition. Lemma 1. Suppose that d k is generated by the LMYCD1 method and 0 < σ < 1. en, Proof. We prove (9) by induction. For k � 1, it is easy to then obviously β CD k > 0 holds by its definition. Furthermore, from the strong Wolfe line search, we have is, together with β LMYCD1 k and relation (10), further implies that Now, we prove that (9) holds for k via the following three cases: 2 Complexity (iii) If g T k d k− 1 < 0, then, using d k and β LMYCD1 k > 0, we also obtain erefore, the assertion is satisfied for all k ≥ 1.

Lemma 2.
Let the search direction d k be yielded by the LMYCD2 method and 0 < σ < 1/2. en, Proof. For k � 1, one has g T 1 d 1 /‖g 1 ‖ 2 � − 1, so (15) clearly holds. Suppose that relation (15) is satisfied for k − 1. Now, we continue to prove that (15) holds for k. By the strong Wolfe line search and us, using β LMYCD2 k , we obtain Furthermore, recalling d k in Algorithm 2, β CD k and β LMYCD2 k , one has is together with the right-hand side of (16) further shows that Initialization. Given constants ϵ > 0 and 0 < δ < σ < 1, as well as x 1 ∈ R n . Let d 1 � − g 1 , k : � 1.
Step 2. Determine a step length α k by the strong Wolfe line search (3).
Step 1 and Step 2 are the same as the step 1 and step 2 of LMYCD1 method.
Step 3. Let Step 4 is the same as the step 4 of LMYCD1 method. ALGORITHM 2: (LMYCD2 method).

Complexity 3
Next, based on the strong Wolfe line search, from the left-hand side of (19) and assertion (15) for k − 1, we have Similarly, from the right-hand side of (19), we also obtain us, the proof is completed.

Convergence Results
roughout this paper, we make the following elementary assumptions for the objective function: To proceed, the well-known Zoutendijk condition [16] is reviewed in the following.

Zoutendijk Condition.
Suppose that assumptions (H1)-(H2) hold, the search direction d k is a descent direction and the step length α k satisfies the Wolfe line search condition, then we have ∞ k�1 (g T k d k ) 2 /‖d k ‖ 2 < ∞. In particular, if the sufficient descent condition is satisfied, then Now, before establishing the global convergence of the LMYCD1 method, we show that the LMYCD1 method has similar properties to that of the DY method, which is very important to analyze the global convergence property of the method.

Lemma 3. Let the sequence x k be generated by the LMYCD1 method, then
by the following two cases: . en, dividing this inequality by the negative term and hence   Proof. By contradiction, we suppose that the conclusion is not true, then there exists a constant c > 0 such that (24) Notice that ‖d 1 ‖ 2 /(g T 1 d 1 ) 2 � 1/‖g 1 ‖ 2 , using the abovementioned formula, we have      Table 2. Here, parameters σ � 0.1 and δ � 0.001. In addition, we make use of the performance profiles of Dolan and Moré in [21] to compare the performance of the  Complexity tested methods listed above, and readers refer to this literature for details about the introduction of the performance profiles. It is worth noting that the left side of each performance profile figure indicates the percentage of the test problems, which is the fastest among these methods, whereas the right side gives the percentage of the test problems that are successfully solved by each method. e top curve means that the corresponding method implements best in contrast to other methods. Figures 1 and 2 illustrate the performance profiles for the LMYCD1, hDY, NHS, JMJ, MDL1, and MDL2 methods, by the total number of iterations, function evaluations, gradient evaluations, and the CPU time (s), respectively. In Figures 3  and 4, the performance profiles of the LMYCD2, NPRP, JMJ, MDL3, and MDL4 methods are described.
Observing from all Figures 1-4, the LMYCD1 and LMYCD2 methods, are competitive and both of them outperform the tested methods in each group, with respect to the characteristics Itr, NF, NG, and Tcpu, respectively. In addition, the two proposed methods in this paper ultimately solve 100% of the respective test problems successfully. All numerical results show that the efficiency of the LMYCD1 and LMYCD2 methods is encouraging.

Conclusion
In this paper, we construct two new formulas for setting parameter β k by using substantially the information of the JMJ and CD formulas as well as the second inequality in the strong Wolfe line search (3). Under the usual assumptions, the presented methods are proved to be sufficient descent and globally convergent. Elementary numerical experiments demonstrate that the two proposed methods perform effectively.

Data Availability
No data were used to support this study.

Conflicts of Interest
e authors declare that there are no conflicts of interest regarding the publication of this paper.