On the Strong Convergence of a Sufficient Descent Polak-Ribière-Polyak Conjugate Gradient Method

and Applied Analysis 3 Thus, f (xk) − f (xk + δkρ m dk) δkρ m < μ 󵄩󵄩󵄩gk 󵄩󵄩󵄩 2 , ∀m. (14) Letting m → +∞, by the continuity of f(x) and −g k dk = ‖gk‖ , we can obtain 󵄩󵄩󵄩gk 󵄩󵄩󵄩 2 ≤ μ 󵄩󵄩󵄩gk 󵄩󵄩󵄩 2 . (15) This and μ ∈ (0, 1) yield that 󵄩󵄩󵄩gk 󵄩󵄩󵄩 = 0, (16) which contradicts to ‖gk‖ > 0. The proof is completed. Lemma 5. Assume that (H2) and (H3) hold. If ‖gk‖ > 0, then the new Armijo-type line search II is well-defined for the index k. Proof. The lemma is also proved by contradiction. Suppose that the conclusion does not hold; then for k, the inequality (11) does not hold for any nonnegative integer m; that is, f (xk + ρ m dk) > f (xk) − μρ 󵄩󵄩󵄩dk 󵄩󵄩󵄩 4 , ∀m. (17) That is, f (xk + ρ m dk) − f (xk) ρ > −μρ 󵄩󵄩󵄩dk 󵄩󵄩󵄩 4 , ∀m. (18) Letting m → +∞, by the continuity of f(x) and −g k dk = ‖gk‖ , we can obtain − 󵄩󵄩󵄩gk 󵄩󵄩󵄩 2 ≥ 0, (19) that is, 󵄩󵄩󵄩gk 󵄩󵄩󵄩 = 0, (20) which contradicts to ‖gk‖ > 0. The proof is completed. 3. Strongly Global Convergence Throughout this section, we assume that ‖gk‖ > 0, for all k ≥ 0; otherwise a stationary point of the objective function f(x) has been found. 3.1. Global Convergence of SDPRPMethodwith the Line Search I. We first prove the global convergent of SDPRP method with the Armijo-type line search I. Lemma 6. For all k ≥ 0, one has 󵄩󵄩󵄩dk 󵄩󵄩󵄩 ≤ (1 + 2L (1 − c) m0 ) 󵄩󵄩󵄩gk 󵄩󵄩󵄩 , ∀k, (21) wherem0 is defined in Lemma 3. Proof. If k = 0 then 󵄩󵄩󵄩dk 󵄩󵄩󵄩 = 󵄩󵄩󵄩gk 󵄩󵄩󵄩 ≤ (1 + 2L (1 − c) m0 ) 󵄩󵄩󵄩gk 󵄩󵄩󵄩 . (22) If k ≥ 1 then, from (3), (4), and (H2), we can get that 󵄩󵄩󵄩dk + gk 󵄩󵄩󵄩 = 󵄩󵄩󵄩󵄩 β PRP k dk−1 − θkyk−1 󵄩󵄩󵄩󵄩 ≤ 2 󵄩󵄩󵄩gk − gk−1 󵄩󵄩󵄩 󵄩󵄩󵄩dk−1 󵄩󵄩󵄩 󵄩󵄩󵄩gk−1 󵄩󵄩󵄩 2 󵄩󵄩󵄩gk 󵄩󵄩󵄩 ≤ 2Lαk−1 󵄩󵄩󵄩dk−1 󵄩󵄩󵄩 2 󵄩󵄩󵄩gk−1 󵄩󵄩󵄩 2 󵄩󵄩󵄩gk 󵄩󵄩󵄩 ≤ 2Lδk−1 󵄩󵄩󵄩dk−1 󵄩󵄩󵄩 2 󵄩󵄩󵄩gk−1 󵄩󵄩󵄩 2 󵄩󵄩󵄩gk 󵄩󵄩󵄩 ≤ 2L (1 − c) m0 󵄩󵄩󵄩gk 󵄩󵄩󵄩 , (23) which together with the triangular inequality implies that 󵄩󵄩󵄩dk 󵄩󵄩󵄩 ≤ 󵄩󵄩󵄩dk + gk 󵄩󵄩󵄩 + 󵄩󵄩󵄩gk 󵄩󵄩󵄩 ≤ (1 + 2L (1 − c) m0 ) 󵄩󵄩󵄩gk 󵄩󵄩󵄩 . (24) This completes the proof. The following lemma shows that the stepsize sequence {αk} generated by the Armijo-type line search I is bounded from below. Lemma 7. For all k ≥ 0, there exists a constant C > 0, such that αk ≥ C, (25) in which αk is generated by the Armijo-type line search I. Proof. We divide the proof into two cases: αk = δk and αk < δk. For the first case, by (12) and (21), we get αk ≥ (1 − c) M0 (1 + 2L (1 − c) m0 ) −2


Introduction
In this paper, we are concerned with the following unconstrained minimization problem: min  () ,  ∈   , where  :   →  1 is a smooth function whose gradient ∇() is often denoted by ().The related problem is called large-scale minimization problem when its dimension  is very large (e.g.,  > 10 6 ).For solving large-scale minimization problems, the matrices-free methods are quite efficient.Among such methods, the conjugate gradient method is very famous for its excellent numerical performance in the practical computation.Much progress has been achieved in the study of global convergence of the various conjugate gradient methods, such as the Polak-Ribière-Polyak (PRP) [1,2], the Fletcher-Reeves (FR) [3], the Hestenes-Stiefel (HS) [4,5], and the Dai-Yuan (DY) [6] conjugate gradient methods, et al.
Recently, Zhang et al. [7] presented a sufficient descent Polak-Ribière-Polyak (SDPRP) conjugate gradient method for solving large-scale problem (1), whose most important property is that its generated direction is always a sufficient descent direction for the objective function.Moreover, this property is independent of the line search used, and it reduces to the classical PRP method when the exact line search is used.The iterative process of the SDPRP method is given by  +1 =   +     ,  = 0, 1, . . ., where   is the current iterate,   > 0 is called the stepsize which can be obtained by some line search techniques, such as the Armijo line search, the Goldstein line search, and the (strong) Wolfe line search, and   is the search direction determined by with where It is easy to deduce from (3) and ( 4) that which indicates that   is a sufficient descent direction of () at the current iterate   if ‖  ‖ ̸ = 0; that is,   is not a stationary point of the objective function ().It has been proved that SDPRP method has global convergence under an Armijo-type line search [7] in the sense that lim inf which means that at least one cluster point of the sequence {  } is a stationary point if it is bounded.In another recent paper, Shi and Shen [8] showed that the classical PRP method in [1] has strong convergence and linear convergence rate under a customized Armijo-type line search, which is somewhat complicated.The new Armijotype line search ensures that the search direction generated by the classical PRP method possesses the sufficient descent property, which is helpful to prove the global convergence.
In this paper, motivated by the Armijo-type line search in [8], we first propose a similar but simple line search, which can ensure that the SDPRP method has strongly global convergence in the sense that lim that is, any cluster point of the sequence {  } is a stationary point of the objective function ().Noting that the above new line search needs to estimate the Lipschitz constant, which is not easy even for linear function, we present another Armijo-type line search, which is motivated by the line search in [7].This new line search can also guarantee the global convergence of the SDPRP method in the above sense.The remainder of the paper is organized as follows.In Section 2 we introduce the two new Armijo-type line searches and present the strongly convergent SDPRP method.The global convergence is established under the above two new Armijo-type line searches in Section 3. Some numerical results are presented in Section 4, and in the last section, we conclude the paper with some remarks.
Lemma 4. Assume that (H1) and (H2) hold.If ‖  ‖ > 0, then the new Armijo-type line search I is well-defined for the index .
Proof.The proof is easy; for completeness, we give the proof here.In fact, we can prove this lemma by contradiction.Suppose that the conclusion does not hold; then for , the inequality (10)  Proof.The lemma is also proved by contradiction.Suppose that the conclusion does not hold; then for , the inequality (11) does not hold for any nonnegative integer ; that is, That is, Letting  → +∞, by the continuity of () and − ⊤    = ‖  ‖ 2 , we can obtain that is, which contradicts to ‖  ‖ > 0. The proof is completed.

Strongly Global Convergence
Throughout this section, we assume that ‖  ‖ > 0, for all  ≥ 0; otherwise a stationary point of the objective function () has been found.This completes the proof.

Global Convergence of SDPRP
The following lemma shows that the stepsize sequence {  } generated by the Armijo-type line search I is bounded from below.Lemma 7.For all  ≥ 0, there exists a constant  > 0, such that in which   is generated by the Armijo-type line search I.
Proof.We divide the proof into two cases:   =   and   <   .For the first case, by ( 12) and (21), we get For the second case, that is   <   , which indicates that   / does not satisfy (10) Therefore, we have that Obviously, ( 26) and (30) show that (25) holds with This completes the proof.
We are now ready to establish the strong convergence of SDPRP method using the Armijo-type line search I. Proof.Since the generated sequence {  } ⊆  0 and the objection function () is bounded below on the level set  0 , by (10) and (25), we have This completes the proof.(11); that is

Global Convergence of SDPRP
From the mean value theorem and (H2), there exists a constant   ∈ (0, 1), such that which together with (36) shows that (35) holds.This completes the proof.
We are now ready to establish the strong convergence of SDPRP method using the Armijo-type line search II.The proof is motivated by the proof of Theorem 2.2 in [10].
Theorem 10.Suppose that (H2) and (H3) hold.Then Proof.For the sake of contradiction, we suppose that the conclusion is not right.Then there exist a constant  > 0 and an infinite index set  such that Moreover, the fact   ≤ 1, (35) and (H2) imply that This and (42) indicate that there exists a positive constant  1 such that for sufficiently large  ∈ , we have Then by ( 36) and (44), we can get lim which contradicts (45).The proof is then completed.

Numerical Results
In this section, we present some numerical results to compare the performance of SDPRP method with the two new Armijotype line searches I and II and the three-term PRP method in [7].
(iii) TTPRP: the two-term PRP method with the following Armijo-type line search: let   be the largest  in {1, ,  2 , . ..} such that where  = 10 −4 ,  = 0.5.All codes were written in Matlab 7.1 and run on a portable computer.We stopped the iteration if the number of iteration exceeds 10000 or ‖  ‖ < 10 −5 .Tables 1 and 2 list the numerical results for solving some test problems numbered from 1 to 30 in [11]   CPU mean the number of iterations, the number of function evaluations, and the CPU time in seconds, respectively.
Figures 1 and 2 show the performance of these methods relative to the number of function evaluations and CPU time, respectively, which are evaluated using the profiles of Dolan and Moré [12].That is, for each method, we plot the fraction  of problems for which the method is within a factor  of the best time.The left side of the figure gives the percentage of the test problems for which a method is fastest; while the right side gives the percentage of the test problems that are successfully solved by each of the methods.The top curve is the method that solved most problems in a time that was within a factor  of the best time.Figures 1 and 2 show that SDPRPI method performs a little better than TTPRP method and obviously better than SDPRPII method.It solves about 72% and 63% of the problems with the smallest number of function evaluations and CPU time, respectively.Obviously, the performance of SDPRPII method is not so good, and, in the future, we will further study the corresponding line search.Of course, more numerical experiments should be carried out to test our proposed methods.

Conclusion
In this paper, we have proposed two new Armijo-type line searches and proved that the sufficient descent PRP method proposed by Zhang et al. is strongly global convergent with the two new line searches.Numerical results show that the SDPRP method with the proposed line searches is efficient for the test problems.

Figure 1 :
Figure 1: Performance profiles of three methods about the number of function evaluations.

Figure 2 :
Figure 2: Performance profiles of three methods about CPU time.
Proof.If   ̸ = 1, then    =   / does not satisfy Method with the Line Search II.Then, we prove the strongly global convergent of SDPRP method with the Armijo-type line search II.It is obvious that   ∈  0 for all  ≥ 0. Therefore, from the line search II, we have lim

Table 1 :
The results for the methods on the tested problems.
with different dimension .Our numerical results are listed in the form NI/NF/CPU, where the symbols NI, NF, and