Abstract and Applied Analysis

Volume 2013 (2013), Article ID 814912, 8 pages

http://dx.doi.org/10.1155/2013/814912

Research Article

A Self-Adjusting Spectral Conjugate Gradient Method for Large-Scale Unconstrained Optimization

Yuanying Qiu,¹ Dandan Cui,² Wei Xue,² and Gaohang Yu²

¹School of Foreign Languages, Gannan Normal University, Ganzhou 341000, China

²School of Mathematics and Computer Sciences, Gannan Normal University, Ganzhou 341000, China

Received 21 February 2013; Accepted 17 March 2013

Academic Editor: Guoyin Li

Copyright © 2013 Yuanying Qiu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

This paper presents a hybrid spectral conjugate gradient method for large-scale unconstrained optimization, which possesses a self-adjusting property. Under the standard Wolfe conditions, its global convergence result is established. Preliminary numerical results are reported on a set of large-scale problems in CUTEr to show the convergence and efficiency of the proposed method.

1. Introduction

Consider the following unconstrained optimization problem:

where is a nonlinear smooth function and its gradient is available. Conjugate gradient methods are very efficient for solving (1), especially when the dimension is large, and have the following iterative form:

where is a steplength obtained by a line search, and is the search direction defined by

where is a scalar and denotes the gradient of at point .

There are at least six formulas for , which are given below:

where and denotes the Euclidean norm. In the above six methods, HS, PR, and LS methods are especially efficient in real computations, but one may not globally converge for general functions. FR, CD, and DY methods are globally convergent, but they perform much worse. To combine the good numerical performance of HS method and the nice global convergence property of DY method, Dai and Yuan [1] proposed an efficient hybrid formula for which is defined as the following form:

Their studies suggested that the HSDY method (5) has the same advantage of avoiding the propensity of short steps as the HS method [1]. They also proved that the HSDY method with the standard wolfe line search produces a descent search direction at each iteration and converges globally. Descent condition may be crucial for the convergence analysis of conjugate gradient methods with inexact line searches [2, 3]. Further, there are some modified conjugate gradient methods [4–7] which possess the sufficiently descent property without any line search condition. Recently, Yu [8] proposed a spectral version of HSDY method:

where

with ,  . The numerical experiments show that this simple preconditioning technique benefits to its performance.

In this paper, based on a new conjugate condition [9], we propose a new hybrid spectral conjugate gradient method with defined by

where

A full description of DS-HSDY method is formally given as follows.

Algorithm 1 (DS-HSDY conjugate gradient method).

Data. Choose constants , , and . Given an initial point , set . Let .

Step 1. If , then stop.

Step 2. Determine satisfying the standard Wolfe condition:

Then update .

Step 3. Compute , and . Then update such as

Set and go to Step 1.

The rest of the paper is organized as follows. In the next section, we show that the DS-HSDY method possesses a self-adjusting property. In Section 3, we establish its global convergence result under the standard Wolfe line search conditions. Section 4 gives some numerical results on a set of large-scale unconstrained test problems in CUTEr to illustrate the convergence and efficiency of the proposed method. Finally we have a Conclusion section.

2. Self-Adjusting Property

In this section, we prove that the DS-HSDY method possesses a self-adjusting property. To begin with, we assume that

otherwise, a stationary point has been found, and define the two following important quantities:

The quantity shows the size of , where is a quantity showing the descent degree of . In fact, if , is a descent direction. Furthermore, if for some constant , then we have the sufficient descent condition

On the other hand, it follows from (12) that

Hence

Combining with (17) yields

Dividing both sides of (18) by and using (7), we obtain

It follows from (19) and the definitions of and that

Additionally, we assume that there exist positive constants and such that

then we have the following result.

Theorem 2. Consider the method (2), (8) and (12), where is a descent direction. If (21) holds, there exist positive constants , , and such that relations

hold for all .

Proof. Summing (20) over the iterates and noting that , we get

Since , it follows from (25) that

Equations (21), (26), and yield

Furthermore, we have

Thus (24) holds with .

Noting that and , it is easy to derive that (22) and (23) hold with and , respectively. Hence the proof is complete.

Theorem 3. Consider the method (2), (8), and (12), where is a descent direction. If (21) holds, then for any , there exist constants , , and such that, for any k, the relations

hold for at least values of .

Proof. The proof is similar to the Theorem  2 in [10], so we omit it here.

Therefore, by Theorems 2 and 3, it was shown that DS-HSDY method possesses a self-adjusting property which is independent of the line search and the function convexity.

3. Global Convergence

Throughout the paper, we assume that the following assumptions hold.

Assumption 1. (1) is bounded below in the level set ;

(2) in a neighborhood of , is differentiable and its gradient is Lipschitz continuous; namely, there exists a constant such that

Under Assumption 1 on , we could get a useful lemma.

Lemma 4. Suppose that is a starting point for which Assumption 1 holds. Consider any method in the form (2), where is a descent direction and satisfies the weak Wolfe conditions; then one has that

For DS-HSDY method, one has the following global convergence result.

Theorem 5. Suppose that is a starting point for which Assumption 1 hold. Consider DS-HSDY method; if for all , then one has that

Further, the method converges in the sense that

Proof. Since , it is obvious that . Assume that . By (10) and the definition of the , we have , then . In addition, from (8), we have

Let , then we have . By (12) with replaced by , and multiplying it by , we have

From this and the formula for , we get

where

At the same time, if we define

it follows from (39) that

Then we have by (10), with replaced by , that

Furthermore, we have

The above relation, (40), (41), and the fact that imply that . Thus by induction, (32) holds.

We now prove (33) by contradiction and assume that there exists some constant such that

Since , we have that

Dividing both sides of (44) by and using (36) and (40), we obtain

In addition, since and , we have that , or equivalently

which with (37) yields

By (45) and (47), we obtain

Using (48) recursively and noting that ,

Then we get from this and (43) that

which indicates

This contradicts the Zoutendijk condition (31). Hence we complete the proof.

4. Numerical Result

In this section, we compare the performance of DS-HSDY method to method [11], HSDY method [1], and S-HSDY method [8]. The test problems are taken from CUTEr (http://hsl.rl.ac.uk/cuter-www/problems.html) with the standard initial points. All codes are written in double precision Fortran and complied with f77 (default compiler settings) on a PC (AMD Athlon XP 2500 + CPU 1.84 GHz). Our line search subroutine computes such that the Wolfe conditions (10) and (11) hold with and . We use the condition or as the stopping criterion. The numerical results are presented in Tables 1, 2, 3, and 4 with the form NI/Nfg/T, where we report the dimension of the problem (), the number of iteration (NI), the number of function evaluations (Nfg), and the CPU time () in 0.01 seconds.

Table 1: Numerical results for PRP⁺ method.

Function

NI
Nfg
(0.01 S)

Quadratic QF2
10000
2227
2885
2016

Extended EP1
10000
4
7
3

Extended Tridiagonal 2
10000
39
98
47

ARGLINA
10000
5
15
4

ARWHEAD
10000
7
14
21

BDQRTIC
5000
157
720
526

BDEXP
5000
6
8
7

BRYBND
5000
5
11
1215

COSINE
10000
21
45
39

CRAGGLVY
10000
129
250
444

DIXMAANA
10000
6
12
19

DIXMAANB
10000
8
16
26

DIXMAANC
10000
11
23
38

DIXMAAND
10000
13
29
44

DIXMAANE
5000
558
799
712

DIXMAANF
5000
558
598
525

DIXMAANG
5000
519
784
684

DIXMAANH
5000
379
3488
2469

DIXMAANI
5000
593
854
755

DIXMAANJ
5000
492
751
651

DIXMAANK
5000
653
979
863

DQDRTIC
10000
11
23
19

DQRTIC
10000
33
57
42

EDENSCH
10000
26
90
78

EG2
10000
209
1426
473

ENGVAL1
10000
30
93
21

EXTROSNB
10000
29
63
25

FREUROTH
10000
61
145
81

LIARWHD
10000
25
49
36

NONDIA
10000
9
17
17

NONDQUAR
5000
1786
3258
1752

NONSCOMP
10000
5001
6799
9751

Table 2: Numerical results for HSDY method.

Function

NI
Nfg
(0.01 S)

Quadratic QF2
10000
1593
1902
1876

Extended EP1
10000
4
7
4

Extended Tridiagonal 2
10000
34
55
32

ARGLINA
10000
5
15
4

ARWHEAD
10000
13
58
71

BDQRTIC
5000
171
567
422

BDEXP
5000
6
8
6

BRYBND
5000
5
11
1222

COSINE
10000
21
46
41

CRAGGLVY
10000
109
255
434

DIXMAANA
10000
5
10
16

DIXMAANB
10000
9
18
29

DIXMAANC
10000
10
21
33

DIXMAAND
10000
13
29
45

DIXMAANE
5000
446
541
493

DIXMAANF
5000
389
876
690

DIXMAANG
5000
552
660
602

DIXMAANH
5000
202
5106
3417

DIXMAANI
5000
365
450
409

DIXMAANJ
5000
444
532
484

DIXMAANK
5000
367
452
410

DQDRTIC
10000
8
17
14

DQRTIC
10000
37
68
48

EDENSCH
10000
30
99
85

EG2
10000
305
2811
879

ENGVAL1
10000
30
52
21

EXTROSNB
10000
27
54
21

FREUROTH
10000
143
283
177

LIARWHD
10000
32
62
44

NONDIA
10000
7
14
14

NONDQUAR
5000
2049
3730
2011

NONSCOMP
10000
58
100
98

Table 3: Numerical results for S-HSDY method.

Function

NI
Nfg
(0.01 S)

Quadratic QF2
10000
1582
1941
1836

Extended EP1
10000
4
7
3

Extended Tridiagonal 2
10000
34
55
34

ARGLINA
10000
5
15
3

ARWHEAD
10000
13
58
75

BDQRTIC
5000
111
526
377

BDEXP
5000
6
8
5

BRYBND
5000
5
11
1179

COSINE
10000
21
46
39

CRAGGLVY
10000
103
189
332

DIXMAANA
10000
5
10
15

DIXMAANB
10000
9
18
30

DIXMAANC
10000
10
21
33

DIXMAAND
10000
13
29
43

DIXMAANE
5000
422
514
468

DIXMAANF
5000
310
792
618

DIXMAANG
5000
410
495
449

DIXMAANH
5000
217
6957
4642

DIXMAANI
5000
380
450
417

DIXMAANJ
5000
359
438
402

DIXMAANK
5000
404
485
448

DQDRTIC
10000
8
17
14

DQRTIC
10000
37
68
49

EDENSCH
10000
30
99
84

EG2
10000
242
1731
570

ENGVAL1
10000
29
124
22

EXTROSNB
10000
27
54
22

FREUROTH
10000
214
408
260

LIARWHD
10000
27
54
37

NONDIA
10000
7
14
14

NONDQUAR
5000
1782
3210
1738

NONSCOMP
10000
58
100
100

Table 4: Numerical results for DS-HSDY method.

Function

NI
Nfg
(0.01 S)

Quadratic QF2
10000
1623
1978
1783

Extended EP1
10000
4
7
3

Extended Tridiagonal 2
10000
34
55
30

ARGLINA
10000
5
15
3

ARWHEAD
10000
13
58
70

BDQRTIC
5000
165
448
324

BDEXP
5000
6
8
4

BRYBND
5000
5
11
990

COSINE
10000
14
38
28

CRAGGLVY
10000
110
150
266

DIXMAANA
10000
5
10
16

DIXMAANB
10000
9
18
27

DIXMAANC
10000
10
21
31

DIXMAAND
10000
13
29
44

DIXMAANE
5000
410
493
430

DIXMAANF
5000
432
546
469

DIXMAANG
5000
476
582
505

DIXMAANH
5000
442
1204
7792

DIXMAANI
5000
397
467
408

DIXMAANJ
5000
445
594
503

DIXMAANK
5000
403
507
438

DQDRTIC
10000
10
21
17

DQRTIC
10000
35
62
43

EDENSCH
10000
29
87
70

EG2
10000
251
1121
381

ENGVAL1
10000
29
50
19

EXTROSNB
10000
65
122
44

FREUROTH
10000
50
133
67

LIARWHD
10000
47
94
61

NONDIA
10000
7
14
12

NONDQUAR
5000
1831
3262
1665

NONSCOMP
10000
73
126
119

Figure 1 shows the performance of these test methods relative to the CPU time, which were evaluated using the profiles of Dolan and Moré [12]. That is, for each method, we plot the fraction of problems for which the method is within a factor of the best time. The top curve is the method that solved the most problems in a time that was within a factor of the best time. Clearly, the left side of the figure gives the percentage of the test problems for which a method is the fastest. As we can see from Figure 1, DS-HSDY method has the best performance which performs better than S-HSDY method, HSDY method, and the well-known method.

Figure 1: Performance profiles for CPU time.

5. Conclusion

In this paper, we proposed an efficient hybrid spectral conjugate gradient method with self-adjusting property. Under some suitable assumptions, we established the global convergence result for the DS-HSDY method. Numerical results indicated that the proposed method is efficient for large-scale unconstrained optimization problems.

Acknowledgments

This work was partly supported by the National Natural Science Foundation of China (no. 61262026), the JGZX program of Jiangxi Province (20112BCB23027), and the science and technology program of Jiangxi Education Committee (LDJH12088). The authors would also like to thank the editor and an anonymous referees for their comments and suggestions on the first version of the paper, which led to significant improvements of the presentation.

References

Y. H. Dai and Y. Yuan, “An efficient hybrid conjugate gradient method for unconstrained optimization,” Annals of Operations Research, vol. 103, pp. 33–47, 2001.

Y. H. Dai and Y. Yuan, “A nonlinear conjugate gradient method with a strong global convergence property,” SIAM Journal on Optimization, vol. 10, no. 1, pp. 177–182, 1999.

Z. X. Wei, G. Y. Li, and L. Q. Qi, “Global convergence of the Polak-Ribière-Polyak conjugate gradient method with an Armijo-type inexact line search for nonconvex unconstrained optimization problems,” Mathematics of Computation, vol. 77, no. 264, pp. 2173–2193, 2008.

W. W. Hager and H. Zhang, “A new conjugate gradient method with guaranteed descent and an efficient line search,” SIAM Journal on Optimization, vol. 16, no. 1, pp. 170–192, 2005.

L. Zhang, W. Zhou, and D.-H. Li, “A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence,” IMA Journal of Numerical Analysis, vol. 26, no. 4, pp. 629–640, 2006.

G. Yuan, “Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems,” Optimization Letters, vol. 3, no. 1, pp. 11–21, 2009.

G. Yu, L. Guan, and W. Chen, “Spectral conjugate gradient methods with sufficient descent property for large-scale unconstrained optimization,” Optimization Methods and Software, vol. 23, no. 2, pp. 275–293, 2008.

G. Yu, Nonlinear self-scaling conjugate gradient methods for large-scale optimization problems [Ph.D. thesis], Sun Yat-Sen University, Guangzhou, China, 2007.

G. Li, C. Tang, and Z. Wei, “New conjugacy condition and related new conjugate gradient methods for unconstrained optimization,” Journal of Computational and Applied Mathematics, vol. 202, no. 2, pp. 523–539, 2007.

Y.-H. Dai, “New properties of a nonlinear conjugate gradient method,” Numerische Mathematik, vol. 89, no. 1, pp. 83–98, 2001.

J. C. Gilbert and J. Nocedal, “Global convergence properties of conjugate gradient methods for optimization,” SIAM Journal on Optimization, vol. 2, no. 1, pp. 21–42, 1992.

E. D. Dolan and J. J. Moré, “Benchmarking optimization software with performance profiles,” Mathematical Programming, vol. 91, no. 2, pp. 201–213, 2002.