^{1}

A new nonlinear spectral conjugate descent method for solving unconstrained optimization problems is proposed on the basis of the CD method and the spectral conjugate gradient method. For any line search, the new method satisfies the sufficient descent condition

Unconstrained optimization problems have extensive applications, for example, in petroleum exploration, aerospace, transportation, and other domains. However, the amount of necessary calculation also grows exponentially with the increasing scale of the problem. Therefore, it is required to develop new methods to solve the large-scale unconstrained optimization problems. The primary objective of this paper is to study the global convergence properties and practical computational performance of a new nonlinear spectral conjugate gradient method for unconstrained optimization problems without restarts, and with suitable conditions.

Consider the following unconstrained optimization problem

Due to need less computer memory especially, conjugate gradient method is very appealing for solving (

The original CD method proposed by Fletcher [

Another popular method to solving problem (

In this paper, motivated by success of the spectral gradient method, we propose a new spectral conjugate gradient method by combining the CD method and the spectral gradient method. The direction is given by the following way:

This paper is organized as follows. In Section

In order to establish the global convergence of our method, we need the following assumption on objective function, which have often been used in the literatures to analyze the global convergence of nonlinear conjugate gradient method and the spectral conjugate gradient method with inexact line searches.

(i) The level set

(ii) In some neighborhood

The following theorem shows that Algorithm

Let the sequences

We can prove the conclusion by induction. From

If

From (

From (

The conclusion of the following lemma, often called the Zoutendijk condition, is used to prove the global convergence of nonlinear conjugate gradient methods. It was originally given by Zoutendijk [

Suppose that Assumption

The following theorem establishes the global convergence of the new spectral conjugate gradient method with the strong Wolfe line search for the general functions.

Suppose that (Assumption

According to the given conditions, Lemma

In this section, we report some numerical results. Under the strong Wolfe line search, we compare the performances of the CPU time and the iteration number of the SCD method with that of CD, FR, and PRP methods on the given test problems which come from the CUTE test problem library [

In Table

The numerical results of SCD method, CD method, FR method, and PRP method.

Problem | Dim | SCD method | CD method | FR method | PRP method |
---|---|---|---|---|---|

Extended Freudenstein and Roth | 1000 | 815/0.58 | 947/1.20 | 499/0.56 | 13/0.02 |

3000 | 1450/3.26 | 1677/6.35 | 406/1.50 | 1542/6.10 | |

5000 | 1447/6.22 | 1700/10.92 | 537/3.18 | 1566/10.26 | |

Extended trigonometric | 5000 | 32/0.03 | 817/1.13 | 342/0.25 | 73/0.07 |

3000 | 31/0.08 | 119/0.22 | 284/0.56 | 87/0.19 | |

5000 | 36/0.16 | 310/0.97 | 346/1.10 | 38/0.15 | |

Extended Beale | 1000 | 24/0.00 | 44/0.00 | 24/0.00 | 13/0.02 |

3000 | 33/0.02 | 17/0.02 | 24/0.00 | 13/0.00 | |

5000 | 30/0.03 | 43/0.05 | 20/0.04 | 19/0.02 | |

Extended penalty | 1000 | 65/0.05 | 29/0.01 | 16/0.00 | 96/0.11 |

3000 | 10/0.00 | 16/0.00 | 15/0.00 | 14/0.00 | |

5000 | 11/0.02 | 22/0.05 | 35/0.13 | 12/0.01 | |

Perturbed quadratic | 1000 | 334/0.05 | 1847/0.26 | 1088/0.17 | 407/0.08 |

3000 | 679/0.32 | 1736/0.74 | 3191/1.27 | 705/0.31 | |

5000 | 922/0.68 | 1494/1.02 | 2830/1.84 | 1081/0.80 | |

Raydan 1 | 1000 | 331/0.12 | 680/0.25 | 799/0.25 | 450/0.18 |

3000 | 601/0.77 | 1542/1.57 | 1044/0.94 | 760/0.94 | |

5000 | 1229/2.35 | 1857/3.04 | 4192/6.34 | 1058/2.16 | |

Raydan 2 | 1000 | 4/0.00 | 4/0.00 | 4/0.00 | 4/0.00 |

3000 | 4/0.01 | 4/0.02 | 4/0.02 | 4/0.00 | |

5000 | 4/0.02 | 4/0.01 | 4/0.01 | 4/0.02 | |

Hager | 1000 | 434/4.23 | 699/2.53 | 1450/16.01 | 183/1.48 |

3000 | 1404/47.23 | 1591/33.20 | 2193/74.72 | 1019/34.11 | |

5000 | 3163/171.28 | 946/39.49 | 4332/234.59 | 2583/142.81 | |

Generalized tridiagonal 1 | 1000 | 26/0.00 | 72/0.07 | 87/0.05 | 43/0.04 |

3000 | 55/0.14 | 59/0.22 | 167/0.62 | 62/0.20 | |

5000 | 89/0.56 | 61/0.31 | 38/0.09 | 34/0.11 | |

Extended tridiagonal 1 | 1000 | 34/0.00 | 15/0.00 | 86/0.02 | 14/0.00 |

3000 | 10/0.02 | 67/0.03 | 22/0.01 | 12/0.00 | |

5000 | 9/0.02 | 79/0.06 | 16/0.01 | 16/0.00 | |

Extended three expo terms | 1000 | 14/0.01 | 97/1.71 | 23/0.08 | 8/0.00 |

3000 | 15/0.05 | 86/4.27 | 49/1.72 | 8/0.03 | |

5000 | 64/4.88 | 123/11.48 | 247/23.04 | 8/0.06 | |

Generalized tridiagonal 2 | 1000 | 67/0.01 | 370/0.07 | 234/0.05 | 58/0.00 |

3000 | 47/0.04 | 770/0.40 | 281/0.18 | 54/0.04 | |

5000 | 62/0.04 | 261/0.22 | 139/0.11 | 60/0.06 | |

Diagonal 4 | 1000 | 7/0.02 | 4/0.00 | 6/0.00 | 4/0.00 |

3000 | 4/0.00 | 4/0.00 | 6/0.02 | 4/0.00 | |

5000 | 9/0.00 | 6/0.02 | 7/0.00 | 4/0.00 | |

Diagonal 5 | 1000 | 4/0.01 | 4/0.00 | 4/0.01 | 4/0.02 |

3000 | 4/0.02 | 4/0.04 | 4/0.01 | 4/0.01 | |

5000 | 4/0.03 | 4/0.05 | 4/0.04 | 4/0.03 | |

Extended Himmelblau | 1000 | 10/0.00 | 17/0.00 | 17/0.00 | 23/0.02 |

3000 | 11/0.00 | 131/0.06 | 19/0.01 | 24/0.02 | |

5000 | 12/0.02 | 103/0.07 | 20/0.01 | 24/0.02 | |

Extended PSC1 | 1000 | 12/0.02 | 40/0.28 | 17/0.04 | 11/0.00 |

3000 | 11/0.03 | 51/1.18 | 91/2.36 | 18/0.22 | |

5000 | 9/0.03 | 67/2.75 | 39/1.28 | 11/0.03 | |

Extended block-diagonal BD1 | 1000 | 67/0.02 | 71/0.02 | 32/0.02 | 26/0.01 |

3000 | 69/0.06 | 24/0.03 | 29/0.03 | 31/0.07 | |

5000 | 71/0.13 | 23/0.05 | 35/0.06 | 28/0.05 | |

Extended Maratos | 1000 | 981/0.14 | 227/0.03 | 617/0.08 | 59/0.00 |

3000 | 781/0.35 | 313/0.12 | 672/0.27 | 55/0.02 | |

5000 | 715/0.55 | 262/0.17 | 690/0.45 | 49/0.05 | |

Extended Cliff | 1000 | 41/0.01 | 48/0.03 | 154/0.58 | 27/0.01 |

3000 | 51/0.09 | 303/2.22 | 206/4.23 | 39/0.06 | |

5000 | 17/0.04 | 61/1.08 | 113/2.89 | 20/0.06 | |

Quadratic diagonal perturbed | 1000 | 234/0.03 | 456/0.08 | 499/0.08 | 400/0.06 |

3000 | 978/0.45 | 2999/1.57 | 702/0.36 | 817/0.46 | |

5000 | 807/0.64 | 1157/1.03 | 1092/0.91 | 1023/0.89 | |

Extended Wood | 1000 | 661/0.13 | 68/0.02 | 96/0.02 | 107/0.03 |

3000 | 491/0.25 | 113/0.06 | 60/0.03 | 150/0.08 | |

5000 | 514/0.42 | 93/0.08 | 67/0.05 | 206/0.17 | |

Quadratic QF1 | 1000 | 344/0.05 | 1120/0.14 | 949/0.11 | 363/0.05 |

3000 | 607/0.25 | 1677/0.64 | 3576/1.29 | 731/0.28 | |

5000 | 1038/0.72 | 1606/1.04 | 2467/1.47 | 1076/0.72 | |

Extended quadratic enalty QP2 | 1000 | 521/0.36 | 98/0.06 | 2476/1.17 | 28/0.01 |

3000 | 813/1.65 | 191/0.30 | 251/0.37 | 40/0.10 | |

5000 | 561/1.89 | 156/0.39 | 242/0.61 | 35/0.14 | |

Quadratic QF2 | 1000 | 1188/0.19 | 1469/0.20 | 1679/0.21 | 433/0.06 |

3000 | 2949/1.34 | 1867/0.75 | 2952/1.13 | 929/0.40 | |

5000 | 4161/3.25 | 2709/1.78 | 3840/2.39 | 1236/0.08 | |

Extended EP1 | 1000 | 2/V0 | 2/0.00 | 2/0.00 | 2/0.02 |

3000 | 3/0.00 | 3/0.00 | 3/0.00 | 3/0.00 | |

5000 | 3/0.00 | 3/0.00 | 3/0.00 | 3/0.00 | |

Extended tridiagonal 2 | 1000 | 41/0.00 | 137/0.10 | 78/0.01 | 36/0.00 |

3000 | 179/0.74 | 865/1.86 | 236/0.99 | 109/0.44 | |

5000 | 125/0.76 | 381/2.75 | 335/2.47 | 202/1.58 | |

ARWHEAD | 1000 | 5/0.00 | 58/0.03 | 44/0.01 | 5/0.02 |

3000 | 7/0.00 | 81/0.11 | 31/0.03 | 15/0.04 | |

5000 | 13/0.03 | 50/0.18 | 54/0.16 | 31/0.14 | |

NONDIA | 1000 | 16/0.00 | 47/0.02 | 50/0.00 | 10/0.01 |

3000 | 16/0.00 | 10/0.02 | 11/0.00 | 12/0.00 | |

5000 | 11/0.02 | 11/0.01 | 14/0.00 | 13/0.01 | |

DQDRTIC | 1000 | 33/0.00 | 30/0.00 | 7/0.00 | 16/0.00 |

3000 | 41/0.02 | 10/0.00 | 7/0.00 | 11/0.00 | |

5000 | 35/0.02 | 34/0.02 | 7/0.00 | 9/0.02 | |

DIXMAANA | 1000 | 7/0.02 | 11/0.03 | 12/0.03 | 9/0.03 |

3000 | 8/0.02 | 11/0.01 | 13/0.02 | 9/0.01 | |

5000 | 9/0.02 | 11/0.02 | 13/0.01 | 9/0.01 | |

DIXMAANB | 1000 | 11/0.00 | 12/0.00 | 12/0.02 | 12/0.02 |

3000 | 12/0.00 | 12/0.02 | 12/0.01 | 12/0.02 | |

5000 | 12/0.03 | 12/0.01 | 12/0.03 | 13/0.03 | |

DIXMAANC | 1000 | 14/0.00 | 14/0.00 | 16/0.00 | 15/0.00 |

3000 | 15/0.02 | 17/0.02 | 17/0.01 | 16/0.01 | |

5000 | 15/0.01 | 16/0.01 | 17/0.03 | 16/0.04 | |

DIXMAANE | 1000 | 273/0.09 | 1021/0.31 | 579/0.16 | 246/0.07 |

3000 | 521/0.80 | 756/0.70 | 712/0.55 | 484/0.47 | |

5000 | 680/1.44 | 1066/1.50 | 1013/1.28 | 666/1.09 | |

Partial perturbed quadratic PPQ1 | 1000 | 371/3.19 | 707/5.31 | 664/3.86 | 450/4.08 |

3000 | 370/30.39 | 543/45.59 | 802/49.48 | 517/44.19 | |

5000 | 287/65.78 | 523/113.13 | 787/133.55 | 276/63.87 | |

Broyden tridiagonal | 1000 | 41/0.00 | 346/0.08 | 2167/0.34 | 45/0.02 |

3000 | 88/0.05 | 429/0.20 | 497/0.22 | 86/0.05 | |

5000 | 86/0.08 | 734/0.58 | 344/0.26 | 81/0.06 | |

Almost perturbed quadratic | 1000 | 410/0.06 | 809/0.11 | 990/0.13 | 384/0.06 |

3000 | 708/0.31 | 1436/0.57 | 2255/0.85 | 768/0.31 | |

5000 | 848/0.61 | 1770/1.19 | 2400/1.49 | 967/0.66 | |

Tridiagonal perturbed quadratic | 1000 | 351/0.06 | 1028/0.15 | 1265/0.19 | 358/0.06 |

3000 | 663/0.33 | 1330/0.59 | 2074/0.87 | 549/0.25 | |

5000 | 987/0.95 | 1667/1.22 | 3034/2.11 | 1260/0.97 | |

EDENSCH | 1000 | 162/0.28 | 261/0.25 | 128/0.21 | 93/0.16 |

3000 | 45/0.11 | 178/0.89 | 145/0.72 | 47/0.14 | |

5000 | 65/0.42 | 318/1.32 | 164/1.38 | 54/0.25 | |

VARDIM | 1000 | 16/0.02 | 16/0.01 | 16/0.02 | 16/0.00 |

3000 | 19/0.01 | 19/0.02 | 19/0.01 | 19/0.01 | |

5000 | 14/0.01 | 14/0.02 | 14/0.01 | 14/0.01 | |

Diagonal 6 | 1000 | 4/0.00 | 4/0.00 | 4/0.02 | 4/0.01 |

3000 | 4/0.02 | 4/0.02 | 4/0.01 | 4/0.02 | |

5000 | 4/0.02 | 4/0.00 | 4/0.02 | 4/0.01 | |

DIXMAANF | 1000 | 279/0.16 | 587/0.23 | 338/0.12 | 239/0.11 |

3000 | 673/1.44 | 869/0.83 | 599/0.46 | 348/0.36 | |

5000 | 742/2.45 | 692/1.01 | 2588/6.44 | 584/0.95 | |

DIXMAANG | 1000 | 293/0.12 | 673/0.29 | 368/0.11 | 257/0.08 |

3000 | 989/1.48 | 1449/1.35 | 739/0.62 | 448/0.47 | |

5000 | 507/0.97 | 1671/3.17 | 997/1.39 | 742/1.29 | |

DIXMAANH | 1000 | 272/0.11 | 632/1.41 | 1010/0.28 | 247/0.10 |

3000 | 733/1.43 | 653/0.64 | 513/0.41 | 1771/29.08 | |

5000 | 898/3.06 | 604/1.44 | 1296/1.74 | 588/2.11 | |

DIXMAANI | 1000 | 231/0.08 | 876/0.25 | 426/0.11 | 268/0.09 |

3000 | 439/0.50 | 1261/1.06 | 628/0.50 | 542/0.52 | |

5000 | 468/0.81 | 1071/1.59 | 943/1.22 | 654/1.05 | |

DIXMAANJ | 1000 | 266/0.13 | 488/0.14 | 362/0.11 | 281/0.09 |

3000 | 768/1.56 | 720/3.23 | 516/0.41 | 385/0.39 | |

5000 | 471/0.83 | 1250/2.15 | 1589/2.08 | 547/1.64 | |

DIXMAANK | 1000 | 260/0.11 | 1036/0.36 | 500/0.14 | 278/0.11 |

3000 | 420/0.47 | 1103/1.13 | 774/0.62 | 428/0.42 | |

5000 | 620/1.59 | 1386/2.66 | 784/1.06 | 620/1.04 | |

ENGVAL1 | 1000 | 32/0.00 | 242/0.14 | 268/0.38 | 53/0.03 |

3000 | 80/0.28 | 366/1.06 | 950/4.67 | 229/1.05 | |

5000 | 340/2.56 | 267/1.64 | 325/2.37 | 181/1.30 | |

ENSCHNB | 1000 | 10/0.00 | 9/0.00 | 9/0.00 | 9/0.00 |

3000 | 8/0.01 | 9/0.01 | 9/0.02 | 9/0.01 | |

5000 | 9/0.02 | 9/0.02 | 9/0.02 | 8/0.00 | |

ENSCHNF | 1000 | 27/0.00 | 45/0.00 | 1206/1.51 | 23/0.00 |

3000 | 28/0.02 | 85/0.06 | 487/1.33 | 26/0.04 | |

5000 | 27/0.04 | 76/0.07 | 158/0.23 | 24/0.03 |

In this paper, we adopt the performance profiles by Dolan and Moré [

Performance profiles of the given methods with respect to CPU time.

Performance profiles of the given methods with respect to the number of iterations.

From Figures

The authors wish to express their heartfelt thanks to the anonymous referees and the editor for their detailed and helpful suggestions for revising the paper. This work was supported by The Nature Science Foundation of Chongqing Education Committee (KJ121112).