^{1}

^{1}

^{2}

^{1}

^{2}

A mixed spectral CD-DY conjugate descent method for solving unconstrained optimization problems is proposed, which combines the advantages of the spectral conjugate gradient method, the CD method, and the DY method. Under the Wolfe line search, the proposed method can generate a descent direction in each iteration, and the global convergence property can be also guaranteed. Numerical results show that the new method is efficient and stationary compared to the CD (Fletcher 1987) method, the DY (Dai and Yuan 1999) method, and the SFR (Du and Chen 2008) method; so it can be widely used in scientific computation.

The purpose of this paper is to study the global convergence properties and practical computational performance of a mixed spectral CD-DY conjugate gradient method for unconstrained optimization without restarts, and with appropriate conditions.

Consider the following unconstrained optimization problem:

There are many kinds of iterative method that include the steepest descent method, Newton method, and conjugate gradient method. The conjugate direction method is a commonly used and effective method in optimization, and it only needs to use the information of the first derivative. However, it overcomes the shortcoming of the steepest descent method in the slow convergence and avoids the defects of Newton method in storage and computing the second derivative.

The original CD method was proposed by Fletcher [

Quite recently, Birgin and Martinez [

The observation of the above formula motivates us to construct a new formula; which combines the advantage of the spectral gradient method, CD method, and DY method as follows:

This paper is organized as follows. In Section

In order to establish the global convergence of the proposed method, we need the following assumption on objective function, which have been used often in the literature to analyze the global convergence of nonlinear conjugate gradient methods with inexact line search.

(i) The level set

In some neighborhood

Data

Compute

Let

Compute

Set

The following lemma shows that Algorithm

Let the sequences

The conclusion can be proved by induction. Since

From (

From (

The conclusion of the following lemma, often called the Zoutendijk condition, is used to prove the global convergence properties of nonlinear conjugate gradient methods. It was originally given by Zoutendijk [

Suppose that Assumption

Let the sequences

If

The following theorem proves the global convergence of the mixed spectral CD-DY conjugate gradient method with the Wolfe line search.

Suppose that Assumption

Suppose by contradiction that there exists a positive constant

From (

In this section, we report some numerical results. We used MATLAB 7.0 to test some problems that are from [

The numerical results of our tests are reported in the following table. The first column “Problem” represents the name of the tested problem in [

In order to rank the average performance of all above methods, one can compute the total number of function and gradient evaluation by the formula

By making use of (

From Table

The performance of the CD method, DY method, CD-DY method, and SFR method.

Problem | Dim | CD | DY | CD-DY | SFR |
---|---|---|---|---|---|

ROSE | 2 | 88/250/223 | 64/190/172 | 60/188/168 | 64/190/172 |

FROTH | 2 | — | 42/168/138 | 38/151/123 | — |

BADSCP | 2 | 682/1885/1637 | — | 2704/6550/6438 | — |

BADSCB | 2 | 272/941/800 | — | 726/2051/1786 | — |

BEALE | 2 | 73/177/155 | 75/186/164 | 68/175/145 | 75/186/164 |

JENSAM | 17/61/43 | 10/49/26 | 26/80/57 | 15/48/32 | |

HELIX | 3 | 56/157/132 | 37/118/98 | 50/145/120 | 37/118/98 |

BRAD | 3 | 75/224/189 | 66/208/177 | 37/120/98 | 66/208/177 |

SING | 4 | 454/1074/1009 | 2286/4555/4545 | 850/1894/1863 | 1476/2901/2891 |

WOOD | 4 | 184/438/399 | 100/291/240 | 139/396/337 | 100/291/240 |

KOWOSB | 4 | 173/516/449 | 536/1449/1271 | 144/421/365 | 504/1386/1211 |

BD | 4 | 43/169/132 | 39/158/121 | 28/144/113 | 37/152/116 |

WATSON | 5 | 89/279/239 | 127/348/299 | 158/438/373 | 128/352/304 |

BIGGS | 6 | 200/579/509 | 294/824/712 | 236/680/599 | 288/812/703 |

OSB2 | 11 | 3243/5413/5398 | 7006/11059/11048 | 584/1262/1226 | 7013/11102/11091 |

VAEDIM | 5 | 6/57/38 | 6/57/38 | 6/57/38 | 6/57/38 |

10 | 7/81/52 | 7/81/52 | 7/81/52 | 7/81/52 | |

PEN1 | 50 | 2209/2565/2536 | 1727/2117/2043 | 116/221/190 | 1727/2117/2043 |

100 | 62/223/182 | 31/157/121 | 31/167/131 | 31/157/121 | |

TRIG | 100 | — | 305/399/398 | 88/145/144 | 305/399/398 |

500 | — | 343/424/423 | 109/189/188 | 344/427/425 | |

ROSEX | 500 | 92/267/238 | 68/207/186 | 65/205/182 | 68/207/186 |

1000 | 98/287/255 | 68/207/186 | 65/205/182 | 68/207/186 | |

SINGX | 100 | 682/1593/1517 | 1488/3139/3073 | 1159/2366/2614 | 2411/5326/5084 |

1000 | 511/1245/1135 | 2092/4451/4321 | 1374/3104/3064 | 5213/10042/10032 | |

BV | 500 | 1950/2543/2542 | 4796/6823/6822 | 1311/2131/2130 | 4784/6793/6792 |

1000 | 632/833/832 | 414/449/448 | 414/449/448 | 414/449/448 | |

IE | 500 | 7/15/8 | 7/15/8 | 7/15/8 | 7/15/8 |

1000 | 7/15/8 | 7/15/8 | 7/15/8 | 7/15/8 | |

TRID | 500 | 52/112/107 | 49/106/101 | 43/94/89 | 49/106/101 |

1000 | 70/149/145 | 64/137/133 | 55/119/115 | 64/137/133 |

The geometric mean of these ratios for the CD method, the DY method and SFR method, over all the test problems is defined by

According to the above rule, it is clear that

Relative efficiency of the CD, DY, SFR, and the mixed spectral CD-DY methods.

CD | DY | SFR | CD-DY |
---|---|---|---|

1.3956 | 1.6092 | 1.6580 | 1 |

The authors wish to express their heartfelt thanks to the referees and the editor for their detailed and helpful suggestions for revising the paper. This work was supported by The Nature Science Foundation of Chongqing Education Committee (KJ091104).