Journal of Mathematics
Volume 2017 (2017), Article ID 2715854, 12 pages
https://doi.org/10.1155/2017/2715854

Research Article

A New Modified Three-Term Conjugate Gradient Method with Sufficient Descent Property and Its Global Convergence

Bakhtawar Baluch,¹ Zabidin Salleh,¹ Ahmad Alhawarat,² and U. A. M. Roslan¹

¹School of Informatics and Applied Mathematics, Universiti Malaysia Terengganu, 21030 Kuala Nerus, Terengganu, Malaysia
²Department of Mathematics, College of Science, Al-Isra University, Amman, Jordan

Correspondence should be addressed to Zabidin Salleh; zabidin@umt.edu.my

Received 4 April 2017; Revised 30 June 2017; Accepted 3 August 2017; Published 13 September 2017

Academic Editor: Liwei Zhang

Copyright © 2017 Bakhtawar Baluch et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

A new modified three-term conjugate gradient (CG) method is shown for solving the large scale optimization problems. The idea relates to the famous Polak-Ribière-Polyak (PRP) formula. As the numerator of PRP plays a vital role in numerical result and not having the jamming issue, PRP method is not globally convergent. So, for the new three-term CG method, the idea is to use the PRP numerator and combine it with any good CG formula’s denominator that performs well. The new modification of three-term CG method possesses the sufficient descent condition independent of any line search. The novelty is that by using the Wolfe Powell line search the new modification possesses global convergence properties with convex and nonconvex functions. Numerical computation with the Wolfe Powell line search by using the standard test function of optimization shows the efficiency and robustness of the new modification.

1. Introduction

The conjugate gradient method is an efficient and organized tool for solving the large-scale nonlinear optimization problem, due to its simplicity, easiness, and low memory requirements. This method is very popular for mathematician and engineers and those who are interested in solving the large-scale optimization problems [1–3].

Consider the unconstrained optimization problem

where is continuously differentiable and its gradient is available . Generally CG method generates an iterative sequence defined by

where is a line search and is a search direction defined by

where the term is a scalar. There are six essential formulas for , which are stated as

(Hestenes and Stiefel [4], 1952),

(Fletcher and Reeves [5], 1964),

(Polak et al. [6, 7], 1969),

(Conjugate-Descent [8], 1997),

(Liu and Storey [9], 1991),

(Dai and Yuan [10], 2000).

Generally inexact line search is used in order to get the global convergence of conjugate gradient method, such as Wolfe line search or strong Wolfe line search; the Wolfe line search is given as

where The strong Wolfe line search is computing such that

Recently Alhawarat et al. [11, 12] and Alhawarat and Salleh [13, 14] have proposed an efficient and hybrid conjugate gradient method that satisfies the global convergence properties. To enhance the effectiveness of two-term conjugate gradient method, the three-term conjugate gradient has been widely studied and given much importance. The three-term conjugate gradient method attains different numerical outcomes, depending on how the scalar parameter is being selected. The papers by Beale [15], McGuire and Wolfe [16], Nazareth [17], Deng and Li [18], Dai and Yuan [19], Zhang et al. [20, 21], Cheng [22], Zhang et al. [23], Al-Bayati and Sharif [24], Narushima et al. [25], Andrei [26–28], Sugiki et al. [29], Al-Baali et al. [30], Babaie-Kafaki and Ghanbari [31], and Sun and Liu [32] presented different types of three-term conjugate gradient method along with their numerical performance and efficiency and proved their global convergence properties. As a comparison with classical conjugate gradient algorithms, the proposed three-term conjugate gradient algorithms are numerically strong, efficient, reliable, and robust compared to the classical conjugate gradient algorithms and Beale [15] was the first to propose the three-term conjugate gradient method.

In the new three-term modification, we put our attention on the numerator of PRP method, in which the parameter is given as

The PRP method is among one of the most efficient and reliable conjugate gradient method due to good numerical performance. The global convergence of PRP is established when the objective function is strongly convex and the line search is exact [6]. On the other hand Powell [33] through his analysis expressed that there exist nonconvex functions for which PRP method does not converge globally. Gilbert and Nocedal [34] established the so-called method; in this method is restricted to be nonnegative denoted as If the standard Wolfe line search (10) is used, thenmethod attains the global convergence and also sufficient descent conditions are being satisfied.

Recently Sun and Liu [32] proposed a new conjugate gradient method called TMPRP 1 method by using the VFR formula from Wei et al. [35], in which the search direction is stated as

where or .

This method has attractive property of satisfying the sufficient descent condition independent of any line search and attains global convergence if standard Wolfe line is used. As compared with the strong Wolfe line search, the standard Wolfe line search takes less computation in order to get an acceptable step size at each iteration. Hence the standard Wolfe line search increases the effectiveness of the conjugate gradient method [32].

The rest of the paper is organized as follows. In Section 2, the motivation and formula for construction of three-term conjugate gradient method are given. In Section 3 we have presented Algorithm  1.1 in which the general form of three-term conjugate gradient method is shown. In Section 4 the sufficient descent condition and the global convergence properties for convex and nonconvex function are proven. In Section 5, the detailed numerical results to test the proposed method are reported.

2. Motivation and Formula

Wei et al. [35] proposed three new formulas which are given in the following:

There is an efficient conjugate gradient method named In this formula, the denominator plays an important role in satisfying the sufficient descent condition and performs well in terms of global convergence and numerical result. This motivated us to take the denominator from this formula. Secondly, the PRP [6, 7] method is considered to be one of the most proficient CG parameters due to the properties of its numerator . If the step taken becomes very small, then reaches zero such as . Afterwards ; then the search direction continued as the steepest descent method. So the numerator of PRP method worked efficiently and does not jam.

This motivated us to construct a new modified three-term conjugate gradient method, such as

Further, Powell [36] showed that the PRP method can cycle infinitely without approaching a minimum point, even if the step size is chosen to the least positive minimizer. To overcome this, Gilbert and Nocedal [34] showed their analysis

So

In , , and , the parameters and have an important role in the sense that when is getting smaller, the numbers of iteration, function evaluation, and gradient evaluation are decreased and when is getting larger, the numbers of iteration, function evaluation, and gradient evaluation are also decreased. So we observe that the best value for the parameters is .

3. Algorithm  1.1

Step  0. Given an initial point , , , , and set , .

Step  1. If , where , then the algorithm stops; otherwise, go to Step  2.

(Note: all the norm we use in this paper means ).

Step  2. Compute the search direction (19) by using and where

where .

Step  3. Determine the step size by the Wolfe line search (10).

Step  4. Compute where is given in Step  3 and is given in Step  2.

Step  5. Set and go to Step  1.

4. Global Convergence of Modified Three Term

Assumptions 1. (A1) The level set is bounded.
(A2) In some neighborhood of , the gradient is Lipschitz continuous on an open convex set that contains ; that is, there exists a positive constant such that

then Assumptions (A1) and (A2) and [32, 34] imply that there exist positive constants and such that

Since is decreasing as , from Assumption (A1) it is shown that the sequence created by Algorithm  1.1 will be contained in a bounded region. Then the sequence is convergent.

Now we will prove the sufficient descent condition independent of line search and also . From (19),

Multiplying by , we obtain

that is,

Hence the sufficient descent condition independent of line search holds.

Now we prove that . As we have , by taking modulus on both sides

By Schwarz inequality we have

So,

Thus we have

Lemma 1. Assumptions (A1) and (A2) hold if is supposed to be an initial point. Now consider any method in the form of (2), in which is a descent direction and satisfies the Wolfe condition (10) or the strong Wolfe line search condition (11). Then we have the Zoutendijk condition:

which is normally used to prove the global convergence of CG method. From (29) the Zoutendijk condition is equivalent to the following inequality:

Definition 2 (see [32]). The function is called uniformly convex on , if there exists a positive constant such that

where the function has the Hessian matrix .

We now show the global convergence of Algorithm  1.1 for uniformly convex functions.

Lemma 3. Let both sequences and be generated by Algorithm  1.1 and suppose that (32) holds; then

where , is a positive constant, and is a positive number whose range is , from the Wolfe line search (10).

Proof. Detail of proof can be seen in Lemma of [37].

Theorem 4. Suppose that Assumptions (A1) and (A2) hold and the function is uniformly convex; then

Proof. From (15), (33), and (A2), we have

Now

From (18), (33), and (A2), we have

Combining (35) and (37) with (19),

Now letting , we get ,

So by (31), we get

We now show the global convergence for nonconvex functions.

Lemma 5. Let Assumptions (A1) and (A2) hold. Consider the sequence () to be generated by Algorithm  1.1. If there exists a positive constant in such a way that for every ,

where .

Proof. Since , , and for every , then we get for every , so that is well defined. If

then we get . Also and are unit vectors, so

as we know ,

Now from (18), (22), and (A2)

Now from (21), (22), and (45), there is a constant as follows:

Therefore, from (31) and (46), we have

This along with (44) completes the proof.

Theorem 6. Suppose that Assumptions (A1) and (A2) hold. Then the sequence () generated by Algorithm  1.1 satisfies

Proof. Suppose that the conclusion (48) is not true. Then there exists a positive constant in such a way that .
The proof has the following two parts.
Part 1. We noticed that for any and we have , such that

Proceeding the same proof of Theorem , step 1 from [32], we have

Part 2. Taking a bound on the direction . Now from (19) and (46) we have

At the beginning of proof we assume that ; then there exists a constant and also there exists such that . Then

which contradicts with (A2), (31), and (51). Hence it is proved that .

5. Numerical Results

In this part we compare the numerical results of proposed three-term BZAU (Bakhtawar, Zabidin, Ahmad and Ummu) method with recently developed TMPRP1 method and also compare their performance. The Wolfe line search (10) is used and the values of the parameters for BZAU and TMPRP1 method areμ= 2, 10⁽⁻⁴⁾;η= 1, 0; ρ = 0.1, 0.1; and σ = 0.5, 0.5, respectively. The code was written in Matlab 7.1 and run on an i5 computer with 2.40 GHz CPU processor, 2.0 GB RAM memory. We test the functions taken from [38] with dimension ranges . The main purpose in optimization for the selection of large number of test functions is to test the unconstrained optimization algorithms properly. Dantzig (1914–2005) said the final test of a theory is its capacity to solve the problems which originated it. This is one of the main reasons we select the large-scale unconstrained optimization problems to test the theoretical progress in numerical form through mathematical programming [38].

Moré et al. [39] claimed the efficiency of a method and that algorithm for a small number of test functions is not suitable because this will lead to the choice of an algorithm that is not favorable. Testing a method or algorithm for a large number of test functions would lead to large amount of data and from that data we can interpret which method or algorithm is more efficient and robust. But the number of test functions should not be very large nor very small, so there is a benchmark of 75 numbers of test functions which are chosen to test the efficiency of any method.

Practically, optimizers need to evaluate nonlinear optimization method. To prove the global convergence properties of any method, the theory is not enough to determine the reliability and efficiency of a method. As a result, the robustness of any method is established by testing the large number of test problems [38].

In global convergence property is used in case of proving convex function and property is used for proving nonconvex function. But in the numerical part is used for a comparison with TMPRP1 method. The TMPRP1 possesses the sufficient descent property without any line searches. Theoretically, TMPRP1 method established well and converges globally. When it comes to numerical computation, the TMPRP1 method is tested by a benchmark of 75 numbers of test functions and shows a promising result. Hence the TMPRP1 method is then compared with our BZAU method.

In Table 1 number of iterations, number of function evaluations, number of gradient evaluations, and CPU time are represented by NI/NF/GE/CT. If the CT exceeds 500 seconds and the NI is more than 10000 iterations, then the function is given the name of Fail F. This standard is followed by every paper. For most of the function we can get the result within this limit and the function that does not come in this limit is named Fail F.

Table 1: A list of test problem functions.

Problem

BZAU
NI/GE/FE/CT
TMPRP1
NI/GE/FE/CT

Extended white and holst function

31/223/191/1.4647
32/193/160/1.1246

31/223/191/1.845
32/193/160/1.4398

Extended Rosenbrock function

28/170/141/1.1573
21/110/88/0.8869

28/170/141/1.152
21/110/88/0.9232

Extended Himmelblau function

7/34/26/0.6446
10/47/36/0.6638

7/34/26/0.7544
11/51/39/0.697

7/34/26/0.9163
12/55/42/1.0252

Extended DENSCHNB function

5/22/16/0.6135
6/26/19/0.5955

6/26/16/0.68
6/26/19/0.7011

Shallow function

9/41/31/0.6188
8/39/30/0.6341

9/41/31/0.7291
8/39/30/0.764

DIXMAANA function

8/35/26/1.65
10/43/32/1.7494

8/35/26/2.1339
10/67/56/2.7067

NONDIA

8/59/50/0.6976
11/66/54/0.7215

14/84/69/1.4958
16/90/73/1.5903

DQDRTIC

5/21/15/0.6202
5/21/15/0.658

5/21/15/0.5997
5/21/15/0.6591

Extended Block Diagonal BD 1

32/37000/28750/1.9822
35/73750/64750/2.6201

32/75500/59000/2.4557
36/147000/128500/4.261

Extended Tridiagonal 1

4/18/13/0.5789
1694/5096/3401/18.6813

2072/7767/5694/130.172
F

Diagonal 4

2/9/6/0.5781
2/9/6/0.589

2/9/6/0.6138
2/9/6/0.623

Extended Powell

400/41100/31075/7.6346
92/9925/7600/2.4096

400/205500/155375/7.58
92/49625/38000/2.3136

Diagonal 2

44/135/90/1.008
48/147/98/1.2072

964/2895/1930/12.4522
968/2907/1938/12.5155

Sum Squares

57/229/171/1.4846
57/229/171/1.5077

165/661/495/3.1385
165/661/495/3.2041

SINQUAD

75/376/300/2.0587
138/653/514/3.5782

131/630/498/3.9387
594/2252/1657/11.1859

DIXMAANC

6/29/22/0.7458
7/33/25/0.7998

6/29/22/0.7667
7/33/25/0.7857

DIXMAANE

60/300/239/2.1251
434/1374/939/7.1568

244/1220/975/25.8693
F

DIXMAANG

66/330/263/2.3892
681/2073/1391/9.6909

2961/13363/10401/281.4836
F

DIXMAANI

20/90/69/0.8965
43/136/92/1.0161

287/1269/961/6.297
342/1375/1032/6.5883

DIXMAANK

21/94/72/0.87
52/163/110/0.99

684/2069/1384/9.0948
695/2904/1398/9.2842

DENSCHNA

8/40/31/0.6462
8/37/28/0.7929

8/40/31/0.8236
8/37/28/0.8122

DENSCHNB

5/22/16/0.674
6/26/19/0.997

5/22/16/0.6559
6/26/19/0.6583

DENSCHNC

7/38/30/0.7599
8/42/33/0.6927

7/38/30/0.8327
8/42/33/0.8875

ARGLINB

2/9/6/0.5714
2/9/6/0.5631

2/9/6/0.5777
2/9/6/0.6239

Power

2/9/6/0.5587
2/9/6/0.5702

66/265/198/1.7672
66/265/198/1.7369

Fletcher

22/120/97/0.964
26/143/116/1.0727

242/1333/1090/8.6331
278/1495/1216/9.7661

HIMMELBH

9/77/67/0.8398
8/74/65/1.1388

9/77/67/1.1324
8/74/65/1.7183

NONDQUAR

660/2649/1988/14.3065
739/2964/2224/16.1369

7170/28703/21532/110.70
F

Generalized quartic GQ1

11/47/35/0.706
13/55/41/0.6912

11/47/35/0.987
12/52/39/1.219

Generalized quartic GQ2

24/106/81/0.9497
26/115/88/1.0062

23/102/78/0.9412
24/107/82/0.9761

Extended Quadratic Penalty

12/61/48/0.7067
10/43/32/0.642

15/92/76/1.235
18/119/100/1.2217

ARWHEAD

8/36/27/0.6249
7/32/24/0.5877

7/39/31/0.6106
6/35/28/0.6174

Quartic

3941/11826/7884/56.888
3945/11838/7892/59.0699

5358/16077/10718/80.029
5362/16089/10726/78.05

Extrosnb

0/1/0/0.5027
0/1/0/0.5208

0/496/334/0.5188
0/1/0/0.5168

LIARWHD

11/56/44/0.7354
12/66/53/0.794

16/94/77/0.8861
20/134/113/1.3092

ENGVAL8

7/36/26/0.6582
8/37/28/0.6912

F
F

Hager

4/17/12/0.6103
4/17/12/0.5686

25/151/125/1.2086
25/102/76/0.9215

Extended Wood

91/509/417/3.0717
116/627/510/3.6229

68/423/354/2.7678
87/535/447/3.3762

BIGGSB1

1579/6316/4736/31.3271
1627/6508/4880/32.1275

2463/9852/7388/48.4366
3509/14036/10526/68.93

Extended Beale

15/71/55/0.7541
12/59/46/0.7065

15/71/55/0.8143
12/59/46/0.8638

Extended Penalty

9/43/33/0.6341
10/46/35/0.6735

37/252/214/1.7596
F

Extended Maratos Function

33/223/189/2.1234
31/213/181/1.5054

F
31/213/181/1.5017

Quadratic QF2 Function

6/26/19/0.6122
6/26/19/0.6209

293/1269/975/6.6934
277/996/768/5.1248

SINCOS

8/39/30/0.8623
12/51/38/0.9807

8/39/30/1.1289
12/51/38/1.2905

STAIRCASE S1

31/125/93/1.0555
32/129/96/1.0384

135/540/404/2.7473
197/790/592/4.0499

GENROSEN-2

25/159/133/1.3379
21/110/88/0.9879

6889/36000/29110/185.5
74/488/413/3.4692

TRIDIAGONAL DOUBLE BORDED

395/1585/1189/7.8008
540/2165/1624/11.11

773/3099/2325/15.481
1016/4073/3056/20.09

EDENSCH FUNCTION

3090/9276/6185/40.9297
3094/9288/6193/34.4141

3090/9276/6185/135.232
3094/9288/6193/131.676

DENSCHNF

10/59/48/0.9205
12/73/60/0.9032

10/59/48/0.7546
12/73/60/0.8977

Generalized Triagonal-1

23/95/71/0.9284
F

25/124/98/1.5822
24/100/75/0.9883

Generalized Triagonal-2

43/205/161/1.225
35/165/129/1.5369

39/188/148/1.2548
39/191/151/1.2668

Extended Trigonomenctric

313/965/651/5.1384
324/990/665/5.2108

97/437/339/2.6942
318/997/678/5.3339

DIAGONAL-1

5/22/16/0.6467
6/27/20/0.6174

23/95/71/0.889
25/103/77/0.9587

EG3

32/138/105/1.1712
33/141/107/1.1336

35/180/144/1.4381
F

Extended Three Exponential

8/1700/1250/0.6441
6/1300/950/0.6111

8/17000/12500/0.999
6/13000/9500/0.8019

Extended Freudenstein and Roth

16/91/74/0.7806
F

16/141/124/1.0795
F

Dixon3DQ

3049/12196/9146/49.19
3409/13636/10226/55.20

4959/19836/14876/80.41
6473/25892/19418/117.9

Quadratic Qf1

187/749/561/4.2752
187/749/561/4.3368

426/1705/1278/10.7051
426/1705/1278/11.6042

Extended Quadratic penalty Q2

46/466/419/2.9755
51/576/524/4.0679

61/740/678/6.627
86/1052/970/9.3083

Sphere

1/5/3/0.5581
1/5/3/0.549

1/5/3/0.5498
1/5/3/0.5447

Raydan1

101/304/202/1.6957
105/316/210/2.0696

66/266/199/1.5276
69/278/208/2.0739

BDQRTIC

37/179/141/1.1653
52/240/187/1.4611

F
F

DIXMAANJ

23/88/64/0.809
52/162/109/1.0504

3015/13079/10063/63.498
7074/21256/14181/93.00

DIXMAANB

5/23/17/0.5848
6/27/20/0.652

8/34/25/0.945
8/34/25/0.954

DIXMAAND

8/38/29/0.932
9/42/32/0.9416

7/34/26/1.0641
10/46/35/1.215

DIXMAANF

90/451/360/3.386
1327/3998/2670/19.805

1071/4832/3760/29.3661
3488/10489/7000/103.27

DIXMAANH

81/408/326/2.6333
1319/3974/2654/17.820

1476/6413/4936/76.60
3462/10409/6946/121.09

DIXMAANL

16/76/59/0.7726
52/164/111/1.1245

3023/13115/10091/57.84
7031/21130/14098/85.66

Raydan 2

4/17/12/0.5985
2/9/6/0.5584

5/21/15/0.6352
2/9/6/0.5593

Perturbed Quadratic

187/749/561/3.9731
187/749/561/3.7987

425/1701/1275/11.039
425/1701/1275/10.9626

A Quadratic Function

6/26/19/0.6334
6/26/19/0.6101

91/396/304/2.2252
96/417/320/2.421

TRIDIA

240/961/720/5.146
240/961/720/5.5388

349/1397/1047/7.1665
349/1397/1047/7.090

Zett1

12/52/39/0.6578
14/61/46/0.6748

Six hump

7/31/23/0.6482
8/36/27/0.6714

Booth

2/9/6/0.5722
2/9/6/0.5513

Three hump

6/28/21/0.5959
7/31/23/0.6031

EG2

4/20/15/0.6011
5/26/20/1.3894

The performance profiles are adopted by Dolan and Moré [40]. In Figures 1–4 we compare the performance based on the NI/NF/GE/CT. For every method, we plot fraction of problems for which the method is within a factor of best time. The left hand side of the figure represents the percentage of the test problem of which method is robust and the fastest; the right hand side of the figure shows the percentage of test problems that are solved successfully by either the BZAU or TMPRP1 method. In the graph there are two axes and , as there are much values of which creates difficulty in understanding the graph. The value of -axis is then converted in natural log of so it shows -axis values in exponent like , and the values of -axis are taken in linear form of . Comparing Figures 1–4 shows that the BZAU method outperforms the TMPRP1 method in every case. The top curve is the most efficient method, so the new modified three-term CG method is also efficient in terms of numerical result.

Figure 1: Performance profiles based on the number of iterations.

Figure 2: Performance profiles based on the CPU time.

Figure 3: Performance profiles based on number of gradient evaluations.

Figure 4: Performance profiles based on number of function evaluations.

6. Conclusion

In this paper, we have proposed a new modified three-term conjugate gradient method for unconstrained optimization. The new modified three-term BZAU method possesses the sufficient descent property independent of any line search. Global convergence is shown for both convex and nonconvex functions using the Wolfe line search. In numerical result we compare the three-term BZAU method with TMPRP1 method [32]. As in [32] the TMPRP1 method is shown to be numerically efficient when it comes to comparison with other two robust methods such as CG_Descent method [41] and DTPRP method [42]. That is the reason for comparing our BZAU method with TMPRP1 method.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

References

J. Nocedal, “Conjugate gradient methods and nonlinear optimization,” in Linear and Nonlinear Conjugate Gradient Related Methods, J. L. Nazareth, Ed., pp. 9–23, SIAM, Philadelphia, PA, USA, 1995.

E. Polak, Optimization: Algorithms and Consistent Approximations, Springer, New York, NY, USA, 1997.

R. Ziadi, R. Ellaia, and A. Bencherif-Madani, “Global optimization through a stochastic perturbation of the Polak-Ribière conjugate gradient method,” Journal of Computational and Applied Mathematics, vol. 317, pp. 672–684, 2017.

M. R. Hestenes and E. Stiefel, “Methods of conjugate gradients for solving linear systems,” Journal of Research of the National Bureau of Standards, vol. 49, pp. 409–436 (1953), 1952.

R. Fletcher and C. M. Reeves, “Function minimization by conjugate gradients,” The Computer Journal, vol. 7, pp. 149–154, 1964.

E. Polak and G. Ribière, “Note Sur la convergence de directions conjugeès, Rev,” Revue Française d'Informatique et de Recherche Opérationnelle, vol. 3, no. 16, pp. 35–43, 1969.

B. T. Polyak, “The conjugate gradient method in extreme problems,” USSR Computational Mathematics and Mathematical Physics, vol. 9, no. 4, pp. 94–112, 1969.

R. Fletcher, Practical Method of Optimization, Vol. I: Unconstrained Optimization, Wiley, New York, NY, USA, 2nd edition, 1997.

Y. Liu and C. Storey, “Efficient generalized conjugate gradient algorithms. I. Theory,” Journal of Optimization Theory and Applications, vol. 69, no. 1, pp. 129–137, 1991.

Y. H. Dai and Y. Yuan, “A nonlinear conjugate gradient with a strong global convergence properties,” SIAM Journal on Optimization, vol. 10, no. 1, pp. 177–182, 2000.

A. Alhawarat, M. Mamat, M. Rivaie, and Z. Salleh, “An efficient hybrid conjugate gradient method with the strong Wolfe-Powell line search,” Mathematical Problems in Engineering, vol. 2015, Article ID 103517, 7 pages, 2015.

A. Alhawarat, Z. Salleh, M. Mamat, and M. Rivaie, “An efficient modified Polak–Ribière–Polyak conjugate gradient method with global convergence properties,” Optimization Methods and Software, pp. 1–14, 2016.

A. Alhawarat and Z. Salleh, “Modification of nonlinear conjugate gradient method with weak Wolfe-Powell line search,” Abstract and Applied Analysis, Article ID 7238134, Art. ID 7238134, 6 pages, 2017.

Z. Salleh and A. Alhawarat, “An efficient modification of the Hestenes-Stiefel nonlinear conjugate gradient method with restart property,” Journal of Inequalities and Applications, vol. 2016, no. 1, article no. 110, 2016.

E. M. Beale, “A derivative of conjugate gradients,” in Numerical Methods for Nonlinear Optimization, F. A. Lootsma, Ed., pp. 39–43, Academic Press, London, 1972.

M. F. McGuire and P. Wolfe, “Evaluating a Restart Procedure for Conjugate Gradients,” Tech. Rep., IBM Research Center, Yorktown Heights, 1973.

L. Nazareth, “A conjugate direction algorithm without line searches,” Journal of Optimization Theory and Applications, vol. 23, no. 3, pp. 373–387, 1977.

N. Y. Deng and Z. Li, “Global convergence of three terms conjugate gradient methods,” Optim. Method Softw, vol. 4, pp. 273–282, 1995.

Y. H. Dai and Y. Yuan, “A nonlinear conjugate gradient method with a strong global convergence property,” SIAM Journal on Optimization, vol. 10, no. 1, pp. 177–182, 1999.

L. Zhang, W. Zhou, and D.-H. Li, “A descent modified Polak-Ribiére-Polyak conjugate gradient method and its global convergence,” IMA Journal of Numerical Analysis, vol. 26, no. 4, pp. 629–640, 2006.

L. Zhang, W. Zhou, and D. Li, “Some descent three-term conjugate gradient methods and their global convergence,” Optimization Methods and Software, vol. 22, no. 4, pp. 697–711, 2007.

W. Cheng, “A two-term PRP-based descent method,” Numerical Functional Analysis and Optimization, vol. 28, no. 11-12, pp. 1217–1230, 2007.

J. Zhang, Y. Xiao, and Z. Wei, “Nonlinear conjugate gradient methods with sufficient descent condition for large-scale unconstrained optimization,” Mathematical Problems in Engineering, Article ID 243290, Art. ID 243290, 16 pages, 2009.

A. Y. Al-Bayati and W. H. Sharif, “A new three-term conjugate gradient method for unconstrained optimization,” Canadian Journal on Science & Engineering Mathematics, vol. 1, no. 5, pp. 108–124, 2010.

Y. Narushima, H. Yabe, and J. A. Ford, “A three-term conjugate gradient method with sufficient descent property for unconstrained optimization,” SIAM Journal on Optimization, vol. 21, no. 1, pp. 212–230, 2011.

N. Andrei, “A modified Polak-Ribière-Polyak conjugate gradient algorithm for unconstrained optimization,” Optimization. A Journal of Mathematical Programming and Operations Research, vol. 60, no. 12, pp. 1457–1471, 2011.

N. Andrei, “On three-term conjugate gradient algorithms for unconstrained optimization,” Applied Mathematics and Computation, vol. 219, no. 11, pp. 6316–6327, 2013.

N. Andrei, “A simple three-term conjugate gradient algorithm for unconstrained optimization,” Journal of Computational and Applied Mathematics, vol. 241, pp. 19–29, 2013.

K. Sugiki, Y. Narushima, and H. Yabe, “Globally convergent three-term conjugate gradient methods that use secant conditions and generate descent search directions for unconstrained optimization,” Journal of Optimization Theory and Applications, vol. 153, no. 3, pp. 733–757, 2012.

M. Al-Baali, Y. Narushima, and H. Yabe, “A family of three-term conjugate gradient methods with sufficient descent property for unconstrained optimization,” Computational Optimization and Applications. An International Journal, vol. 60, no. 1, pp. 89–110, 2015.

S. Babaie-Kafaki and R. Ghanbari, “Two modified three-term conjugate gradient methods with sufficient descent property,” Optimization Letters, vol. 8, no. 8, pp. 2285–2297, 2014.

M. Sun and J. Liu, “Three modified Polak-Ribière-Polyak conjugate gradient methods with sufficient descent property,” Journal of Inequalities and Applications, vol. 2015, no. 1, 2015.

M. J. Powell, “Restart procedures for the conjugate gradient method,” Mathematical Programming, vol. 12, no. 2, pp. 241–254, 1977.

J. C. Gilbert and J. Nocedal, “Global convergence properties of conjugate gradient methods for optimization,” SIAM Journal on Optimization, vol. 2, no. 1, pp. 21–42, 1992.

Z. Wei, G. Li, and L. Qi, “New nonlinear conjugate gradient formulas for large-scale unconstrained optimization problems,” Applied Mathematics and Computation, vol. 179, no. 2, pp. 407–430, 2006.

M. J. D. Powell, “Nonconvex minimization calculations and the conjugate gradient method,” in Numerical Analysis, vol. 1066 of Lecture Notes in Mathematics, pp. 122–141, Springer, Berlin, Germany, 1984.

Z.-f. Dai and B.-S. Tian, “Global convergence of some modified PRP nonlinear conjugate gradient methods,” Optimization Letters, vol. 5, no. 4, pp. 615–630, 2011.

N. Andrei, “An unconstrained optimization test functions collection,” Advanced Modeling and Optimization, vol. 10, no. 1, pp. 147–161, 2008.

J. J. Moré, B. S. Garbow, and K. E. Hillstrom, “Testing unconstrained optimization software,” ACM Transactions on Mathematical Software, vol. 7, no. 1, pp. 17–41, 1981.

E. D. Dolan and J. J. Moré, “Benchmarking optimization software with performance profiles,” Mathematical Programming. A Publication of the Mathematical Programming Society, vol. 91, no. 2, Ser. A, pp. 201–213, 2002.

W. W. Hager and H. Zhang, “A survey of nonlinear conjugate gradient methods,” Pacific Journal of Optimization. An International Journal, vol. 2, no. 1, pp. 35–58, 2006.

Z. Dai and F. Wen, “Another improved Wei-Yao-Liu nonlinear conjugate gradient method with sufficient descent property,” Applied Mathematics and Computation, vol. 218, no. 14, pp. 7421–7430, 2012.