This article considers modified formulas for the standard conjugate gradient (CG) technique that is planned by Li and Fukushima. A new scalar parameter
Conjugate gradient (CG) strategies consists of a category of nonlinear optimization algorithms, which needs low memory and powerful local and global convergence properties [
On the understanding that the function is defined in the form
The most important component of this formula is
Here in this part of this article, we proposed a new version for the parameter
The researchers Li and Fukushima presented an appropriate modified BFGS technique which is globally and super-linearly convergent, even though while not requiring convex objective functions. The subsequent modified secant equation is outlined consistent with the subsequent formula as follows:
Specifically, take value
There are three different cases for the term Case 1: if Moreover, the form of Case 2: if
If we use
When any positive value to
Hence,
Using equation (
By substituting equations (
Hence, we conclude from equation (
If we presume that the line search satisfies conditions (
From equations (
By using Powell restart equation (i.e.,
If
Using strong Wolfe line search condition (5a) yields
This latter equation implies that
Thus, our requirement is complete.
Step 1: select the initial point Step 2: test for stopping criterion. If satisfied, then stop; otherwise, continue. Step 3: determine Step 4: compute the second iterative point Step 5: calculate the scalar parameter Step 6: calculate the new search directions, namely, Step 7: test Powell restarting criterion, namely, if Step 8: set the next iteration
In the following parts, we have a tendency to discuss the convergence analysis property for the new algorithm thoroughly. First, we offer an assumption for the convergence analysis property for the new algorithms. Then, we offer another well-known lemma needed within the study of convergence analysis property. Finally, we have a tendency to set new theorems aboard their proofs that area unit associated with the convergence analysis for the new algorithm.
(i) The level set is bounded, that is, there exists a constant In neighbourhood N of S,
From the assumptions (i) and (ii) on
If we suppose that [ Assumption holds. Search direction Optimal step The convergence condition is satisfied, i.e., if
Then,
If we suppose that Assumption holds. The new search direction The optimal step The objective function
Consider the new direction in equation (
Well parameter
Moreover, by combining the results, we obtained
We got the required proof. We put similar points to the previous hypotheses, but there are some variations in the formulas.
If we suppose that Assumption holds. The new search direction The optimal step The objective function
Using the same proof style of the previous theorem with the difference in the fact that the functions of the algorithm are general functions,
Then, we obtain
Therefore, the proof of the new theorems in regards to the convergence analyses of the proposed algorithms is complete.
In this section, we have reported some numerical experiments that are performed on a set of (60) unconstrained optimization test problems to analyse the efficiency of
In our comparisons below, we employ the following algorithms: LS: with the Wolfe line search CD: with the Wolfe line search HS: with the Wolfe line search PR: with the Wolfe line search New Algorithm, using equation ( New Algorithm, using equation (
In Tables NOI = the total number of calculated iterative iterations NOFG = the total number of function and gradient calculations TIME = the total CPU time required for the processor to execute the CG algorithm and reach the minimum value of the required function minimization
Comparisons of new algorithm against (LS) & (CD) algorithms for the total of (60) test problems with (1000
Prob. | New algorithm ( |
New algorithm ( |
LS |
CD |
---|---|---|---|---|
1 | 99/252/0.14 | 101/263/0.14 | 323/7045/2.31 | 414/9608/2.64 |
2 | 408/884/2.02 | 405/880/1.97 | 412/888/1.98 | 395/878/2.08 |
3 | 853/2183/1.06 | 830/2171/0.98 | 823/2159/1.02 | 824/2154/0.97 |
4 | 123/308/0.14 | 123/308/0.15 | 114/300/0.17 | 121/314/0.15 |
5 | 100/388/0.15 | 100/388/0.20 | 274/5586/1.14 | 167/1918/0.77 |
6 | 585/995/1.56 | 585/995/1.54 | 6576/19689/4.54 | 6848/20496/5.70 |
7 | 30/80/0.15 | 30/80/0.13 | 40/100/0.21 | 40/100/0.16 |
8 | 1032/2705/1.58 | 1020/2668/1.42 | 1033/2768/1.49 | 1026/2972/1.52 |
9 | 2388/4881/2.40 | 2503/5048/3.06 | 3520/7182/7.49 | 3676/7485/8.83 |
10 | 477/871/2.66 | 478/858/2.62 | 18795/503441/7.60 | 19346/518146/9.14 |
11 | 182/423/1.03 | 174/418/1.09 | 8669/270951/5.83 | 8252/257171/6.83 |
12 | 113/302/0.16 | 113/302/0.20 | 318/3881/1.69 | 279/2792/5.80 |
13 | 80/226/0.11 | 78/222/0.08 | 73/241/0.10 | 73/241/0.09 |
14 | 61/131/0.50 | 61/131/0.56 | 124/2140/7.12 | 160/2833/3.54 |
15 | 460/991/0.62 | 452/969/0.59 | 456/953/0.61 | 467/997/0.64 |
16 | 66/132/0.03 | 66/132/0.04 | 67/134/0.14 | 68/136/0.06 |
17 | 70/160/0.11 | 70/160/0.06 | 69/158/0.10 | 71/162/0.08 |
18 | 753/1577/0.79 | 791/1668/0.85 | 691/1477/0.74 | 782/1687/0.82 |
19 | 74/158/0.33 | 74/158/0.42 | 123/1905/2.31 | 70/150/0.30 |
20 | 110/349/0.41 | 109/349/0.37 | 114/334/0.43 | 111/329/0.41 |
21 | 806/3224/1.60 | 600/1665/1.26 | 429/1895/4.93 | 561/5284/5.69 |
22 | 72/275/0.51 | 72/285/0.49 | 84/366/0.70 | 2084/2322/9.54 |
23 | 4470/9572/1.33 | 5033/10748/2.42 | 20010/168275/4.95 | 19334/128670/5.02 |
24 | 62/201/0.42 | 62/201/0.40 | 546/15413/3.73 | 495/13595/3.48 |
25 | 459/1091/0.60 | 521/1192/0.78 | 494/1101/0.65 | 531/1154/0.76 |
26 | 56/153/0.08 | 65/373/0.13 | 56/155/0.06 | 66/328/0.12 |
27 | 85/203/0.11 | 85/203/0.12 | 80/190/0.07 | 80/190/0.09 |
28 | 534/1139/0.65 | 514/1073/0.68 | 493/1055/0.63 | 492/1075/0.61 |
29 | 540/1274/0.62 | 537/1267/0.58 | 941/2237/1.03 | 943/2239/1.11 |
30 | 576/1440/0.96 | 591/1504/1.05 | 19026/581537/2.57 | 18144/543276/2.03 |
31 | 113/236/0.16 | 113/236/0.11 | 118/245/0.11 | 123/253/0.11 |
32 | 813/2181/3.28 | 735/1997/3.03 | 18140/560188/8.67 | 18320/564064/8.04 |
33 | 98/268/0.18 | 98/268/0.17 | 359/5355/2.21 | 387/6200/3.06 |
34 | 346/766/0.48 | 348/766/0.50 | 356/786/0.44 | 332/732/0.44 |
35 | 7635/12820/8.59 | 7554/12715/8.73 | 7463/12576/8.29 | 7183/12051/8.37 |
36 | 280/978/0.43 | 280/978/0.35 | 264/935/0.45 | 273/978/0.39 |
37 | 217/534/0.31 | 217/534/0.27 | 221/551/0.31 | 219/546/0.31 |
38 | 121/287/0.14 | 120/285/0.13 | 814/20969/8.83 | 786/20117/7.46 |
39 | 150/329/0.64 | 153/328/0.65 | 141/305/0.59 | 137/297/0.58 |
40 | 107/217/0.65 | 107/217/0.68 | 144/1165/3.67 | 124/476/1.35 |
41 | 120/330/0.23 | 120/330/0.25 | 100/290/0.17 | 100/290/0.18 |
42 | 3832/9998/2.51 | 3373/8928/7.18 | 3404/9046/8.47 | 3182/8618/7.82 |
43 | 40/80/0.11 | 40/80/0.08 | 40/80/0.12 | 40/80/0.12 |
44 | 50/110/0.10 | 50/110/0.05 | 50/110/0.09 | 50/110/0.05 |
45 | 43/184/0.08 | 43/184/0.11 | 3903/129109/7.10 | 7265/242193/4.78 |
46 | 427/1323/2.56 | 427/1320/2.60 | 413/1307/2.53 | 428/1252/2.44 |
47 | 64/249/0.09 | 64/249/0.06 | 118/447/0.22 | 114/425/0.19 |
48 | 308/798/1.39 | 293/772/1.28 | 773/7349/0.72 | 1731/23050/5.88 |
49 | 22/89/0.11 | 22/89/0.13 | 316/8561/9.47 | 298/8022/8.33 |
50 | 20/50/0.03 | 20/50/0.02 | 20/50/0.00 | 20/50/0.04 |
51 | 92/1755/2.25 | 42/136/0.17 | 361/9646/9.02 | 315/8202/8.61 |
52 | 107/418/0.16 | 107/418/0.15 | 107/418/0.16 | 107/418/0.16 |
53 | 6199/52426/6.75 | 6199/52426/6.78 | 8071/53291/6.20 | 8071/53291/9.30 |
54 | 51/151/0.20 | 51/151/0.19 | 51/151/0.21 | 51/151/0.18 |
55 | 60/140/0.19 | 60/140/0.19 | 60/140/0.18 | 60/140/0.20 |
56 | 70/140/0.20 | 70/140/0.17 | 70/140/0.19 | 70/140/0.16 |
57 | 79/158/0.22 | 79/158/0.22 | 79/158/0.24 | 74/148/0.22 |
58 | 143/570/0.25 | 143/570/0.25 | 143/570/0.23 | 143/570/0.21 |
59 | 188/498/0.29 | 172/453/0.27 | 176/449/0.27 | 177/461/0.22 |
60 | 83/236/0.13 | 83/236/0.11 | 985/27409/9.27 | 917/25010/8.38 |
Total | 37602/124886/55.54 | 37426/121643/59.2 | 131933/2455353/154.98 | 136949/1242169/166.56 |
Comparisons of new algorithm against (HS) & (PR) algorithms for the total of (60) test problems with (1000
Prob. | New algorithm ( |
New algorithm ( |
HS |
PR |
---|---|---|---|---|
1 | 99/252/0.14 | 101/263/0.14 | 5902/173484/0.72 | 11798/270815/6.07 |
2 | 408/884/2.02 | 405/880/1.97 | 362/637/1.83 | 416/720/2.07 |
3 | 853/2183/1.06 | 830/2171/0.98 | 789/1817/0.86 | 989/1979/1.11 |
4 | 123/308/0.14 | 123/308/0.15 | 141/281/0.19 | 254/430/0.26 |
5 | 100/388/0.15 | 100/388/0.20 | 653/17073/3.79 | 410/7948/2.73 |
6 | 585/995/1.56 | 585/995/1.54 | 9881/16243/3.07 | 20010/22091/4.04 |
7 | 30/80/0.15 | 30/80/0.13 | 40/90/0.18 | 40/90/0.18 |
8 | 1032/2705/1.58 | 1020/2668/1.42 | 997/2279/1.28 | 8780/10109/1.57 |
9 | 2388/4881/2.40 | 2503/5048/3.06 | 4658/7644/2.48 | 14945/16013/9.92 |
10 | 477/871/2.66 | 478/858/2.62 | 20010/98744/4.70 | 20010/292528/9.39 |
11 | 182/423/1.03 | 174/418/1.09 | 14053/39805/1.28 | 15593/489487/8.58 |
12 | 113/302/0.16 | 113/302/0.20 | 428/6427/2.19 | 609/11752/4.81 |
13 | 80/226/0.11 | 78/222/0.08 | 113/234/0.08 | 291/509/0.26 |
14 | 61/131/0.50 | 61/131/0.56 | 906/23792/1.66 | 318/6032/3.76 |
15 | 460/991/0.62 | 452/969/0.59 | 636/1006/0.73 | 964/1479/1.25 |
16 | 66/132/0.03 | 66/132/0.04 | 60/120/0.03 | 1043/1116/0.38 |
17 | 70/160/0.11 | 70/160/0.06 | 207/339/0.19 | 110/230/0.10 |
18 | 753/1577/0.79 | 791/1668/0.85 | 821/1545/0.78 | 3732/4630/3.99 |
19 | 74/158/0.33 | 74/158/0.42 | 108/1352/2.44 | 303/7194/2.66 |
20 | 110/349/0.41 | 109/349/0.37 | 135/321/0.42 | 154/339/0.45 |
21 | 806/3224/1.60 | 600/1665/1.26 | 875/14122/5.89 | 929/10442/9.02 |
22 | 72/275/0.51 | 72/285/0.49 | 2104/2442/3.61 | 161/440/0.92 |
23 | 4470/9572/1.33 | 5033/10748/2.42 | 18912/38658/8.87 | 20010/25808/6.95 |
24 | 62/201/0.42 | 62/201/0.40 | 1853/6983/3.39 | 2527/76854/9.01 |
25 | 459/1091/0.60 | 521/1192/0.78 | 304/606/0.40 | 1199/1793/1.55 |
26 | 56/153/0.08 | 65/373/0.13 | 128/1103/0.37 | 2697/14700/7.09 |
27 | 85/203/0.11 | 85/203/0.12 | 91/193/0.13 | 132/264/0.16 |
28 | 534/1139/0.65 | 514/1073/0.68 | 288/558/0.35 | 556/925/0.60 |
29 | 540/1274/0.62 | 537/1267/0.58 | 852/1783/0.98 | 1014/2180/1.14 |
30 | 576/1440/0.96 | 591/1504/1.05 | 20010/98171/6.44 | 20010/317766/8.02 |
31 | 113/236/0.16 | 113/236/0.11 | 79/168/0.10 | 147/287/0.13 |
32 | 813/2181/3.28 | 735/1997/3.03 | 20010/91480/9.47 | 20010/179051/5.16 |
33 | 98/268/0.18 | 98/268/0.17 | 631/11069/5.15 | 837/17999/8.99 |
34 | 346/766/0.48 | 348/766/0.50 | 716/1148/0.90 | 744/1213/0.93 |
35 | 7635/12820/8.59 | 7554/12715/8.73 | 8375/13146/9.98 | 8539/12513/15.46 |
36 | 280/978/0.43 | 280/978/0.35 | 330/695/0.37 | 401/868/0.47 |
37 | 217/534/0.31 | 217/534/0.27 | 610/6778/2.40 | 820/11112/5.50 |
38 | 121/287/0.14 | 120/285/0.13 | 1565/42467/4.89 | 1624/45918/6.43 |
39 | 150/329/0.64 | 153/328/0.65 | 174/290/0.54 | 193/323/0.69 |
40 | 107/217/0.65 | 107/217/0.68 | 253/426/1.03 | 4281/4461/11.17 |
41 | 120/330/0.23 | 120/330/0.25 | 118/286/0.19 | 124/298/0.20 |
42 | 3832/9998/2.51 | 3373/8928/7.18 | 3685/8619/8.30 | 17419/22467/5.83 |
43 | 40/80/0.11 | 40/80/0.08 | 99/119/0.19 | 99/119/0.19 |
44 | 50/110/0.10 | 50/110/0.05 | 50/110/0.02 | 70/282/0.13 |
45 | 43/184/0.08 | 43/184/0.11 | 12047/91618/9.93 | 15601/521624/4.14 |
46 | 427/1323/2.56 | 427/1320/2.60 | 409/1040/2.07 | 597/1232/2.51 |
47 | 64/249/0.09 | 64/249/0.06 | 133/380/0.15 | 143/393/0.40 |
48 | 308/798/1.39 | 293/772/1.28 | 2549/27198/2.61 | 15662/156182/6.10 |
49 | 22/89/0.11 | 22/89/0.13 | 1377/37674/8.58 | 1571/43940/8.14 |
50 | 20/50/0.03 | 20/50/0.02 | 20/50/0.02 | 20/50/0.03 |
51 | 92/1755/2.25 | 42/136/0.17 | 1620/5439/3.43 | 1680/46332/4.04 |
52 | 107/418/0.16 | 107/418/0.15 | 125/360/0.14 | 125/360/0.16 |
53 | 6199/52426/6.75 | 6199/52426/6.78 | 4160/9534/3.23 | 2233/8070/9.66 |
54 | 51/151/0.20 | 51/151/0.19 | 75/173/0.25 | 77/144/0.20 |
55 | 60/140/0.19 | 60/140/0.19 | 60/120/0.14 | 60/120/0.17 |
56 | 70/140/0.20 | 70/140/0.17 | 70/140/0.24 | 80/160/0.22 |
57 | 79/158/0.22 | 79/158/0.22 | 79/158/0.24 | 86/172/0.23 |
58 | 143/570/0.25 | 143/570/0.25 | 177/506/0.25 | 177/506/0.25 |
59 | 188/498/0.29 | 172/453/0.27 | 209/460/0.30 | 641/917/1.01 |
60 | 83/236/0.13 | 83/236/0.11 | 1615/43549/5.17 | 2005/57225/7.75 |
Total | 37602/124896/60.1 | 37426/121943/70.14 | 167737/10741186/139.61 | 271227/7136001/214.33 |
Therefore, among these CG algorithms, the new algorithm appears to generate the best search direction. In Table For 100% LS algorithm: the new algorithm is improved by (71.5%) NOI, improved by (94.92%) NOFG, and improved by (64.2%) time For 100% CD algorithm: the new algorithm is improved by (72.6%) NOI, improved by (89.95%) NOFG, and improved by (66.7%) time
Standardizing perceptual of the new algorithm vs. LS and CD algorithms.
Algorithm | Tools | LS (1991) (%) | CD (1987) (%) |
---|---|---|---|
When |
NOI | 28.5 | 27.4 |
NOFG | 5.08 | 10.05 | |
TIME | 35.8 | 33.3 | |
|
|||
When |
NOI | 28.3 | 27.3 |
NOFG | 4.9 | 9.7 | |
TIME | 38.1 | 35.5 |
And (when For 100% LS algorithm: the new algorithm is improved by (64.34%) NOI, improved by (16.99%) NOFG, and improved by (16.08%) time For 100% CD algorithm: the new algorithm is improved by (52.60%) NOI, improved by (14.63%) NOFG, and improved by (12.75%) time In Table For 100% HS algorithm: the new algorithm is improved by (77.6%) NOI, improved by (98.9%) NOFG, and improved by (57%) time For 100% PR algorithm: the new algorithm is improved by (86.2%) NOI, improved by (98.3%) NOFG, and improved by (72%) time and (when For 100% HS algorithm: the new algorithm is improved by (77.7%) NOI, improved by (98.9%) NOFG, and improved by (49.8%) time For 100% PR algorithm: the new algorithm is improved by (86.3%) NOI, improved by (98.3%) NOFG, and improved by (67.3%) time
Standardizing perceptual of the new algorithm vs. HS and PR algorithms.
Algorithm | Tools | HS (1952) (%) | PR (1969) (%) |
---|---|---|---|
When |
NOI | 22.4 | 13.8 |
NOFG | 1.1 | 1.7 | |
TIME | 43 | 28 | |
|
|||
When |
NOI | 22.3 | 13.7 |
NOFG | 1.1 | 1.7 | |
TIME | 50.2 | 32.7 |
What can be deduced from the above tables and experiments are summarized in the following: Points (a to d) above are that our new proposed algorithms in the field of CG-type methods are economic and robust as compared to the standard LS and CD algorithms The abovementioned points (e to h) are that our new proposed algorithms in the field of CG-type methods are economic and robust as compared to the standard HS and PR algorithms
All these comparisons were made using the performance profile of Dolan and Moré [ Figure Figure Figure Figure Figure
Performance profiles based on number of iterations (NOI). (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
Performance profiles based on function and gradient evaluations (NOFG). (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
Performance profiles based on CPU Time. (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
Performance profiles based on the best NOI. (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
Performance profiles based on the best NOFG. (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
In this study, we have submitted two proposed new CG methods (by changing the value of
The data used the support the findings of this study are available from the corresponding author upon request.
The authors declare that there are no conflicts of interest regarding the publication of this paper.
The research was supported by College of Computer Sciences and Mathematics, University of Mosul, Republic of Iraq, under Project no. 3615208.