This article considers modified formulas for the standard conjugate gradient (CG) technique that is planned by Li and Fukushima. A new scalar parameter
Conjugate gradient (CG) strategies consists of a category of nonlinear optimization algorithms, which needs low memory and powerful local and global convergence properties [
On the understanding that the function is defined in the form
The most important component of this formula is
Here in this part of this article, we proposed a new version for the parameter
The researchers Li and Fukushima presented an appropriate modified BFGS technique which is globally and superlinearly convergent, even though while not requiring convex objective functions. The subsequent modified secant equation is outlined consistent with the subsequent formula as follows:
Specifically, take value
There are three different cases for the term
Case 1: if
Moreover, the form of
Case 2: if
If we use
When any positive value to
Hence,
Using equation (
By substituting equations (
Hence, we conclude from equation (
If we presume that the line search satisfies conditions (
From equations (
By using Powell restart equation (i.e.,
If
Using strong Wolfe line search condition (5a) yields
This latter equation implies that
Thus, our requirement is complete.
Step 1: select the initial point
Step 2: test for stopping criterion. If satisfied, then stop; otherwise, continue.
Step 3: determine
Step 4: compute the second iterative point
Step 5: calculate the scalar parameter
Step 6: calculate the new search directions, namely,
Step 7: test Powell restarting criterion, namely, if
Step 8: set the next iteration
In the following parts, we have a tendency to discuss the convergence analysis property for the new algorithm thoroughly. First, we offer an assumption for the convergence analysis property for the new algorithms. Then, we offer another wellknown lemma needed within the study of convergence analysis property. Finally, we have a tendency to set new theorems aboard their proofs that area unit associated with the convergence analysis for the new algorithm.
(i) The level set
is bounded, that is, there exists a constant
In neighbourhood N of S,
From the assumptions (i) and (ii) on
If we suppose that [
Assumption holds.
Search direction
Optimal step
The convergence condition is satisfied, i.e., if
Then,
If we suppose that
Assumption holds.
The new search direction
The optimal step
The objective function
Consider the new direction in equation (
Well parameter
Moreover, by combining the results, we obtained
We got the required proof. We put similar points to the previous hypotheses, but there are some variations in the formulas.
If we suppose that
Assumption holds.
The new search direction
The optimal step
The objective function
Using the same proof style of the previous theorem with the difference in the fact that the functions of the algorithm are general functions,
Then, we obtain
Therefore, the proof of the new theorems in regards to the convergence analyses of the proposed algorithms is complete.
In this section, we have reported some numerical experiments that are performed on a set of (60) unconstrained optimization test problems to analyse the efficiency of
In our comparisons below, we employ the following algorithms:
LS: with the Wolfe line search
CD: with the Wolfe line search
HS: with the Wolfe line search
PR: with the Wolfe line search
New Algorithm, using equation (
New Algorithm, using equation (
In Tables
NOI = the total number of calculated iterative iterations
NOFG = the total number of function and gradient calculations
TIME = the total CPU time required for the processor to execute the CG algorithm and reach the minimum value of the required function minimization
Comparisons of new algorithm against (LS) & (CD) algorithms for the total of (60) test problems with (1000
Prob.  New algorithm ( 
New algorithm ( 
LS 
CD 

1  99/252/0.14  101/263/0.14  323/7045/2.31  414/9608/2.64 
2  408/884/2.02  405/880/1.97  412/888/1.98  395/878/2.08 
3  853/2183/1.06  830/2171/0.98  823/2159/1.02  824/2154/0.97 
4  123/308/0.14  123/308/0.15  114/300/0.17  121/314/0.15 
5  100/388/0.15  100/388/0.20  274/5586/1.14  167/1918/0.77 
6  585/995/1.56  585/995/1.54  6576/19689/4.54  6848/20496/5.70 
7  30/80/0.15  30/80/0.13  40/100/0.21  40/100/0.16 
8  1032/2705/1.58  1020/2668/1.42  1033/2768/1.49  1026/2972/1.52 
9  2388/4881/2.40  2503/5048/3.06  3520/7182/7.49  3676/7485/8.83 
10  477/871/2.66  478/858/2.62  18795/503441/7.60  19346/518146/9.14 
11  182/423/1.03  174/418/1.09  8669/270951/5.83  8252/257171/6.83 
12  113/302/0.16  113/302/0.20  318/3881/1.69  279/2792/5.80 
13  80/226/0.11  78/222/0.08  73/241/0.10  73/241/0.09 
14  61/131/0.50  61/131/0.56  124/2140/7.12  160/2833/3.54 
15  460/991/0.62  452/969/0.59  456/953/0.61  467/997/0.64 
16  66/132/0.03  66/132/0.04  67/134/0.14  68/136/0.06 
17  70/160/0.11  70/160/0.06  69/158/0.10  71/162/0.08 
18  753/1577/0.79  791/1668/0.85  691/1477/0.74  782/1687/0.82 
19  74/158/0.33  74/158/0.42  123/1905/2.31  70/150/0.30 
20  110/349/0.41  109/349/0.37  114/334/0.43  111/329/0.41 
21  806/3224/1.60  600/1665/1.26  429/1895/4.93  561/5284/5.69 
22  72/275/0.51  72/285/0.49  84/366/0.70  2084/2322/9.54 
23  4470/9572/1.33  5033/10748/2.42  20010/168275/4.95  19334/128670/5.02 
24  62/201/0.42  62/201/0.40  546/15413/3.73  495/13595/3.48 
25  459/1091/0.60  521/1192/0.78  494/1101/0.65  531/1154/0.76 
26  56/153/0.08  65/373/0.13  56/155/0.06  66/328/0.12 
27  85/203/0.11  85/203/0.12  80/190/0.07  80/190/0.09 
28  534/1139/0.65  514/1073/0.68  493/1055/0.63  492/1075/0.61 
29  540/1274/0.62  537/1267/0.58  941/2237/1.03  943/2239/1.11 
30  576/1440/0.96  591/1504/1.05  19026/581537/2.57  18144/543276/2.03 
31  113/236/0.16  113/236/0.11  118/245/0.11  123/253/0.11 
32  813/2181/3.28  735/1997/3.03  18140/560188/8.67  18320/564064/8.04 
33  98/268/0.18  98/268/0.17  359/5355/2.21  387/6200/3.06 
34  346/766/0.48  348/766/0.50  356/786/0.44  332/732/0.44 
35  7635/12820/8.59  7554/12715/8.73  7463/12576/8.29  7183/12051/8.37 
36  280/978/0.43  280/978/0.35  264/935/0.45  273/978/0.39 
37  217/534/0.31  217/534/0.27  221/551/0.31  219/546/0.31 
38  121/287/0.14  120/285/0.13  814/20969/8.83  786/20117/7.46 
39  150/329/0.64  153/328/0.65  141/305/0.59  137/297/0.58 
40  107/217/0.65  107/217/0.68  144/1165/3.67  124/476/1.35 
41  120/330/0.23  120/330/0.25  100/290/0.17  100/290/0.18 
42  3832/9998/2.51  3373/8928/7.18  3404/9046/8.47  3182/8618/7.82 
43  40/80/0.11  40/80/0.08  40/80/0.12  40/80/0.12 
44  50/110/0.10  50/110/0.05  50/110/0.09  50/110/0.05 
45  43/184/0.08  43/184/0.11  3903/129109/7.10  7265/242193/4.78 
46  427/1323/2.56  427/1320/2.60  413/1307/2.53  428/1252/2.44 
47  64/249/0.09  64/249/0.06  118/447/0.22  114/425/0.19 
48  308/798/1.39  293/772/1.28  773/7349/0.72  1731/23050/5.88 
49  22/89/0.11  22/89/0.13  316/8561/9.47  298/8022/8.33 
50  20/50/0.03  20/50/0.02  20/50/0.00  20/50/0.04 
51  92/1755/2.25  42/136/0.17  361/9646/9.02  315/8202/8.61 
52  107/418/0.16  107/418/0.15  107/418/0.16  107/418/0.16 
53  6199/52426/6.75  6199/52426/6.78  8071/53291/6.20  8071/53291/9.30 
54  51/151/0.20  51/151/0.19  51/151/0.21  51/151/0.18 
55  60/140/0.19  60/140/0.19  60/140/0.18  60/140/0.20 
56  70/140/0.20  70/140/0.17  70/140/0.19  70/140/0.16 
57  79/158/0.22  79/158/0.22  79/158/0.24  74/148/0.22 
58  143/570/0.25  143/570/0.25  143/570/0.23  143/570/0.21 
59  188/498/0.29  172/453/0.27  176/449/0.27  177/461/0.22 
60  83/236/0.13  83/236/0.11  985/27409/9.27  917/25010/8.38 
Total  37602/124886/55.54  37426/121643/59.2  131933/2455353/154.98  136949/1242169/166.56 
Comparisons of new algorithm against (HS) & (PR) algorithms for the total of (60) test problems with (1000
Prob.  New algorithm ( 
New algorithm ( 
HS 
PR 

1  99/252/0.14  101/263/0.14  5902/173484/0.72  11798/270815/6.07 
2  408/884/2.02  405/880/1.97  362/637/1.83  416/720/2.07 
3  853/2183/1.06  830/2171/0.98  789/1817/0.86  989/1979/1.11 
4  123/308/0.14  123/308/0.15  141/281/0.19  254/430/0.26 
5  100/388/0.15  100/388/0.20  653/17073/3.79  410/7948/2.73 
6  585/995/1.56  585/995/1.54  9881/16243/3.07  20010/22091/4.04 
7  30/80/0.15  30/80/0.13  40/90/0.18  40/90/0.18 
8  1032/2705/1.58  1020/2668/1.42  997/2279/1.28  8780/10109/1.57 
9  2388/4881/2.40  2503/5048/3.06  4658/7644/2.48  14945/16013/9.92 
10  477/871/2.66  478/858/2.62  20010/98744/4.70  20010/292528/9.39 
11  182/423/1.03  174/418/1.09  14053/39805/1.28  15593/489487/8.58 
12  113/302/0.16  113/302/0.20  428/6427/2.19  609/11752/4.81 
13  80/226/0.11  78/222/0.08  113/234/0.08  291/509/0.26 
14  61/131/0.50  61/131/0.56  906/23792/1.66  318/6032/3.76 
15  460/991/0.62  452/969/0.59  636/1006/0.73  964/1479/1.25 
16  66/132/0.03  66/132/0.04  60/120/0.03  1043/1116/0.38 
17  70/160/0.11  70/160/0.06  207/339/0.19  110/230/0.10 
18  753/1577/0.79  791/1668/0.85  821/1545/0.78  3732/4630/3.99 
19  74/158/0.33  74/158/0.42  108/1352/2.44  303/7194/2.66 
20  110/349/0.41  109/349/0.37  135/321/0.42  154/339/0.45 
21  806/3224/1.60  600/1665/1.26  875/14122/5.89  929/10442/9.02 
22  72/275/0.51  72/285/0.49  2104/2442/3.61  161/440/0.92 
23  4470/9572/1.33  5033/10748/2.42  18912/38658/8.87  20010/25808/6.95 
24  62/201/0.42  62/201/0.40  1853/6983/3.39  2527/76854/9.01 
25  459/1091/0.60  521/1192/0.78  304/606/0.40  1199/1793/1.55 
26  56/153/0.08  65/373/0.13  128/1103/0.37  2697/14700/7.09 
27  85/203/0.11  85/203/0.12  91/193/0.13  132/264/0.16 
28  534/1139/0.65  514/1073/0.68  288/558/0.35  556/925/0.60 
29  540/1274/0.62  537/1267/0.58  852/1783/0.98  1014/2180/1.14 
30  576/1440/0.96  591/1504/1.05  20010/98171/6.44  20010/317766/8.02 
31  113/236/0.16  113/236/0.11  79/168/0.10  147/287/0.13 
32  813/2181/3.28  735/1997/3.03  20010/91480/9.47  20010/179051/5.16 
33  98/268/0.18  98/268/0.17  631/11069/5.15  837/17999/8.99 
34  346/766/0.48  348/766/0.50  716/1148/0.90  744/1213/0.93 
35  7635/12820/8.59  7554/12715/8.73  8375/13146/9.98  8539/12513/15.46 
36  280/978/0.43  280/978/0.35  330/695/0.37  401/868/0.47 
37  217/534/0.31  217/534/0.27  610/6778/2.40  820/11112/5.50 
38  121/287/0.14  120/285/0.13  1565/42467/4.89  1624/45918/6.43 
39  150/329/0.64  153/328/0.65  174/290/0.54  193/323/0.69 
40  107/217/0.65  107/217/0.68  253/426/1.03  4281/4461/11.17 
41  120/330/0.23  120/330/0.25  118/286/0.19  124/298/0.20 
42  3832/9998/2.51  3373/8928/7.18  3685/8619/8.30  17419/22467/5.83 
43  40/80/0.11  40/80/0.08  99/119/0.19  99/119/0.19 
44  50/110/0.10  50/110/0.05  50/110/0.02  70/282/0.13 
45  43/184/0.08  43/184/0.11  12047/91618/9.93  15601/521624/4.14 
46  427/1323/2.56  427/1320/2.60  409/1040/2.07  597/1232/2.51 
47  64/249/0.09  64/249/0.06  133/380/0.15  143/393/0.40 
48  308/798/1.39  293/772/1.28  2549/27198/2.61  15662/156182/6.10 
49  22/89/0.11  22/89/0.13  1377/37674/8.58  1571/43940/8.14 
50  20/50/0.03  20/50/0.02  20/50/0.02  20/50/0.03 
51  92/1755/2.25  42/136/0.17  1620/5439/3.43  1680/46332/4.04 
52  107/418/0.16  107/418/0.15  125/360/0.14  125/360/0.16 
53  6199/52426/6.75  6199/52426/6.78  4160/9534/3.23  2233/8070/9.66 
54  51/151/0.20  51/151/0.19  75/173/0.25  77/144/0.20 
55  60/140/0.19  60/140/0.19  60/120/0.14  60/120/0.17 
56  70/140/0.20  70/140/0.17  70/140/0.24  80/160/0.22 
57  79/158/0.22  79/158/0.22  79/158/0.24  86/172/0.23 
58  143/570/0.25  143/570/0.25  177/506/0.25  177/506/0.25 
59  188/498/0.29  172/453/0.27  209/460/0.30  641/917/1.01 
60  83/236/0.13  83/236/0.11  1615/43549/5.17  2005/57225/7.75 
Total  37602/124896/60.1  37426/121943/70.14  167737/10741186/139.61  271227/7136001/214.33 
Therefore, among these CG algorithms, the new algorithm appears to generate the best search direction. In Table
For 100% LS algorithm: the new algorithm is improved by (71.5%) NOI, improved by (94.92%) NOFG, and improved by (64.2%) time
For 100% CD algorithm: the new algorithm is improved by (72.6%) NOI, improved by (89.95%) NOFG, and improved by (66.7%) time
Standardizing perceptual of the new algorithm vs. LS and CD algorithms.
Algorithm  Tools  LS (1991) (%)  CD (1987) (%) 

When 
NOI  28.5  27.4 
NOFG  5.08  10.05  
TIME  35.8  33.3  


When 
NOI  28.3  27.3 
NOFG  4.9  9.7  
TIME  38.1  35.5 
And (when
For 100% LS algorithm: the new algorithm is improved by (64.34%) NOI, improved by (16.99%) NOFG, and improved by (16.08%) time
For 100% CD algorithm: the new algorithm is improved by (52.60%) NOI, improved by (14.63%) NOFG, and improved by (12.75%) time
In Table
For 100% HS algorithm: the new algorithm is improved by (77.6%) NOI, improved by (98.9%) NOFG, and improved by (57%) time
For 100% PR algorithm: the new algorithm is improved by (86.2%) NOI, improved by (98.3%) NOFG, and improved by (72%) time
and (when
For 100% HS algorithm: the new algorithm is improved by (77.7%) NOI, improved by (98.9%) NOFG, and improved by (49.8%) time
For 100% PR algorithm: the new algorithm is improved by (86.3%) NOI, improved by (98.3%) NOFG, and improved by (67.3%) time
Standardizing perceptual of the new algorithm vs. HS and PR algorithms.
Algorithm  Tools  HS (1952) (%)  PR (1969) (%) 

When 
NOI  22.4  13.8 
NOFG  1.1  1.7  
TIME  43  28  


When 
NOI  22.3  13.7 
NOFG  1.1  1.7  
TIME  50.2  32.7 
What can be deduced from the above tables and experiments are summarized in the following:
Points (a to d) above are that our new proposed algorithms in the field of CGtype methods are economic and robust as compared to the standard LS and CD algorithms
The abovementioned points (e to h) are that our new proposed algorithms in the field of CGtype methods are economic and robust as compared to the standard HS and PR algorithms
All these comparisons were made using the performance profile of Dolan and Moré [
Figure
Figure
Figure
Figure
Figure
Performance profiles based on number of iterations (NOI). (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
Performance profiles based on function and gradient evaluations (NOFG). (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
Performance profiles based on CPU Time. (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
Performance profiles based on the best NOI. (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
Performance profiles based on the best NOFG. (a) Comparison of the new algorithm vs. LS and CD. (b) Comparison of the new algorithm vs. HS and PR.
In this study, we have submitted two proposed new CG methods (by changing the value of
The data used the support the findings of this study are available from the corresponding author upon request.
The authors declare that there are no conflicts of interest regarding the publication of this paper.
The research was supported by College of Computer Sciences and Mathematics, University of Mosul, Republic of Iraq, under Project no. 3615208.