Mathematical Problems in Engineering
Volume 2020 (2020), Article ID 4381515, 14 pages
https://doi.org/10.1155/2020/4381515

Research Article

A Modified Three-Term Type CD Conjugate Gradient Algorithm for Unconstrained Optimization Problems

Zhan Wang,¹ Pengyuan Li,¹ Xiangrong Li,² and Hongtruong Pham³

¹College of Mathematics and Information Science, Guangxi University, Nanning, Guangxi, China
²Business School, College of Mathematics and Information Science, Guangxi University, Nanning, Guangxi, China
³Thai Nguyen University of Economics and Business Administration, Thai Nguyen, Vietnam

Correspondence should be addressed to Xiangrong Li; xrli68@163.com

Received 26 July 2020; Accepted 13 August 2020; Published 4 September 2020

Guest Editor: Maojun Zhang

Copyright © 2020 Zhan Wang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Conjugate gradient methods are well-known methods which are widely applied in many practical fields. CD conjugate gradient method is one of the classical types. In this paper, a modified three-term type CD conjugate gradient algorithm is proposed. Some good features are presented as follows: (i) A modified three-term type CD conjugate gradient formula is presented. (ii) The given algorithm possesses sufficient descent property and trust region property. (iii) The algorithm has global convergence with the modified weak Wolfe–Powell (MWWP) line search technique and projection technique for general function. The new algorithm has made great progress in numerical experiments. It shows that the modified three-term type CD conjugate gradient method is more competitive than the classical CD conjugate gradient method.

1. Introduction

Considering the problem

where is a continuously differentiable function. This kind of model is often used to solve some problems in applied mathematics, economics, engineering, and so on. Generally, the following iteration formula is used to generate the next iteration point:

where and denote the next iteration point and current iteration point, respectively. is a step-length and is the search direction. The search direction generated by Conjugated Gradient (CG) method is defined by the following formula:

where is the gradient of at and is a parameter. Different will generate different CG methods [1–10]. There are six classical forms of :

where , and is the Euclidean norm. Those formulas can be divided into two categories: One includes PRP method, HS method, and LS method, which have good numerical performance; the other includes FR method, CD method, and DY method, which have good theory convergence. About these methods, many scholars had applied them to solve nonlinear monotone equations and normal optimization problems, and some good results were achieved [11–16].

Zhang et al. [17] presented a modified PRP CG formula as follows:

They proved that the modified PRP method is globally convergent with Armijo-type line search.

Yuan et al. [18] proposed another modified PRP CG formula with

where and . Yuan et al. [18] obtained the global convergence of the modified PRP method with a modified weak Wolfe–Powell (MWWP) line search technique, which was proposed by Yuan et al. [19]:

where , , and . Yuan et al. [16] obtained the global convergence of PRP method by using the above modified weak Wolfe–Powell line search technique and a projection technique:

where is the next point and parameter . With this projection technique, any unsatisfactory iteration points generated by the normal PRP algorithm will be projected onto a surface to overcome the failure to converge.

Motivated by the above researches, a modified three-term CD conjugate gradient algorithm is presented with (7), (8), and (9) in this paper. Some good properties are obtained as follows:

(i)A modified three-term type CD conjugate gradient formula is presented(ii)The given algorithm possesses sufficient descent property and trust region property(iii)The algorithm has global convergence with the MWWP line search technique and projection technique for general function

This paper is organized as follows: the next section will introduce the modified CD formula and relative algorithm; section 3 gives the proof of the global convergence of the new algorithm; numerical experiments are given in section 4; some conclusions are presented in section 5. Throughout this paper, we use to denote the Euclidean norm, and are replaced by and , respectively.

2. Motivation and Algorithm

The convergence of CD conjugate gradient method has been proved [4]; however, the numerical results of this method are worse than the PRP method and others. Therefore, it is necessary to propose a new search direction to improve the numerical performance of the CD method. Meanwhile, the sufficient descent property is significant for obtaining the convergence of the conjugate gradient method:

Then, we also hope that we can propose a new method that possesses this property. Inspired by the above discussion, a modified three-term type CD conjugate gradient formula is designed by the following:

where , and . The steps of the given algorithm are listed as follows.

3. Convergence Analysis

In this section, we are going to analyse the convergence of the proposed algorithm. The following assumptions are needed.

Assumption 1.

(i)The level set is bounded.(ii) is twice continuously differentiable and bounded below, and the gradient function is Lipschitz continuous, which means that there exists a constant such that

Lemma 1. Let the search direction be generated by formula (11), then, the following relations hold:

where constants .

Proof. According to the definition of ,

For the parameter , we have the following:

then holds. Letting , the relation (13) is obtained.
Using the definition of the parameter , we analyse the value of in two cases:

Case 1. . Similar to Lemma 1 of [20], , where is a scalar; then

Case 2. . Then

Letting , we have . Thus,

Let ; then (14) holds. The proof is complete.

Remark 1. The relation (14) shows that the optimization algorithm possesses the trust region feature.
The following theorem is obtained for the global convergence of Algorithm 1.

Step 1: Given the initial point , constants , , , let , .

 Step 2: If , then, stop; otherwise, proceed to the next step.

 Step 3: Compute step size by the line search (7) and (8).

 Step 4: Let .

 Step 5: If holds, let , , and go to Step 7; otherwise, go to Step 6.

 Step 6: Let be defined by (9), , and .

 Step 7: If , stop; otherwise, proceed to next step.

 Step 8: Calculate the search direction by (11).

 Step 9: Let , and go to step 2.

Algorithm 1: The modified three-term type CD conjugate gradient algorithm.

Theorem 1. Assume that and are generated by Algorithm 1 and Lemma 1 holds. Then,

Proof. From the line search (8), we have the following:

According to (ii) of Assumption 1,

Then, we have the following:

Using Lemma 1,

From Assumption 1, line search (7) and sufficient descent property (13),

Summing these inequalities from to ,

From Assumption 1, it is easy to know that is bounded. Then, . The proof is complete.

4. Numerical Results

This section will report the numerical experiments with some classical optimization problems, the nonlinear Muskingum model, and the application in image restoration problems. All the tests are coded in MATLAB R2014a, run on a PC with a 2.50 GHz CPU, and 4.00 GB of memory running the Windows 10 operating system.

4.1. Normal Unconstrained Optimization Problems

In this subsection, the numerical experiments would be done with some test problems from [20], and all test problems are listed in Table 1. We compare Algorithm 1 with the classical CD conjugated method (called Algorithm 2) and the classical PRP conjugated method (called Algorithm 3). The detailed experimental data are list in Table 2. Figures 1–3 show the performance of these three algorithms related to CPU, NI, and NFG. Some columns of Tables 1 and 2 and Figures 1–3 have the following meanings:

(i)No: the serial number of the problem.(ii)Dim: the dimension of the variable .(iii)NI: the iteration numbers.(iv)NFG: the sum of function value and gradient value.(v)CPU: the calculation time in seconds.(vi)Dimension: the dimensions are 3000, 9000, and 15000.(vii)Initialization: the parameters of the algorithms are chosen to be , , and , and the initial search direction .(viii)Stop rules: the following Himmeblau stop rule [21] is used: if , let ; otherwise, let . For every problem, if the conditions or are satisfied, then the program is stopped. This program is also stopped when the number of iterations is greater than one thousand.

Table 1: Test problems.

No
Test problems

1
Extended freudenstein and roth function

2
Extended trigonometric function

3
Extended rosenbrock function

4
Extended white and holst function

5
Extended beale function

6
Extended penalty function

7
Perturbed quadratic function

8
Raydan 1 function

9
Raydan 2 function

10
Diagonal 1 function

11
Diagonal 2 function

12
Diagonal 3 function

13
Hager function

14
Generalized tridiagonal 1 function

15
Extended tridiagonal 1 function

16
Extended three exponential terms function

17
Generalized tridiagonal 2 function

18
Diagonal 4 function

19
Diagonal 5 function

20
Extended himmelblau function

21
Generalized PSC1 function

22
Extended PSC1 function

23
Extended powell function

24
Extended block diagonal BD1 function

25
Extended maratos function

26
Extended cliff function

27
Quadratic diagonal perturbed function

28
Extended wood function

29
Extended hiebert function

30
Quadratic function QF1 function

31
Extended quadratic penalty QP1 function

32
Extended quadratic penalty QP2 function

33
A quadratic function QF2 function

34
Extended EP1 function

35
Extended Tridiagonal-2 function

36
BDQRTIC function (CUTE)

37
TRIDIA function (CUTE)

38
ARWHEAD function (CUTE)

39
ARWHEAD function (CUTE)

40
NONDQUAR function (CUTE)

41
DQDRTIC function (CUTE)

42
EG2 function (CUTE)

43
DIXMAANA function (CUTE)

44
DIXMAANB function (CUTE)

45
DIXMAANC function (CUTE)

46
DIXMAANE function (CUTE)

47
Partial perturbed quadratic function

48
Broyden tridiagonal function

49
Almost perturbed quadratic function

50
Tridiagonal perturbed quadratic function

51
EDENSCH function (CUTE)

52
VARDIM function (CUTE)

53
STAIRCASE S1 function

54
LIARWHD function (CUTE)

55
DIAGONAL 6 function

56
DIXON3DQ function (CUTE)

57
DIXMAANF function (CUTE)

58
DIXMAANG function (CUTE)

59
DIXMAANH function (CUTE)

60
DIXMAANI function (CUTE)

61
DIXMAANJ function (CUTE)

62
DIXMAANK function (CUTE)

63
DIXMAANL function (CUTE)

64
DIXMAAND function (CUTE)

65
ENGVAL1 function (CUTE)

66
FLETCHCR function (CUTE)

67
COSINE function (CUTE)

68
Extended DENSCHNB function (CUTE)

69
DENSCHNF function (CUTE)

70
SINQUAD function (CUTE)

71
BIGGSB1 function (CUTE)

72
Partial perturbed quadratic PPQ2 function

73
Scaled quadratic SQ1 function

Table 2: The numerical results of Algorithms 1–3.

No.
Dim
Algorithm 1
Algorithm 2
Algorithm 3

NI
NFG
CPU time
NI
NFG
CPU time
NI
NFG
CPU time

1
3000
7
41
0.484375
20
73
0.078125
16
65
0.109375

1
9000
13
89
0.328125
20
70
0.265625
16
62
0.28125

1
15000
7
41
0.203125
42
144
0.8125
20
85
0.5625

2
3000
169
1138
1.46875
83
260
0.328125
89
361
0.484375

2
9000
108
396
1.34375
91
286
0.875
89
298
0.890625

2
15000
125
590
2.765625
93
294
1.328125
95
306
1.453125

3
3000
9
49
0.03125
109
415
0.203125
32
106
0.0625

3
9000
5
26
0.078125
91
348
0.265625
6
29
0.015625

3
15000
5
26
0.015625
96
371
0.4375
9
50
0.078125

4
3000
43
200
0.203125
167
572
0.515625
29
104
0.109375

4
9000
21
91
0.203125
51
202
0.515625
29
109
0.21875

4
15000
21
91
0.359375
45
176
0.625
43
164
0.625

5
3000
2
14
0.015625
37
130
0.109375
88
367
0.6875

5
9000
41
128
0.296875
36
125
0.28125
29
109
0.71875

5
15000
3
26
0.046875
38
131
0.421875
53
171
0.734375

6
3000
92
327
0.171875
113
358
0.15625
124
391
0.3125

6
9000
104
340
0.390625
133
420
0.40625
144
453
0.5625

6
15000
116
370
0.65625
151
474
0.640625
145
466
0.765625

7
3000
423
4746
2.40625
968
3023
1.203125
1000
3041
1.265625

7
9000
102
373
0.359375
1000
3075
2.453125
1000
3062
3.09375

7
15000
272
987
1.578125
214
718
0.921875
1000
3046
4.296875

8
3000
18
65
0.046875
23
74
0.03125
25
79
0.046875

8
9000
17
62
0.078125
23
74
0.09375
25
79
0.109375

8
15000
17
62
0.15625
23
74
0.125
25
79
0.171875

9
3000
19
65
0.03125
9
28
0.015625
3
25
0.03125

9
9000
3
26
0.046875
9
28
0.046875
23
100
0.09375

9
15000
3
26
0.046875
9
28
0.03125
24
110
0.234375

10
3000
2
14
0.015625
2
14
0.015625
2
14
0

10
9000
2
14
0
2
14
0
2
14
0

10
15000
2
14
0.03125
2
14
0.015625
2
14
0.03125

11
3000
3
19
0.03125
75
263
0.140625
4
21
0

11
9000
3
19
0.03125
175
593
0.828125
6
36
0.0625

11
15000
3
19
0.140625
22
88
0.140625
7
37
0.078125

12
3000
17
57
0.03125
15
46
0.0625
17
55
0.046875

12
9000
16
54
0.125
16
49
0.09375
17
56
0.1875

12
15000
16
54
0.125
16
49
0.109375
17
56
0.15625

13
3000
2
14
0
23
138
0.078125
2
14
0.03125

13
9000
2
14
0.03125
3
18
0.046875
2
14
0

13
15000
2
14
0.015625
2
14
0.015625
2
14
0.0625

14
3000
18
85
0.5625
8
27
0.15625
37
169
1.0625

14
9000
7
39
0.796875
4
14
0.21875
45
274
4.890625

14
15000
7
41
1.296875
4
14
0.359375
50
343
9.75

15
3000
17
59
0.484375
52
177
1.046875
33
175
0.796875

15
9000
25
83
1.328125
50
169
2.65625
33
175
2.359375

15
15000
29
95
2.5625
41
138
3.578125
33
175
3.796875

16
3000
3
16
0
13
40
0.03125
40
151
0.109375

16
9000
8
23
0.046875
17
52
0.0625
24
73
0.09375

16
15000
17
59
0.15625
15
46
0.09375
27
82
0.171875

17
3000
2
14
0.015625
40
122
0.1875
53
164
0.234375

17
9000
36
148
0.578125
35
107
0.375
47
149
0.546875

17
15000
5
42
0.15625
27
83
0.546875
49
155
1.078125

18
3000
36
163
0.078125
3
12
0
384
4591
1.875

18
9000
40
229
0.15625
3
12
0.046875
391
4675
3.65625

18
15000
51
325
0.359375
3
12
0
394
4711
5.421875

19
3000
4
30
0.015625
3
11
0.015625
2
14
0

19
9000
4
39
0.078125
3
11
0
2
14
0.015625

19
15000
4
39
0.09375
3
11
0.046875
2
14
0.015625

20
3000
38
134
0.09375
22
70
0.03125
3
18
0

20
9000
40
142
0.15625
30
94
0.0625
3
18
0.015625

20
15000
40
137
0.1875
38
118
0.125
3
18
0.046875

21
3000
21
64
0.046875
38
121
0.09375
27
103
0.125

21
9000
21
64
0.265625
38
121
0.21875
27
103
0.1875

21
15000
21
64
0.1875
38
121
0.296875
26
92
0.234375

22
3000
2
14
0.015625
6
35
0.0625
6
28
0.09375

22
9000
2
14
0.046875
6
35
0.15625
6
28
0.171875

22
15000
2
14
0.09375
6
35
0.328125
6
28
0.515625

23
3000
23
109
0.078125
180
610
0.546875
20
71
0.09375

23
9000
23
109
0.28125
190
657
1.5
1000
3003
7.625

23
15000
23
109
0.359375
206
707
2.734375
1000
3003
11.65625

24
3000
9
44
0.015625
50
191
0.109375
6
68
0.015625

24
9000
9
44
0.125
25
107
0.125
6
69
0.125

24
15000
9
44
0.125
31
129
0.25
6
69
0.15625

25
3000
6
27
0
18
53
0.046875
33
98
0.046875

25
9000
9
35
0.078125
28
83
0.046875
51
165
0.171875

25
15000
6
26
0.0625
34
101
0.09375
15
69
0.078125

26
3000
86
273
0.28125
96
305
0.171875
95
304
0.15625

26
9000
103
351
0.59375
107
338
0.453125
94
301
0.40625

26
15000
96
402
1.03125
98
313
0.671875
95
304
0.640625

27
3000
244
2809
1.625
22
94
0.03125
29
119
0.046875

27
9000
195
2290
2.40625
64
256
0.203125
1000
3004
3.25

27
15000
946
3784
5.984375
50
211
0.296875
21
80
0.078125

28
3000
9
46
0.03125
44
151
0.09375
36
126
0.078125

28
9000
9
46
0.0625
50
178
0.15625
36
123
0.09375

28
15000
9
46
0.078125
37
124
0.171875
34
125
0.203125

29
3000
3
18
0
4
25
0
4
24
0.046875

29
9000
3
18
0
4
24
0
4
24
0.046875

29
15000
3
18
0.03125
4
32
0.03125
4
24
0.03125

30
3000
135
1388
0.609375
1000
3063
0.921875
1000
3013
1.125

30
9000
591
4199
4.0625
1000
3066
1.9375
1000
3013
2.328125

30
15000
107
394
0.421875
1000
3083
2.609375
1000
3013
3.359375

31
3000
32
108
0.078125
40
125
0.0625
44
139
0.0625

31
9000
31
98
0.09375
43
136
0.109375
45
142
0.125

31
15000
33
104
0.1875
44
139
0.21875
48
151
0.25

32
3000
31
94
0.078125
33
102
0.046875
50
153
0.078125

32
9000
27
84
0.140625
64
195
0.328125
41
128
0.234375

32
15000
29
90
0.265625
45
140
0.3125
46
143
0.34375

33
3000
4
13
0
4
12
0
3
9
0

33
9000
2
6
0.046875
2
6
0
2
6
0

33
15000
2
6
0
2
6
0
2
6
0.015625

34
3000
4
12
0.03125
4
11
0
3
16
0

34
9000
4
12
0.015625
5
14
0.015625
7
20
0.03125

34
15000
4
12
0.015625
6
17
0.03125
9
26
0.09375

35
3000
3
17
0.015625
8
23
0.03125
14
41
0.03125

35
9000
3
17
0.03125
12
35
0.03125
21
62
0.09375

35
15000
5
14
0.03125
15
44
0.078125
25
74
0.140625

36
3000
88
344
1.75
20
81
0.359375
7
38
0.203125

36
9000
35
149
2.125
17
69
0.875
7
38
0.453125

36
15000
94
394
9
33
123
2.796875
7
38
0.84375

37
3000
380
1413
0.734375
1000
3111
1.328125
23
85
0.046875

37
9000
321
1183
1.375
344
1155
1.109375
1000
3013
3.671875

37
15000
1000
3686
6.40625
1000
3078
4.375
1000
3005
4.90625

38
3000
94
374
0.1875
9
30
0.015625
5
19
0.03125

38
9000
215
858
1
22
74
0.046875
4
14
0.03125

38
15000
294
1173
2.125
17
57
0.078125
4
14
0.046875

39
3000
6
18
0
18
53
0.015625
38
113
0.046875

39
9000
6
18
0.03125
19
56
0.03125
41
122
0.109375

39
15000
6
18
0.0625
20
59
0.09375
43
128
0.1875

40
3000
59
234
0.40625
1000
3116
6.046875
3
19
0.046875

40
9000
3
18
0.0625
1000
3109
16.03125
3
17
0.046875

40
15000
3
17
0.078125
1000
3102
25.765625
3
17
0.078125

41
3000
16
96
0.0625
28
98
0.03125
16
60
0.046875

41
9000
23
139
0.125
23
78
0.0625
16
60
0.0625

41
15000
34
242
0.34375
19
66
0.078125
16
60
0.078125

42
3000
6
47
0.015625
4
28
0
6
42
0.03125

42
9000
6
47
0.078125
4
28
0.046875
6
42
0.03125

42
15000
7
60
0.125
4
28
0.0625
6
42
0.0625

43
3000
21
188
0.296875
19
60
0.21875
58
186
0.421875

43
9000
2
14
0.0625
20
63
0.359375
68
216
1.234375

43
15000
24
81
0.765625
20
63
0.59375
72
228
2.3125

44
3000
2
14
0.015625
27
84
0.203125
66
216
0.4375

44
9000
2
14
0.046875
28
87
0.5
75
241
1.328125

44
15000
2
14
0.078125
28
87
0.78125
77
247
2.3125

45
3000
13
40
0.09375
43
132
0.28125
75
228
0.515625

45
9000
13
40
0.328125
45
138
0.8125
79
240
1.484375

45
15000
13
40
0.375
46
141
1.34375
81
246
2.28125

46
3000
86
663
0.96875
60
194
0.40625
24
149
0.28125

46
9000
2
14
0.046875
153
510
2.90625
178
1091
5.453125

46
15000
2
14
0.078125
154
519
4.703125
200
1355
10.203125

47
3000
76
230
13.859375
76
229
13.515625
20
71
3.71875

47
9000
55
188
92.234375
34
111
53.96875
11
45
17.125

47
15000
82
304
397.6875
86
276
382.3125
92
306
416.421875

48
3000
12
52
0.09375
30
97
0.140625
23
82
0.109375

48
9000
119
665
2.71875
35
115
0.4375
37
133
0.5

48
15000
177
1485
8.859375
33
109
0.6875
126
383
2.546875

49
3000
516
5422
2.796875
1000
3093
0.890625
1000
3013
1.09375

49
9000
153
560
0.5
1000
3171
1.9375
1000
3013
2.390625

49
15000
487
1800
2.109375
1000
3072
2.890625
1000
3013
3.453125

50
3000
194
1609
8.40625
1000
3025
14.765625
1000
3013
15.203125

50
9000
320
3100
43
169
560
8.484375
1000
3013
42.484375

50
15000
241
888
20.484375
1000
3107
71.375
1000
3013
70.25

51
3000
49
218
0.28125
16
49
0.078125
28
85
0.125

51
9000
2
14
0.03125
15
46
0.171875
28
85
0.28125

51
15000
2
14
0.078125
15
46
0.234375
28
85
0.46875

52
3000
149
474
0.421875
173
548
0.46875
182
575
0.453125

52
9000
170
541
1.140625
192
607
1.15625
206
649
1.28125

52
15000
178
565
2.046875
201
636
2.03125
215
678
1.984375

53
3000
3
26
0.03125
441
1487
0.453125
3
26
0.046875

53
9000
3
27
0.015625
430
1444
1.03125
56
193
0.140625

53
15000
2
14
0.03125
1000
3078
3.46875
2
14
0.015625

54
3000
381
1403
0.765625
21
93
0.03125
3
18
0.046875

54
9000
48
185
0.203125
98
354
0.3125
3
18
0.03125

54
15000
1000
3978
7
65
238
0.390625
3
18
0.015625

55
3000
19
65
0.109375
14
43
0.046875
3
25
0.0625

55
9000
3
26
0.09375
15
46
0.171875
23
100
0.390625

55
15000
3
26
0.140625
15
46
0.34375
23
104
0.75

56
3000
3
16
0
1000
3125
0.984375
1000
2999
1.0625

56
9000
3
16
0.046875
1000
3151
2.078125
1000
2999
2.1875

56
15000
3
16
0.015625
1000
3125
3.359375
1000
2999
3.71875

57
3000
2
14
0.015625
71
226
0.5
137
1398
2.203125

57
9000
2
14
0.046875
67
211
1.234375
102
602
3.046875

57
15000
2
14
0.078125
69
221
1.984375
210
1546
10.84375

58
3000
13
40
0.09375
109
358
0.796875
584
1985
3.921875

58
9000
13
40
0.21875
119
398
2.234375
22
88
0.453125

58
15000
13
40
0.390625
149
490
4.65625
22
88
0.6875

59
3000
3
18
0.015625
48
170
0.34375
189
2038
3.140625

59
9000
3
18
0.09375
46
163
0.875
239
2683
10.328125

59
15000
3
18
0.09375
54
187
1.71875
256
2912
17.8125

60
3000
44
377
1
110
357
0.734375
80
466
0.859375

60
9000
2
14
0.046875
184
619
3.515625
97
743
3.453125

60
15000
2
14
0.078125
184
624
5.859375
104
1000
6.4375

61
3000
2
14
0.015625
73
236
0.515625
175
1891
2.875

61
9000
2
14
0.046875
78
255
1.453125
188
1193
5.796875

61
15000
2
14
0.109375
69
229
2.03125
114
649
4.96875

62
3000
3
18
0.015625
86
280
0.625
27
103
0.1875

62
9000
3
18
0.078125
71
240
1.28125
126
702
3.609375

62
15000
3
18
0.078125
47
160
1.390625
75
415
3.25

63
3000
15
46
0.15625
139
455
1
157
627
1.1875

63
9000
15
46
0.265625
252
818
4.734375
72
309
1.515625

63
15000
15
46
0.453125
221
742
6.953125
87
363
3.046875

64
3000
18
55
0.140625
73
222
0.4375
24
86
0.1875

64
9000
18
55
0.34375
77
234
1.328125
24
86
0.484375

64
15000
18
55
0.578125
79
240
2.328125
24
86
0.859375

65
3000
3
19
0.109375
31
94
0.515625
33
102
0.578125

65
9000
3
19
0.515625
30
91
1.4375
31
96
1.5

65
15000
3
19
0.6875
30
91
2.484375
31
96
2.484375

66
3000
6
30
0.203125
4
21
0.078125
4
19
0.078125

66
9000
6
30
0.5
4
21
0.234375
4
19
0.265625

66
15000
120
479
11.015625
4
21
0.453125
4
19
0.375

67
3000
10
64
0.09375
7
69
0.03125
11
72
0.046875

67
9000
7
42
0.046875
38
258
0.796875
11
69
0.15625

67
15000
22
136
0.890625
24
170
0.71875
11
67
0.234375

68
3000
24
80
0.078125
22
67
0.03125
44
491
0.328125

68
9000
2
14
0
23
70
0.046875
2
14
0.03125

68
15000
2
14
0.0625
23
70
0.078125
2
14
0.03125

69
3000
5
33
0
53
163
0.109375
28
88
0.0625

69
9000
5
33
0.046875
51
157
0.21875
20
71
0.140625

69
15000
5
33
0.046875
51
157
0.359375
20
71
0.171875

70
3000
66
277
1.125
43
167
0.671875
13
70
0.203125

70
9000
10
45
0.5
21
90
1.03125
15
77
0.71875

70
15000
68
245
4.96875
9
34
0.59375
25
98
2.03125

71
3000
3
16
0
42
133
0.046875
3
16
0.046875

71
9000
3
16
0.015625
42
133
0.0625
3
16
0

71
15000
3
16
0.015625
42
133
0.125
3
16
0.046875

72
3000
206
765
41.4375
163
583
31.453125
15
60
2.78125

72
9000
1000
3694
1734.984375
1000
3036
1571.609375
15
60
24.125

72
15000
1000
3611
4739.8125
163
568
751.625
15
60
66.703125

73
3000
562
6118
2.671875
320
1055
0.28125
1000
3017
0.984375

73
9000
1000
9069
8.703125
1000
3048
1.90625
1000
3017
2.375

73
15000
798
5688
6.53125
1000
3041
2.71875
1000
3017
3.078125

Figure 1: Performance profiles of these methods (CPU).

Figure 2: Performance profiles of these methods (NI).

Figure 3: Performance profiles of these methods (NGF).

From the detailed experimental data of Table 2, it is obvious to see that most of the problems can be solved quickly. For most of the problems, it takes less CPU time to solve those problems with the proposed algorithm. Meanwhile, progress has also been made in NI and NFG. Generally, the algorithm of the proposed method is promising versus the other algorithms. About the numerical results, the algorithm of Dolan and More [22] will be used to more directly show the performance profiles of these algorithms. In Figure 1, the curve of Algorithm 1 is always above the other algorithms. In Figure 2, the curves have the same trend. Algorithm 1 almost solves about of the test problems in , while Algorithm 2 just solves and Algorithm 3 just solves in . Figure 3 shows a similar trend as Figure 2. All the above pictures show that the modified CD conjugate gradient algorithm is more robust and effective compared with the normal CD method and PRP method. In summary, Algorithm 1 is more competitive versus others.

4.2. The Muskingum Model

It is generally known that parameter estimation is a significant task in engineering applications. The nonlinear Muskingum model will be discussed as a common example of such an application in this subsection.

The Muskingum Model [23] is defined by the following:

Some of the variables have the following meanings:

(i): the total time number.(ii): the storage time constant.(iii): the observed inflow discharge.(iv): the weighting factor.(v): the observed outflow discharges.(vi) : an additional parameter.(vii) : the time step at time .

In the experiment, the observed data of the flood run-off process from Chenggouwan and Linqing of Nanyunhe River in the Haihe Basin, Tianjin, China, are used. We choose the initial point ; detailed data about and for the years 1960, 1961, and 1964 were obtained (see [24] in detail). was selected. The results of these three algorithms are listed in Table 3. The performance of the presented algorithm is shown in Figures 4–6.

Table 3: Results of the three algorithms.

Algorithms

BFGS [25]
10.8156
0.9826
1.0219

HIWO [23]
13.2813
0.8001
0.9933

Algorithm 1
11.1848
1.0000
0.9995

Figure 4: Performance of date in 1960.

Figure 5: Performance of date in 1961.

Figure 6: Performance of date in 1961.

Some conclusions are obtained from this experiment: (1) from Figures 4–6, we conclude that we can calculate approximations of flood outflows by using Algorithm 1, and Algorithm 1 is effective for the nonlinear Muskingum model; (2) the final points (, and ) of these three algorithms are interesting, and they are competitive with the final points of similar algorithms; and (3) the final points of Algorithm 1 are different from those of the BFGS method and HIWO method, which implies that the nonlinear Muskingum model has different optimum approximation solutions.

4.3. Image Restoration Problems

In this subsection, the above algorithms will be applied to image restoration problems. The original images corrupted by impulse noise are treated as objects here. These problems are regarded as one of the most difficult problems in optimization fields. Related parameter settings are similar to the above subsections, and the program will be stopped when the condition  or  holds. The following three images are selected as processing objects: Banoon (512512), Barbara (512512), and Lena (512512). The detail performances are shown in Figures 7–9. The CPU time taken to process images is listed in Table 4.

Figure 7: Restoration of the Banoon image, Barbara image, and Lena image by Algorithms 1–3. From left to right: a noisy image with salt-and-pepper noise, the restorations obtained with Algorithms 1–3.

Figure 8: Restoration of the Banoon image, Barbara image, and Lena image by Algorithms 1–3. From left to right: a noisy image with salt-and-pepper noise, the restorations obtained with the Algorithms 1–3.

Figure 9: Restoration of the Banoon image, Barbara image, and Lena image by Algorithms 1–3. From left to right: a noisy image with salt-and-pepper noise, the restorations obtained with the Algorithms 1–3.

Table 4: CPU time of those algorithms in seconds.

noise

Total

Algorithm 1
4.438
4.390
4.656
13.484

Algorithm 2
4.563
5.234
5.406
15.203

Algorithm 3
5.156
6.313
6.781
18.250

noise

Total

Algorithm 1
8.609
7.891
8.141
24.641

Algorithm 2
9.219
10.688
10.547
30.454

Algorithm 3
11.297
14.391
14.859
40.547

noise

Total

Algorithm 1
15.750
10.578
11.984
38.312

Algorithm 2
13.891
14.438
14.703
43.032

Algorithm 3
14.859
19.922
20.375
55.156

It is easy to see that all the algorithms are successful for image restoration problems. The results in Table 4 reveal that the CPU time of Algorithm 1 is less than that of other algorithms, whether for $30\%$ noise problems, noise problems, or for noise problems.

5. Conclusion

In this paper, a modified three-term type CD conjugate gradient algorithm is presented. Some good features are also presented: (i) sufficient descent property holds, (ii) trust region feature also holds, (iii) the algorithm has global convergence with the MWWP line search technique and projection technique for general function, and (iv) numerical results reveal that the new algorithm is more competitive than the normal CD algorithm and PRP algorithm.

In recent years, there have been considerable researches about other types of CG method, while the study of CD method is not enough, and it should not be ignored. We have many works to do in the future: whether this method is suitable for other line search technique (such as Armijo line search, nonmonotone line search), or whether there exist other better modification methods to improve the numerical results of the CD method. All these are worth studying in the next work.

Data Availability

The data used to support the findings of this study are included within the article.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant no. 11661009), the High Level Innovation Teams and Excellent Scholars Program in Guangxi Institutions of Higher Education (Grant No.[2019]52), and the Guangxi Natural Science Key Fund (No. 2017GXNSFDA198046). The authors would like to thank the editor and the referees for their valuable comments, which greatly improve this paper.

References

M. Al-Baali, “Descent property and global convergence of the fletcher-reeves method with inexact line search,” IMA Journal of Numerical Analysis, vol. 5, no. 1, pp. 121–124, 1985.

Y. Dai and Y. Yuan, “A nonlinear conjugate gradient with a strong global convergence properties,” SIAM Journal on Optimization, vol. 10, pp. 177–182, 2000.

J. W. Daniel, “The conjugate gradient method for linear and nonlinear operator equations,” SIAM Journal on Numerical Analysis, vol. 4, no. 1, pp. 10–26, 1967.

R. Fletcher, Practical Methods of Optimization, Wiley, New York, NY, USA, 2nd. edition, 1987.

R. Fletcher and C. M. Reeves, “Function minimization by conjugate gradients,” The Computer Journal, vol. 7, no. 2, pp. 149–154, 1964.

M. R. Hestenes and E. Stiefel, “Methods of conjugate gradients for solving linear systems,” Journal of Research of the National Bureau of Standards, vol. 49, no. 6, pp. 409–436, 1952.

Y. Liu and C. Storey, “Efficient generalized conjugate gradient algorithms, part 1: theory,” Journal of Optimization Theory and Applications, vol. 69, no. 1, pp. 129–137, 1991.

E. Polak and G. Ribiere, “Note sur la convergence de méthodes de directions conjuguées,” Revue française d’informatique et de recherche opérationnelle. Série rouge, vol. 3, no. 16, pp. 35–43, 1969.

B. T. Polyak, “The conjugate gradient method in extremal problems,” USSR Computational Mathematics and Mathematical Physics, vol. 9, no. 4, pp. 94–112, 1969.

G. Yuan, X. Wang, and Z. Sheng, “Family weak conjugate gradient algorithms and their convergence analysis for nonconvex functions,” Numerical Algorithms, vol. 84, no. 3, pp. 935–956, 2020.

S.-Y. Liu, Y.-Y. Huang, and H.-W. Jiao, “Sufficient descent conjugate gradient methods for solving convex constrained nonlinear monotone equations,” Abstract and Applied Analysis, vol. 2014, pp. 1–12, 2014.

X. Y. Wang, S. J. Li, and X. P. Kou, “A self-adaptive three-term conjugate gradient method for monotone nonlinear equations with convex constraints,” Calcolo, vol. 53, no. 2, pp. 133–145, 2016.

G. Yuan and W. Hu, “A conjugate gradient algorithm for large-scale unconstrained optimization problems and nonlinear equations,” Journal of Inequalities and Applications, vol. 113, no. 1, 2018.

G. Yuan, T. Li, and W. Hu, “A conjugate gradient algorithm for large-scale nonlinear equations and image restoration problems,” Applied Numerical Mathematics, vol. 147, pp. 129–141, 2020.

G. Yuan, J. Lu, and Z. Wang, “The PRP conjugate gradient algorithm with a modified WWP line search and its application in the image restoration problems,” Applied Numerical Mathematics, vol. 152, pp. 1–11, 2020.

G. Yuan, Z. Wei, and Y. Yang, “The global convergence of the polak-ribière-polyak conjugate gradient algorithm under inexact line search for nonconvex functions,” Journal of Computational and Applied Mathematics, vol. 362, pp. 262–275, 2019.

L. Zhang, W. Zhou, and D.-H. Li, “A descent modified polak-ribière-polyak conjugate gradient method and its global convergence,” IMA Journal of Numerical Analysis, vol. 26, no. 4, pp. 629–640, 2006.

G. Yuan, W. Hu, and Z. Sheng, “A conjugate gradient algorithm with yuan-wei-lu line search,” in International Conference on Cloud Computing and Security, Springer, Cham, Switzerland, 2017.

G. Yuan, Z. Wei, and X. Lu, “Global convergence of BFGS and PRP methods under a modified weak Wolfe-Powell line search,” Applied Mathematical Modelling, vol. 47, pp. 811–825, 2017.

G. Yuan, Z. Sheng, B. Wang, W. Hu, and C. Li, “The global convergence of a modified BFGS method for nonconvex functions,” Journal of Computational and Applied Mathematics, vol. 327, pp. 274–294, 2018.

Y. Yuan and W. Sun, Theory and Methods of Optimization, Science Press of China, Beijing, China, 1999.

E. D. Dolan and J. J. Moré, “Benchmarking optimization software with performance profiles,” Mathematical Programming, vol. 91, no. 2, pp. 201–213, 2002.

A. Ouyang, L.-B. Liu, Z. Sheng, and F. Wu, “A class of parameter estimation methods for nonlinear muskingum model using hybrid invasive weed optimization algorithm,” Mathematical Problems in Engineering, vol. 2015, pp. 1–15, 2015.

A. Ouyang, Z. Tang, K. Li, A. Sallam, and E. Sha, “Estimating parameters of muskingum model using an adaptive hybrid PSO algorithm,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 28, pp. 1–29, 2014.

Z. W. Geem, “Parameter estimation for the nonlinear muskingum model using the BFGS technique,” Journal of Irrigation and Drainage Engineering, vol. 132, no. 5, pp. 474–478, 2006.