Mathematical Problems in Engineering
Volume 2021 (2021), Article ID 9919595, 10 pages
https://doi.org/10.1155/2021/9919595

Research Article

A Modified Conjugate Gradient Method for Solving Large-Scale Nonlinear Equations

Hongbo Guan and Sheng Wang

School of Mathematics, Physics and Energy Engineering, Hunan Institute of Technology, Hengyang 421002, China

Correspondence should be addressed to Sheng Wang; wangshengmath@163.com

Received 26 March 2021; Revised 24 May 2021; Accepted 5 June 2021; Published 24 June 2021

Academic Editor: Zhifeng Dai

Copyright © 2021 Hongbo Guan and Sheng Wang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

In this paper, we propose a modified Polak–Ribière–Polyak (PRP) conjugate gradient method for solving large-scale nonlinear equations. Under weaker conditions, we show that the proposed method is globally convergent. We also carry out some numerical experiments to test the proposed method. The results show that the proposed method is efficient and stable.

1. Introduction

Solving nonlinear equations is an important problem which appears in various models of science and engineering such as computer vision, computational geometry, signal processing, computational chemistry, and robotics. More specifically, the subproblems in the generalized proximal algorithms with Bergman distances is a monotone nonlinear equations [1], and -norm regularized optimization problems can be reformulated as monotone nonlinear equations [2]. Due to its wide applications, the studies in the numerical methods for solving the monotone nonlinear equations have received much attention [3–10]. In this paper, we are interested in the numerical methods for solving monotone nonlinear equations with convex constraints:

where is a continuous function and is a nonempty, closed, and convex set. The monotonicity of the mapping means that

The methods for solving monotone nonlinear equations (1) are closely relevant to the methods for solving the following optimization problems:

Notice that if is strictly convex, then is strictly monotone which means It is well known that the strictly convex function exists a unique solution , satisfying To sum up, if there is a convex function satisfying , then solving the optimization problems (3) is equivalent to solving monotone nonlinear equations (1). So, a natural idea to solve monotone nonlinear equations (1) is to use the existing efficient methods for solving optimization problems (3). There are many methods for solving optimization problems (3), such as the Newton method, quasi-Newton method, trust region method, and conjugate gradient method. Among these methods, the conjugate gradient method is a very effective method for solving optimization problems (3) due to their simplicity and low storage. A conjugate gradient method generates a sequence of iterates:

where is the step length and direction is defined by

where is a parameter and is the gradient of the objective function . The choice of determines different conjugate gradient methods [11–17]. We are interested in the PRP conjugate gradient method in which the parameter is defined by

where . Based on the idea of [18, 19], Zhang et al. [20] proposed a new modified nonlinear PRP method in which is defined by

where , and are two constants. There is a mistake about the definition of . By this definition, we will not be able to prove Lemma 1 in [20]. It should be

There are many conjugate gradient methods for solving nonlinear equations (1). Zhang and Zhou [4] proposed a spectral gradient method by combining the modified spectral gradient and projection method, which can be applied to solve nonsmooth equations. Xiao and Zhou [10] extended the CG-DESCENT method to solve large-scale nonlinear monotone equations and extended this method to decode a sparse signal in compressive sensing. Dai and Zhu [21] proposed a derivative-free method for solving large-scale nonlinear monotone equations and proposed a new line search for the derivative-free method. Other related works can be found [3, 5–8, 10, 22–30]. In this paper, we combined the projection method [3], the modified nonlinear PRP conjugate gradient method for unconstrained optimization [20] and the iterative method [10] and proposed a modified nonlinear conjugate gradient method for solving large-scale nonlinear monotone equations with convex constrains.

This paper is organized as follows. In Section 2, we propose a modified nonlinear PRP method for solving monotone nonlinear equations with convex constraints. Under reasonable conditions, we prove its global convergence. In Section 3, we make some improvement to the proposed method and give the convergence theorem of the improved method. In Section 4, we do some numerical experiments to test the proposed methods. The results show that our methods are efficient and promising. Furthermore, we use the proposed methods to solve practical problems in compressed sensing.

2. A Modified Nonlinear PRP Method

In this section, we develop a modified nonlinear PRP method for solving the nonlinear equations with convex constraints. Based on the modified nonlinear PRP method [20], we now introduce our method for solving (1). Inspired by (8), we define as

where The parameter is computed as

where and are two constants.

The lemma below shows a good property of The steps of the method are given in Algorithm 1.

Lemma 1. Let be generated by Algorithm 1. If , then there exists a constant such that

Proof. For we have

For we obtain

Denote

Then, we obtain

Let ; then, inequality (11) is satisfied.
Next, we establish the global convergence of the proposed method. Without specification, we always suppose that the solution set of equation (1) is nonempty and the following assumption holds.

Assumption 1.

(i)The mapping is Lipchitz continuous, and it means that the mapping satisfies

(ii)The projection operator is nonexpansive, i.e.,

Lemma 2. Suppose that Assumption 1 holds and is a solution of (1), and the sequence and are generated by Algorithm 1. Then, the sequence , , and are bounded.

Initial. Given a small constant and constants . Choose an initial point . Let .

Step 1. Stop if .

Step 2. Compute (9) to get .

Step 3. Let satisfying

denote

Step 4. Compute

where

and is a projection operator, defined by .

Step 5. Let and go to Step 1.

Algorithm 1: Modified NPRP method.

Proof. We first show that is bounded. From the monotonicity of function , we have

It is easy to see that

The last inequality implies

It obviously that the sequences is bounded, i.e., there is a constant such that

Next, we show that is bounded. Since is Lipchitz continuous, we obtain

Let ; then, is bounded. That is,

At last, we prove that the sequence is bounded. From (2) and (23) and Algorithm 1, we obtain

So, the following inequality holds:

It implies that the sequence is bounded.

Lemma 3. Suppose that Assumption 1 holds, and the sequence and is generated by Algorithm 1. Then, there exists a constant such that

Proof. We first prove the right side of inequality (26). For , from (9), we have

For , by the definition of and (21), we obtain

By the definition of (9) and the last inequality, we obtain

Let ; then, we have
Now, we turn to prove the left side of the inequality. It follows from (11) that

Therefore, we have

Lemma 4. Suppose Assumption 1 holds; then, the step length satisfies

Proof. If the Algorithm 1 terminates in a finite number of steps, then there is a such that is a solution of equation (1) and . From now on, we assume that , for any . It is easy to see that from (11).
If , by the line search process, we know that does not satisfy Algorithm 1, that is,

It follows from (11) and Assumption 1 that

From the last inequality and Lemma 3, we obtain

Hence, it holds that

Lemma 5. Suppose that Assumption 1 holds, and the sequence and are generated by Algorithm 1. Then, we have the following:

Proof. Noticed that

Since the function is continuous, and the sequence is bounded, so the sequence is bounded. That is, for all , there exists a positive constant , such that Then, we obtain

So, we have

Theorem 1. Suppose that Assumption 1 holds. The sequence is generated by Algorithm 1. Then, we have

Proof. Suppose that (41) does not hold; then, there exists such that, for any

From (26) and the last inequality, it is easy to see

From (41) and (42), we obtain

The last inequality yields a contradiction with (37), so (41) is satisfied.

3. An Improvement

In this section, we make some improvement to the modified nonlinear PRP method proposed in Section 2. In Algorithm 1, we take the step length . Is there a better choice for ? This is our purpose to improve Algorithm 1. Under the condition of ensuring the convergence of the algorithm and the related good properties and results, we improve Algorithm 1 in order to get better numerical results.

From Algorithm 1, to make the inequality hold, we only need to satisfy

By solving the last inequality, we have

It is easy to see that is the minimum point of the function . This is the reason why Algorithm 1 takes . Under reasonable conditions, we hope to get a large step length than Algorithm 1. So, we obtain

Based on the above arguments, we propose an improved algorithm of Algorithm 1. In the improved algorithm, we make the step length:

Similar to the proof of Theorem 1, we have the following results.

Theorem 2. Suppose that Assumption 1 holds. The sequence is generated by Algorithm 2; then, we have

The iterative process of the improved method is stated as follows.

Initial. Given a small constant and constants . Choose an initial point . Let .

Step 1. Stop if .

Step 2. Compute (9) to get .

Step 3. Let satisfying

denote

Step 4. Compute

where

Step 5. Let and go to Step 1.

Algorithm 2: The improved modified nonlinear PRP method.

4. Numerical Results

In this section, we do some numerical experiments to test the performance of the proposed methods. We implemented our methods in MATLAB R2020b and run the codes on a personal computer with 2.3 GHz CPU and 16 GB RAM.

We first solve Problems 1 and 2.

Problem 1. (see [8]). The mapping is taken as where

Problem 2. (see [4]). The mapping is taken as where

and
We set The stopping criterion of the algorithm is set to or the number of iteration reach to 500. The latter case means that the method is a failure for the test problems. We test both problems with the dimensions of variables , and . Start from different initial points, and we list all results in Tables 1 and 2. We compared the performance of the proposed methods with the classical Newton method and an efficient algorithm CGD [10] in the total number of iterations as well as the computational time. The meaning of each column is given below.

 ‘Init’: the initial point ’: the dimension of the problem ‘Iter’: the total number of iterations ‘Time’: the CPU time (in seconds) used for the method ‘NM’: the Newton method ‘CGD’: the conjugate gradient method in [10] ‘MNPRP’: the modified nonlinear PRP method ‘IMNPRP’: the improved modified nonlinear PRP method

The results in Tables 1 and 2 show that our methods performs very well both in the number of iterations and CPU time. The IMNPRP performs best among these methods. It is worth noting that the number of iterations does not increase significantly as n increases. Hence, the proposed method is very suitable for solving large-scale problems. Because of the lack of memory, the dimension of the problems solved by the Newton method is no more than 100,000.
The following example is a signal reconstruction problem from compressed sensing.

Table 1: Comparison on Problem 1.

Init
n
NM
CGD
MNPRP
IMNPRP

Iter
Time
Iter
Time
Iter
Time
Iter
Time

1000
5
0.0345
10
0.0006
4
0.0003
1
0.0002

10000
5
2.0438
11
0.0030
4
0.0015
1
0.0009

100000
Out of memory
11
0.0229
4
0.0119
1
0.0073

1000000
Out of memory
12
0.4002
5
0.1595
1
0.0921

10000000
Out of memory
13
4.1112
5
1.7468
1
0.9706

1000
6
0.0325
12
0.0005
5
0.0004
1
0.0003

10000
6
2.4640
13
0.0041
5
0.0030
1
0.0023

100000
Out of memory
13
0.0299
6
0.0299
1
0.0175

1000000
Out of memory
14
0.4547
6
0.2838
1
0.1970

10000000
Out of memory
15
5.1400
7
3.4031
1
2.0876

1000
4
0.0226
10
0.0004
5
0.0002
1
0.0001

10000
4
1.6051
11
0.0031
5
0.0012
1
0.0006

100000
Out of memory
12
0.0247
5
0.0088
1
0.0040

1000000
Out of memory
12
0.3757
6
0.1368
1
0.0531

10000000
Out of memory
13
4.1184
6
1.5107
1
0.5842

1000
5
0.0335
10
0.0004
4
0.0002
1
0.0001

10000
5
2.0122
11
0.0027
4
0.0015
1
0.0010

100000
Out of memory
11
0.0187
4
0.0118
1
0.0071

1000000
Out of memory
12
0.3797
5
0.1575
1
0.0882

10000000
Out of memory
13
4.2217
5
1.7638
1
1.0510

Table 2: Comparison on Problem 2.

Init
n
NM
CGD
MNPRP
IMNPRP

Iter
Time
Iter
Time
Iter
Time
Iter
Time

1000
3
0.0187
11
0.0006
4
0.0002
1
0.0001

10000
3
1.1675
12
0.0033
4
0.0009
1
0.0004

100000
Out of memory
12
0.0245
4
0.0058
1
0.0028

1000000
Out of memory
13
0.4156
5
0.1036
1
0.0350

10000000
Out of memory
14
4.3595
5
1.1379
1
0.3650

1000
4
0.0207
11
0.0005
5
0.0003
1
0.0002

10000
4
1.6028
12
0.0036
5
0.0016
1
0.0009

100000
Out of memory
12
0.0312
6
0.0124
1
0.0061

1000000
Out of memory
13
0.4153
6
0.1628
1
0.0771

10000000
Out of memory
14
4.5275
6
1.7940
1
0.8300

1000
3
0.0226
11
0.0009
4
0.0002
1
0.0001

10000
3
1.1711
11
0.0030
4
0.0007
1
0.0002

100000
Out of memory
12
0.0248
4
0.0049
1
0.0015

1000000
Out of memory
13
0.3998
5
0.0936
1
0.0236

10000000
Out of memory
14
4.3801
5
0.9911
1
0.2697

1000
3
0.0188
11
0.0005
4
0.0002
1
0.0001

10000
3
1.1775
11
0.0029
4
0.0008
1
0.0003

100000
Out of memory
12
0.0234
4
0.0044
1
0.0023

1000000
Out of memory
13
0.3964
4
0.0828
1
0.0320

10000000
Out of memory
14
4.2691
5
1.1491
1
0.3430

Problem 3. (see [10]). Consider a typical compressive sensing scenario, where we aim to reconstruct a length-n sparse signal form m observations . In this test, the measurement b contains noise:

where is the Gaussian noise distributed as with . The random is the Gaussian matrix which is generated by command randn (m, n) in Matlab. The merit function

where the value τ is forced to decrease as the measure in [31]. The iterative process starts at the measurement image, i.e., , and terminates when the relative change between successive iterates falls below , i.e.,

where denotes the function value at . By the discussion in [10], we know that the -norm problem can transformed a monotone nonlinear equation. Hence, it can be solved by Algorithms 1 and 2.
Due to the storage limitations of the PC, we test a small size signal with , and the original contains randomly nonzero elements. The quality of restoration is measured by the mean of squared error (MSE) to the original signal , that is,

where is the restored signal. We take the parameters , and in CGD, MNPRP, and IMNPRP.
In order to test the effectiveness of the proposed methods, we compare the proposed methods with the CGD method [10] and the solver SGCS which is specially designed to solve monotone equations for recovering a large sparse signal in compressive sensing. The results are listed in Figures 1 and 2.
It can be seen from Figures 1 and 2 that all methods have recovered the original sparse signal almost exactly. Among these methods, the IMNPRP method performs best.

Figure 1: The original signal (top), the measurement (bottom), and the reconstructed signals by SGCS, CGD, MNPRP, and IMNPRP.

(a)

(b)

(c)

(d)

Figure 2: Comparison results on Problem 3. The y-axes represent the MSE (a, b) and the function values (c, d).

5. Conclusions

In this paper, a modified conjugate gradient method and its improved method are proposed for solving the large-scale nonlinear equations. Under some assumptions, global convergence of the proposed methods are established. Numerical results show that the proposed methods are very efficient and competitive.

Data Availability

All data generated or analysed during this study are included within the article.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This research was funded by the Education Department of Hunan Province (Grant no. 20C0559).

References

N. A. Iusem and V. M. Solodov, “Newton-type methods with generalized distances for constrained optimization,” Optimization, vol. 41, no. 3, pp. 257–278, 1997.

M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, “Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems,” IEEE Journal of Selected Topics in Signal Processing, vol. 1, no. 4, pp. 586–597, 2007.

M. V. Solodov and B. F. Svaiter, “A globally convergent inexact Newton method for systems of monotone equations,” in Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing-Methods, pp. 355–369, Kluwer Academic Publishers, Dordrecht,The Netherlands, 1998.

L. Zhang and W. Zhou, “Spectral gradient projection method for solving nonlinear monotone equations,” Journal of Computational and Applied Mathematics, vol. 196, no. 2, pp. 478–484, 2006.

G. Zhou and K. C. Toh, “Superlinear convergence of a Newton-type Algorithm for monotone equations,” Journal of Optimization Theory and Applications, vol. 125, no. 1, pp. 205–221, 2005.

W.-J. Zhou and D.-H. Li, “A globally convergent BFGS method for nonlinear monotone equations without any merit functions,” Mathematics of Computation, vol. 77, no. 264, pp. 2231–2240, 2008.

W. Zhou and D. Li, “Limited memory BFGS method for nonlinear monotone equations,” Journal of Computational Mathematics, vol. 25, pp. 89–96, 2007.

C. Wang, Y. Wang, and C. Xu, “A projection method for a system of nonlinear monotone equations with convex constraints,” Mathematical Methods of Operations Research, vol. 66, no. 1, pp. 33–46, 2007.

Z. Yu, J. Lin, J. Sun, Y. Xiao, L. Liu, and Z. Li, “Spectral gradient projection method for monotone nonlinear equations with convex constraints,” Applied Numerical Mathematics, vol. 59, no. 10, pp. 2416–2423, 2009.

Y. Xiao and H. Zhu, “A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing,” Journal of Mathematical Analysis and Applications, vol. 405, no. 1, pp. 310–319, 2013.

E. Polak and G. Ribiere, “Note sur la convergence de méthodes de directions conjuguées,” Revue française d'informatique et de recherche opérationnelle. Série rouge, vol. 3, no. 16, pp. 35–43, 1969.

B. T. Polyak, “The conjugate gradient method in extremal problems,” USSR Computational Mathematics and Mathematical Physics, vol. 9, no. 4, pp. 94–112, 1969.

M. R. Hestenes and E. Stiefel, “Methods of conjugate gradients for solving linear systems,” Journal of Research of the National Bureau of Standards, vol. 49, no. 6, pp. 409–436, 1952.

Y. Liu and C. Storey, “Efficient generalized conjugate gradient algorithms, part 1: theory,” Journal of Optimization Theory and Applications, vol. 69, no. 1, pp. 129–137, 1991.

Y. H. Dai and Y. Yuan, “A nonlinear conjugate gradient method with a strong global convergence property,” SIAM Journal on Optimization, vol. 10, no. 1, pp. 177–182, 1999.

R. Fletcher and C. M. Reeves, “Function minimization by conjugate gradients,” The Computer Journal, vol. 7, no. 2, pp. 149–154, 1964.

R. Fletcher, “Practical methods of optimization,” in Unconstrained Optimization, Wiley, New York, NJ, USA, 1980.

G. Yu, L. Guan, and W. Chen, “Spectral conjugate gradient methods with sufficient descent property for large-scale unconstrained optimization,” Optimization Methods and Software, vol. 23, no. 2, pp. 275–293, 2008.

M. Li and A. Qu, “Some sufficient descent conjugate gradient methods and their global convergence,” Computational and Applied Mathematics, vol. 33, no. 2, pp. 333–347, 2014.

M. Zhang, Y. Zhou, and S. Wang, “A modified nonlinear conjugate gradient method with the armijo line search and its application,” Mathematical Problems in Engineering, vol. 2020, Article ID 6210965, 14 pages, 2020.

Z. Dai and H. Zhu, “A modified hestenes-stiefel-type derivative-free method for large-scale nonlinear monotone equations,” Mathematics, vol. 8, no. 168, 2020.

N. Andrei, “A scaled BFGS preconditioned conjugate gradient algorithm for unconstrained optimization,” Applied Mathematics Letters, vol. 20, no. 6, pp. 645–650, 2007.

L. Zheng, L. Yang, and Y. Liang, “A conjugate gradient projection method for solving equations with convex constraints,” Journal of Computational and Applied Mathematics, vol. 375, Article ID 112781, 2020.

A. H. Ibrahim, P. Kumam, A. B. Abubakar, W. Jirakitpuwapat, and J. Abubakar, “A hybrid conjugate gradient algorithm for constrained monotone equations with application in compressive sensing,” Heliyon, vol. 6, no. 3, Article ID e03466, 2020.

A. M. Awwal, P. Abubakar, and A. B. Abubakara, “A modified conjugate gradient method for monotone nonlinear equations with convex constraints,” Applied Numerical Mathematics, vol. 145, pp. 507–520, 2019.

W. La Cruz, J. M. Martnez, and M. Raydan, “Spectral residual method without gradient information for solving large-scale nonlinear systems,” Mathematics of Computation, vol. 75, pp. 1449–1466, 2006.

W. L. Cruz, “A spectral algorithm for large-scale systems of nonlinear monotone equations,” Numerical Algorithms, vol. 76, no. 4, pp. 1109–1130, 2017.

Z. Dai, H. Zhou, J. Kang et al., “The skewness of oil price returns and equity premium predictability,” Energy Economics, vol. 94, Article ID 105069, 2021.

Z. Dai and J. Kang, “Some new efficient mean-variance portfolio selection models,” International Journal of Finance and Economics, vol. 7, pp. 1–13, 2021.

Z. F. Dai and H. Zhu, “Stock return predictability from a mixed model perspective,” Pacific-Basin Finance Journal, vol. 60, Article ID 101267, 2020.

M. Figueiredo, R. D. Nowak, and S. J. Wright, “Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems,” IEEE Journal of Selected Topics in Signal Processing, vol. 1, pp. 586–597, 2008.