Abstract and Applied Analysis

Volume 2014 (2014), Article ID 386030, 6 pages

http://dx.doi.org/10.1155/2014/386030

Research Article

A Globally Convergent Matrix-Free Method for Constrained Equations and Its Linear Convergence Rate

Min Sun¹ and Jing Liu²

¹School of Mathematics and Statistics, Zaozhuang University, Shandong 277160, China

²School of Mathematics and Statistics, Zhejiang University of Finance and Economics, Hangzhou 310018, China

Received 24 January 2014; Accepted 8 May 2014; Published 19 May 2014

Academic Editor: Vladimir Danilov

Copyright © 2014 Min Sun and Jing Liu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

A matrix-free method for constrained equations is proposed, which is a combination of the well-known PRP (Polak-Ribière-Polyak) conjugate gradient method and the famous hyperplane projection method. The new method is not only derivative-free, but also completely matrix-free, and consequently, it can be applied to solve large-scale constrained equations. We obtain global convergence of the new method without any differentiability requirement on the constrained equations. Compared with the existing gradient methods for solving such problem, the new method possesses linear convergence rate under standard conditions, and a relax factor is attached in the update step to accelerate convergence. Preliminary numerical results show that it is promising in practice.

1. Introduction

Let be a continuous nonlinear mapping and a nonempty closed convex set of . In this paper, we consider the problem of finding such that

Nonlinear constrained equations (1), denoted by CES (), arise in various applications, for instance, ballistic trajectory computation and vibration systems [1], the power flow equations [2], chemical equilibrium systems [3], and so forth.

In recent years, many numerical methods have been proposed to find a solution of nonsmooth CES (), which include the trust region methods [4, 5], the Levenberg-Marquardt method [6], and the projection methods [7–9]. Compared with the trust region method and the Levenberg-Marquardt method, the projection method is more efficient for solving large-scale CES (). Noting this, Wang et al. [7] proposed a projection method for solving CES (), which possesses global convergence property without the differentiability. A drawback of this method is that it needs to solve a linear equation inexactly at each iteration, and its variants [8, 10] also have this drawback.

It is well-known that the spectral gradient method and the conjugate gradient method are two efficient methods for solving large-scale unconstrained optimization problems due to their simplicity and low storage. Recently, La Cruz and Raydan [11] successfully applied the famous spectral gradient method to solve unconstrained equations by using some merit function. Then, Zhang and Zhou [12] presented a spectral gradient projection method (SGP) for solving unconstrained monotone equations, which does not utilize any merit function. Later, the SGP was extended by Yu et al. [9] to solve monotone constrained equations. However, the study of conjugate gradient methods for large-scale (un)constrained equations is relatively rare. Cheng [13] proposed a PRP type method (PRPT) for systems of monotone equations, which is a combination of the well-known PRP method and the hyperplane projection method, and the numerical results in [13] show that the PRPT method performs better than the SGP method in [12].

Different from the methods in [7, 8, 10], the methods in [9, 11–13] do not need to solve a linearized equation at each iteration; however, the latter do not investigate the convergent rate, and even we do not know whether they possess the linear convergence rate. In this paper, motivated by the projection methods in [7, 8, 10] and the gradient methods in [9, 12, 13], we propose a matrix-free method for solving nonlinear constrained equations, which can be viewed as a combination of the well-known PRP conjugate gradient method and the famous hyperplane projection method, and it possesses linear convergence rate under standard conditions. The remainder of this paper is organized as follows. Section 2 describes the new method and presents its global convergence analysis. The linear convergence rate of the new method is established in Section 3. Numerical results are reported in Section 4. Finally, some final remarks are included in Section 5.

2. Algorithm and Convergence Analysis

Let denote the solution set of CES (). Throughout this paper, we assume that is nonempty and is monotone; that is,

which implies that the solution set is closed. Then let denote the orthogonal projection of a point onto the convex set , which has the following nonexpansive property:

Now, we describe the matrix-free method for nonlinear constrained equations.

Algorithm 1. Consider the following.

Step  0. Given an arbitrary initial point , the parameters , , , and . Given the initial steplength and set .

Step  1. If , then stop; otherwise go to Step  2.

Step  2. Compute by

where

If , set .

Step  3. Find the trial point , where with being the smallest nonnegative integer such that

Step  4. Compute

where

Choose an initial steplength such that . Set and go to Step  1.

Remark 2. Obviously , defined by (4), is motivated by [14], and it is not difficult to deduce that satisfies

Therefore, by Cauchy-Schwartz inequality, we have . This together with Step  2 of Algorithm 1 implies

Remark 3. In (7), we attach a relax factor (better when close to 2) to based on numerical experiences.

Remark 4. Line search (6) is different from that of [12, 13], which is well-defined by the following Lemma.

Lemma 5. For all , there exists a nonnegative number satisfying (6).

Proof. In fact, if , then from (10), we have , which means that Algorithm 1 terminates with being a solution of CES (). Now, we consider for all . For the sake of contradiction, we suppose that there exists such that (6) is not satisfied for any nonnegative integer ; that is,

Letting and using the continuity of yield

On the other hand, by (10), we obtain

which together with (12) means that ; however, this contradicts the fact that . Therefore the assertion holds. This completes the proof.

Lemma 6. Suppose that is monotone and let and be the sequences generated by Algorithm 1; then and are both bounded; furthermore, it holds that

Proof. From (6), we have

For any , from (3), the nonexpansiveness of the projection operator, it holds that

By the monotonicity of mapping , we have

Substituting (15) and (17) into (16), we have

which together with indicates that, for all ,

which shows that the sequence is bounded. By (10), it holds that is bounded and so is . Then, by the continuity of , there exists a constant such that for all . Therefore it follows from (18) that

which implies that the assertion (14) holds. The proof is completed.

Now, we prove the global convergence of Algorithm 1.

Theorem 7. Suppose that the conditions in Lemma 6 hold. Then the sequence generated by Algorithm 1 globally converges to a solution of CES ().

Proof. We consider the following two possible cases.

Case  1. Consider . Thus, by (10), we have . This together with the continuity of implies that the sequence has some accumulation point such that . From (19), it holds that converges, and since is an accumulation point of , it must hold that converges to .

Case  2. Consider . Then by (14), it follows that . Therefore, from the line search (6), for sufficiently large , we have

Since are both bounded, we can choose a sequence and letting in (21), we can obtain

where and are limit points of corresponding subsequences. On the other hand, by (10), we obtain

Letting in the above inequality, we obtain

Thus, by (22) and (24), we get , and this contradicts the fact that . Therefore, does not hold. This completes the proof.

3. Convergence Rate

By Theorem 7, we know that the sequence generated by Algorithm 1 converges to a solution of CES (). In what follows, we always assume that as , where . To establish the local convergence rate of the sequence generated by Algorithm 1, we need the following assumption.

Assumption 8. For , there exist three positive constants , , and such that

where dist denotes the distance from to the solution set , and

Now, we analyze the convergence rate of the sequence generated by Algorithm 1 under conditions (25) and (26).

Lemma 9. If the conditions in Assumption 8 hold, then the sequence generated by line search (6) has a positive bound from below.

Proof. We only need to prove that for sufficiently large , has a positive bound from below. If , then by the construction of , we have

In addition, by (10), we have

Then, by the above two inequalities, we can obtain

On the other hand, from (26), we have

By (30) and (31), for sufficiently large we obtain

Therefore, there is a positive constant , such that

for all . The proof is completed.

Theorem 10. In addition to the assumptions in Theorem 7, if conditions (25) and (26) hold, then the sequence Q-linearly converges to 0; hence the whole sequence converges to R-linearly.

Proof. Let be the closest solution to . That is, . By (18), we have

For sufficiently large , it follows from (10) and (26) that

Thus, from (6), (10), (25), and (33), for sufficiently large , it holds that

Substituting the above two inequalities into (34), we have

which implies that the sequence -linearly converges to 0. Therefore, the whole sequence converges to -linearly. The proof is completed.

4. Numerical Results

In this section, we test Algorithm 1 and compared it with the projection method in [7] and the spectral gradient projection method in [9]. We give the following two simple problems to test the efficiency of the three methods.

Problem 11. The mapping is taken as , where

and . Obviously, this problem has a unique solution .

Problem 12. The mapping is taken as , where

and . Obviously, Problem 12 is nonsmooth at .

The codes are written in Mablab7.0 and run on a personal computer with 2.0 GHZ CPU processor. The parameters used in Algorithm 1 are set as , , , and . The initial steplength in Step  2 of Algorithm 1 is set to be the spectral coefficient

where and . By the monotonicity and the Lipschitz continuity of , it is not difficult to show that

where is the Lipschitz constant. If , we replace the spectral coefficient by

where and . This parabolic model is the same as the one described in [15]. We stop the iteration if the iteration number exceeds 1000 or the inequality is satisfied. The method in [7] (denoted by WPM) is implemented with the following parameters: , , , , and . The method in [9] (denoted by YSGP) is implemented with the following parameters: , , and .

For Problem 11, the initial point is set as , and Table 1 gives the numerical results by Algorithm 1, WPM, and YSGP with different dimensions, where Iter. denotes the iteration number, Fn denotes the number of function evaluations, and CPU denotes the CPU time in seconds when the algorithm terminates. Table 2 lists the numerical results of Problem 12 with different initial points. The numerical results given in Tables 1 and 2 show that Algorithm 1 performs a little better than YSGP in [9] and obviously better than WPM in [7], since it requires much lower number of iterations or less CPU time than WPM in [7] and a little lower number of iterations or less CPU time than YSGP in [9]. So the proposed method is promising.

Table 1: Numerical results with different dimensions of Problem 11.

Dimension
Method
Iter.
Fn
CPU

50
Algorithm 1
1
5
0.0000

WPM
614
3703
0.5313

YSGP
6
15
0.0000

500
Algorithm 1
1
5
0.0156

WPM
657
3959
1.1406

YSGP
6
15
0.0156

5000
Algorithm 1
1
5
0.4219

WPM
692
4157
8.4063

YSGP
7
17
0.4375

50000
Algorithm 1
1
5
52.4688

WPM
724
4336
148.9219

YSGP
7
17
53.6719

Table 2: Numerical results with different initial points of Problem 12 with .

Initial point
Method
Iter.
Fn
CPU

Algorithm 1
10
115
0.3438

WPM
330
1981
13.1094

YSGP
16
49
0.6094

Algorithm 1
9
91
0.2031

WPM
331
1985
11.7969

YSGP
16
47
0.6875

Algorithm 1
7
88
0.2031

WPM
331
1987
13.5781

YSGP
16
47
0.6250

Algorithm 1
11
117
0.2500

WPM
331
1987
11.5469

YSGP
16
48
0.6094

Algorithm 1
9
79
0.3281

WPM
331
1987
11.2031

YSGP
16
48
0.6719

5. Conclusions

A globally convergent matrix-free method to solve constrained equations has been developed, which is not only derivative-free but also completely matrix-free. Consequently, it can be applied to solve large-scale nonsmooth constrained equations. We established the global convergence without the requirement of differentiability of the equations and presented the linear convergence rate under standard conditions. We also report some numerical results to show the efficiency of the proposed method.

Numerical results indicate that the parameters and influence the performance of the method, so the choice of the positive constants and is our future work.

Conflict of Interests

All the authors of the paper declare that they do not have any conflict of interests, and there are no financial or personal relationships with other people or organizations that can inappropriately influence our work in this paper.

Acknowledgments

The authors would like to thank the referees for giving them many valuable suggestions and comments, which improve this paper greatly. This work is supported by the Nature Science Foundation of Shandong Province (ZR2012AL08).

References

E. Zeidler, Nonlinear Functional Analysis and Its Applications, Springer, 1990.

A. J. Wood and B. F. Wollenberg, Power Generation, Operation, and Control, John Wiley and Sons, New York, NY, USA, 1996.

K. Meintjes and A. P. Morgan, “A methodology for solving chemical equilibrium systems,” Applied Mathematics and Computation, vol. 22, no. 4, pp. 333–361, 1987.

L. Qi, X. J. Tong, and D. H. Li, “Active-set projected trust-region algorithm for box-constrained nonsmooth equations,” Journal of Optimization Theory and Applications, vol. 120, no. 3, pp. 601–625, 2004.

M. Ulbrich, “Nonmonotone trust-region methods for bound-constrained semismooth equations with applications to nonlinear mixed complementarity problems,” SIAM Journal on Optimization, vol. 11, no. 4, pp. 889–917, 2001.

J. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, NY, USA, 1970.

C. Wang, Y. Wang, and C. Xu, “A projection method for a system of nonlinear monotone equations with convex constraints,” Mathematical Methods of Operations Research, vol. 66, no. 1, pp. 33–46, 2007.

F. Ma and C. Wang, “Modified projection method for solving a system of monotone equations with convex constraints,” Journal of Applied Mathematics and Computing, vol. 34, no. 1-2, pp. 47–56, 2010.

Z. Yu, J. Lin, J. Sun, Y. Xiao, L. Liu, and Z. Li, “Spectral gradient projection method for monotone nonlinear equations with convex constraints,” Applied Numerical Mathematics, vol. 59, no. 10, pp. 2416–2423, 2009.

L. Zheng, “A new projection algorithm for solving a system of nonlinear equations with convex constraints,” Bulletin of the Korean Mathematical Society, vol. 50, no. 3, pp. 823–832, 2013.

W. La Cruz and M. Raydan, “Nonmonotone spectral methods for large-scale nonlinear systems,” Optimization Methods & Software, vol. 18, no. 5, pp. 583–599, 2003.

L. Zhang and W. Zhou, “Spectral gradient projection method for solving nonlinear monotone equations,” Journal of Computational and Applied Mathematics, vol. 196, no. 2, pp. 478–484, 2006.

W. Cheng, “A PRP type method for systems of monotone equations,” Mathematical and Computer Modelling, vol. 50, no. 1-2, pp. 15–20, 2009.

L. Zhang, W. Zhou, and D.-H. Li, “A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence,” IMA Journal of Numerical Analysis, vol. 26, no. 4, pp. 629–640, 2006.

W. La Cruz, J. M. Martínez, and M. Raydan, “Spectral residual method without gradient information for solving large-scale nonlinear systems of equations,” Mathematics of Computation, vol. 75, no. 255, pp. 1429–1448, 2006.