Journal of Applied Mathematics

Volume 2013 (2013), Article ID 976509, 9 pages

http://dx.doi.org/10.1155/2013/976509

Research Article

Global Convergence of a New Nonmonotone Filter Method for Equality Constrained Optimization

Ke Su, Wei Liu, and Xiaoli Lu

College of Mathematics and Computer Science, Hebei University, Baoding 071002, China

Received 6 December 2012; Accepted 5 March 2013

Academic Editor: Hadi Nasseri

Copyright © 2013 Ke Su et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

A new nonmonotone filter trust region method is introduced for solving optimization problems with equality constraints. This method directly uses the dominated area of the filter as an acceptability criterion for trial points and allows the dominated area decreasing nonmonotonically. Compared with the filter-type method, our method has more flexible criteria and can avoid Maratos effect in a certain degree. Under reasonable assumptions, we prove that the given algorithm is globally convergent to a first order stationary point for all possible choices of the starting point. Numerical tests are presented to show the effectiveness of the proposed algorithm.

1. Introduction

We analyze an algorithm for solving optimization problems where a smooth objective function is to be minimized subject to smooth nonlinear equality constraints. More formally, we consider the problem,

where , the functions and are all twice continuously differentiable. For convenience, let and , and refers to , to , to and to , and so forth.

There are many trust region methods for equality constrained nonlinear programming , for example, Byrd et al. [1], Dennis Jr. et al. [2] and Powell and Yuan [3], but in these works, a penalty or augmented Lagrange function is always used to test the acceptability of the iterates. However, there are several difficulties associated with the use of penalty function, and in particular the choice of the penalty parameter. Hence, in 2002, Fletcher and Leyffer [4] proposed a class of filter method, which does not require any penalty parameter and has promising numerical results. Consequently, filter technique has been employed to many approaches, for instance, SLP methods [5], SQP methods [6–8], interior point approaches [9], bundle techniques [10], and so on.

Filter technique, in fact, exhibits a certain degree of nonmonotonicity. The nonmonotone technique was proposed by Grippo et al. in 1986 [11] and combined with many other methods. M. Ulbrich and S. Ulbrich [12] proposed a class of penalty-function-free nonmonotone trust region methods for nonlinear equality constrained optimization without filter technique. Su and Pu [13] introduced a nonmonotone trust region method which used the nonmonotone technique in the traditional filter criteria. Su and Yu [14] presented a nonmonotone method without penalty function or filter. Gould and Toint [15] directly used the dominated area of the filter as an acceptability criteria for trial points and obtained the global convergence properties. We refer the reader [16–18] for some works about this issue.

Motivated by the ideas and methods above, we propose a modified nonmonotone filter trust region method for solving problem . Similar to the Byrd-Omojokun class of algorithms, each step is decomposed into the sum of two distinct components, a quasi-normal step and a tangential step. The main contribution of our paper is to employ the nonmonotone idea to the dominated area of the filter so that the new and more flexible criteria is given, which is different from that of Gould and Toint [15] and Su and Pu [13]. Under usual assumptions, we prove that the given algorithm is globally convergent to first order stationary points.

This paper is organized as follows. In Section 2, we introduce the fraction of Cauchy decrease and the composite SQP step. The new nonmonotone filter technique is given in Section 3. In Section 4, we propose the new nonmonotone filter method and present the global convergence properties in Section 5. Some numerical results are reported in the last section.

2. The Fraction of Cauchy Decrease and the Composite SQP Step

Consider the following unconstraint minimization optimization problem:

where is a continuously differentiable function. A trust region algorithm for solving the above problem is an iterate procedure that computes a trial step as an approximate solution to the following subproblems:

where is the Hessian matrix or an approximate to it and is a given trust region radius.

To assure the global convergence, the step is required only to satisfy a fraction of Cauchy decrease condition. This means that must predict via the quadratic model function at least as much as a fraction of the decreased given by the Cauchy step on ; that is, there exists a constant fixed across all iterations, such that

where is the steepest descent step for inside the trust region.

Lemma 1. If the trial step satisfies a fraction of Cauchy decrease condition, then

Proof (see Powell [19] for the proof). Now, we turn to explain the composite SQP step. Given an approximate estimate of the solution at th iteration, following Dennis Jr. et al. [2] and M. Ulbrich and S. Ulbrich [12], we obtain the trial step by computing a quasi-normal step and a tangential step . The purpose of the quasi-normal step is to improve feasibility. To improve optimality, we seek in the tangential space of the linearized constraints in such a way that it provides sufficient decrease for a quadratic model of the objective function . Let , where is a symmetric approximation of .

is the solution to the subproblem

where is a trust region radius and . In order to improve the value of the objective function, we solve the following subproblem to get

Then we get the current trial step . Let , where and denote a matrix whose columns form a basis of the null space of . We refer to [2] for a more detailed discussion of this issue.

In usual way that impose a trust region in step-decomposition methods, the quasi-normal step and the tangential step are required to satisfy

where . Here, to simplify the proof, we only impose a trust region on and , which is natural.

Note that is the reduced gradient of in terms of the representation of the tangential step:

Define

Then the first order necessary optimality conditions (Karush-Kuhn-Tucker or KKT conditions) at a local solution of problem can be written as

3. A New Nonmonotone Filter Technique

In filter method, originally proposed by Fletcher and Leyffer [4], the acceptability of iterates is determined by comparing the value of constraint violation and the objective function with previous iterates collected in a filter. Define the violation function by , it is easy to see that if and only if is a feasible point, so a trial point should reduce either the value of constraint violation or that of the objective function .

In the process of the algorithm, we need to decide whether the trial point is any better than as an approximate solution to the problem . If we decide that this is the case, we say that the iteration is successful and choose as the next iterate. Let us denote by the set of all successful iterations, that is,

In traditional filter method, a point is called acceptable to the filter if and only if

where , denotes the filter set. Define

A trial point is accepted if and only if .

Now, similar to the idea of Gould and Toint [15], we give a new modified nonmonotone filter technique. For any -pair, define an area that represents its contribution to the area of , we hope this contribution is positive; that is, the area of is increasing. For convenience, we partition the right half-plane into four different regions (see Figure 1). Define to be the complement of . Let

Figure 1

These four parts are

(1)the dominated part of the filter: ;(2)the undominated part of lower left corner of the half plane:

(3)the undominated upper left corner: ;(4)the undominated lower right corner: .

Consider the trial point , if the filter is empty, then define its contribution to the area of the filter by

where is a constant. If the filter is not empty, then define the contribution of to the area of the filter by four different formulae.

If , assume

If , assume

If , assume

If , assume

where , and .

Figure 2 illustrate the corresponding areas in the filter. Horizontally dashed surfaces indicate a positive contribution and vertically dashed ones a negative contribution. Note that is a continuous function of .

Figure 2

Next, we should consider the updating of the filter. If , then

If , then

We now return to the question of deciding whether a trial point is acceptable for the filter or not. We will insist that this is a necessary condition for the iteration to be successful in the sense that . If we consider an iterate , there must exist a predecessor iteration such that . Under the monotonic situation, a trial point would be accepted whenever it results in an sufficient increase in the dominated area of the filter, that means would be accepted whenever

where , is a constant. Under the nonmonotonic situation, we relax condition (23) to be

where , is some reference iteration for , , , .

Compared to condition (2.21) [15], our condition (24) is more flexible if is negative.

According to condition (24), it is possible to accept even though it may be dominated. Then will be accepted if either (23) or (24) holds.

4. The New Nonmonotone Filter Trust Region Algorithm

Our algorithm is based on the usual trust region technique; define the predict reduction for the function to be

and the actual reduction

Moreover, let , if there exists a nonzero constant such that and condition (23) and (24) hold, the trial point will be called acceptable. Then the next trial point is obtained, and for its feasibility, we consider the condition

is true or not, where and are all positive constants, if it is not true, then turn to the feasibility restoration phase and define

A formal description of the algorithm is given as follows.

Algorithm A . Choose an initial point , a symmetric matrix , let , , compute , let .

Step 1. Compute .

Step 2. If , stop.

Step 3. Solve the subproblems (5) and (6) to get the quasi-normal step and the tangential step . Let .

Step 4. If , let , then go to Step 8.

Step 5. If is not acceptable to the filter, go to Step 8. Otherwise and update the filter according to (21) and (22), the trust region radius and , then get the corresponding .

Step 6. If , and go to Step 1, otherwise go to Step 7.

Step 7. By restoration Algorithm B to get , then the trial point .

Step 8. Update the trust region radius by Algorithm C, let and go to Step 3.

 We aim to reduce the value of in the restoration Algorithm B, that is to get by Newton-type method.

Algorithm B . Let .

Step 1. If and is acceptable by the filter, then , stop.

Step 2. Compute

to get , then compute .

Step 3. If , and go to Step 2.

Step 4. If , , compute and go to Step 1, where .

Algorithm C (updating the trust region radius). Given , we have the following.

(1)If or is not acceptable to the filter, .(2)If is acceptable to the filter but does not satisfiy condition (27), .(3)If is acceptable to the filter and satisfies (27),  .

From the description above and the idea of the algorithm, we can see that our algorithm is more flexible. Every successful iterate must be any better than the predecessor one in some degree according to the traditional filter method. But our algorithm relaxes this demand by using the nonmonotone technique and also avoids Maratos effect in a certain degree. Moreover, Algorithm C allows a relatively wide choice of the trust region.

5. The Convergence Properties

In this section, to present a proof of global convergence of algorithm, we always assume that the following conditions hold.

Assumption

(A1) The objective function and the constraint functions are twice continuously differentiable. (A2) For all ,  , and all remain in a closed, bounded convex subset . (A3) The matrix is nonsingular matrix for all . (A4) The matrices are uniformly bounded in , where denotes a matrix whose columns form a basis of the null space of . (A5) The matrix is uniformly bounded.

By the assumptions, we can suppose there exist constants such that .

By (A1) and (A2), it holds

where , hence in the -plane, the -pair lies in the area .

From (A1), (A2), and (A3), it exists a constant such that

Lemma 2. At the current iterate , let the trial point component actually be normal to the tangential space. Under the problem assumptions, there exists a constant independent of the iterates such that

Proof. It is similar to the proof of Lemma 2 in [13].

Lemma 3. Under Assumptions, there exist positive constants independent of the iterates such that

Proof. The proof is an application of Lemma 1 to the two subproblems (5) and (6).

Lemma 4. Suppose that Assumptions hold, then restoration Algorithm B is well defined.

Proof. The conclusion is obvious, if . Otherwise it exists such that for all , it holds . Consider the set

where . By Lemma 3 and the definition of , we have

By , it holds for . From Algorithm B, we can obtain that for all .

On the other side,

for . By the algorithm, the radius should be satisfied , that is contradicted to . The proof is complete.

Now, we analyze the impact of the criteria (23) and (24). Once a trial point is accepted as a new iterate, it must be provided some improvement, and we formalize this by saying that iterate improves on iterate . That is the trial point is accepted at iterate ; it happens under two situations, one is by the criteria (23), that is,

the other is by the criteria (24), that is,

Now consider any iterate , it improved on , which was itself accepted because it improved on , and so on, until back to . Hence we may construct a chain of successful iterations indexed by =  for each , such that

where is the smallest index in the chain of successful iterations.

Lemma 5. Suppose that Assumptions hold and Algorithm A does not terminate finitely, apply Algorithm A to the problem , then for all and , it holds

Proof. For , if , by (24),  , then

If ,

If ,

If ,  . By , it holds

Then from (23), . It implies (43). Moreover

Together with (42) and (43), the result follows.

Lemma 6. Suppose that Assumptions hold. If Algorithm A does not terminate finitely and the filter contains infinite iterates, then .

Proof. Suppose by contradiction that there exists a constant and infinite sequence such that for all . Because there are infinite iterations in the filter, we have , then for .

Then by (31), is upper bounded for each . That means it exists such that , so . Hence must be finite, it contradicts to the infinity of . The proof is complete.

Lemma 7. Suppose that Assumptions hold and Algorithm A terminate finitely, then .

Proof. From the Algorithm A and the definition of filter, the conclusion follows.

Lemma 8. For any trial point , there must be one accepted by the filter.

Lemma 9. Suppose that Assumptions hold, there exists independent of the iterates such that

Proof. By (32), the assumptions and , it is obvious that

The proof is complete.

Lemma 10. Suppose that Assumptions hold and , if

one can deduce

Proof. By the assumptions and the definition of , it holds

From Lemma 3, . Together with

then . It is the conclusion.

Lemma 11. Suppose the conditions of Lemma 10 hold, if

then .

Proof. From the definition of and Lemma 10, together with (31), we have

It is obvious that .

Lemma 12. Suppose the conditions of Lemmas 10 and 11 hold, if

then

Proof. By Lemmas 3, 10, and 11, together with , it holds

From the Algorithm, , then

Hence .

Theorem 13. Suppose the assumptions hold, there must exist such that for each , it holds

Proof. Let be large enough such that , it is true by Lemmas 6 and 7. Suppose by contradiction that the index is the first one after , which satisfies

where is the smallest value of violation function in filter. Then . By the above analysis, we know , that is . From the Algorithm and (60), it concludes

By (60) and (61), (53) can be obtained. In Lemma 11, let instead of , it deduces

Based on Lemma 12, together with (60), (61), and the algorithm, we can see

It can be seen that (53) is true for , with (55), we can deduce

That means can be accepted by the filter. From above and (55), we know . Hence the index is not the first one after which satisfied (60), that is a contradiction. So, for any , it holds . Define

we can see that

holds for each . The proof is complete.

Lemma 14. Suppose that Assumptions hold and Algorithm A does not terminate finitely, then .

Proof. Suppose by contradiction that for , there exists a constant such that .

By Assumption (A3) and (A4), . From Lemma 6, we know . Hence there exists such that

for . Then for .

It is obvious that

By the proof of Lemma 9, it holds . Together with Lemma 6 and the definition of , we have

By the Algorithm, we can get

Then

By (68), (69), and (71), it deduces

Based on the assumptions, Lemma 3 and Theorem 13, for , it holds

which contradicts (72). The conclusion follows.

Theorem 15. Suppose the assumptions hold, and apply the algorithm to problem , then

where ,   denotes a matrix whose columns form a basis of the null space of .

Proof. If the algorithm terminates finitely, it is obvious that it holds. Otherwise, by Lemmas 6 and 14, the conclusion also can be obtained.

Theorem 16. Suppose the assumptions hold, and is the infinite sequence obtained by the algorithm, then there must exist a subsequence such that

and satisfies the one order KKT condition of .

Proof. By Assumption (A1), there exist a subsequence and , such that . Together with Assumption (A3) and (A4), it hods , which means for large enough , lies in the space spaned by the columns of . That is there exists such that

The conclusion follows.

6. Some Numerical Experiments

(1)Updating of is done by , where

and , is the multipluser of corresponding quadratic subproblems.(2)We assume the error toleration is .(3)The algorithm parameters were set as follows: , , , , , , , , , , , , . The program is written in Matlab.

The numerical results for the test problems are listed in Table 1.

Table 1

Problem

NF
NG
L's NF
L's NG

HS6
2
1
11
11
20
12

HS7
2
1
9
3
15
11

HS8
2
2
7
4
—
—

HS9
2
1
6
6
6
6

HS26
3
1
24
24
42
24

HS39
4
2
15
9
26
19

HS40
4
3
7
5
7
5

HS42
4
2
8
8
11
11

HS78
5
3
6
6
8
7

In Table 1, the problems are numbered in the same way as in Schittkowski [20] and Hock and Schittkowski [21]. For example, “S216” is the problem in Schittkowski [20] and “HS6” is the problem in Hock and Schittkowski [21]. NF, NG represent the numbers of function and gradient calculations and “L's” is the solution in [22]. The numerical results show that the our algorithm is more effective than the L's for most test examples. Moreover, the higher the level of nonmonotonic, the better the numerical results. The results show that the new algorithm is robust and effective, and more flexible for the acceptance of the the trial iterate.

Acknowledgments

This research is supported by the National Natural Science Foundation of China (no. 11101115) and the Natural Science Foundation of Hebei Province (nos. A2010000191, Y2012021).

References

R. H. Byrd, R. B. Schnabel, and G. A. Shultz, “A trust region algorithm for nonlinearly constrained optimization,” SIAM Journal on Numerical Analysis, vol. 24, no. 5, pp. 1152–1170, 1987.

J. E. Dennis Jr., M. El-Alem, and M. C. Maciel, “A global convergence theory for general trust-region-based algorithms for equality constrained optimization,” SIAM Journal on Optimization, vol. 7, no. 1, pp. 177–207, 1997.

M. J. D. Powell and Y. Yuan, “A trust region algorithm for equality constrained optimization,” Mathematical Programming A, vol. 49, no. 2, pp. 189–211, 1991.

R. Fletcher and S. Leyffer, “Nonlinear programming without a penalty function,” Mathematical Programming A, vol. 91, no. 2, pp. 239–269, 2002.

C. M. Chin and R. Fletcher, “On the global convergence of an SLP-filter algorithm that takes EQP steps,” Mathematical Programming A, vol. 96, no. 1, pp. 161–177, 2003.

R. Fletcher, N. I. M. Gould, S. Leyffer, P. L. Toint, and A. Wächter, “Global convergence of a trust-region SQP-filter algorithm for general nonlinear programming,” SIAM Journal on Optimization, vol. 13, no. 3, pp. 635–659, 2002.

R. Fletcher, S. Leyffer, and P. L. Toint, “On the global convergence of a filter-SQP algorithm,” SIAM Journal on Optimization, vol. 13, no. 1, pp. 44–59, 2002.

S. Ulbrich, “On the superlinear local convergence of a filter-SQP method,” Mathematical Programming B, vol. 100, no. 1, pp. 217–245, 2004.

M. Ulbrich, S. Ulbrich, and L. N. Vicente, “A globally convergent primal-dual interior-point filter method for nonlinear programming,” Mathematical Programming A, vol. 100, no. 2, pp. 379–410, 2004.

R. Fletcher and S. Leyffer, “A bundle filter method for nonsmooth nonlinear optimization,” Tech. Rep. NA/195, Department of Mathematics, University of Dundee, Dundee, Scotland, December 1999.

L. Grippo, F. Lampariello, and S. Lucidi, “A nonmonotone line search technique for Newton's method,” SIAM Journal on Numerical Analysis, vol. 23, no. 4, pp. 707–716, 1986.

M. Ulbrich and S. Ulbrich, “Non-monotone trust region methods for nonlinear equality constrained optimization without a penalty function,” Mathematical Programming B, vol. 95, no. 1, pp. 103–135, 2003.

K. Su and D. Pu, “A nonmonotone filter trust region method for nonlinear constrained optimization,” Journal of Computational and Applied Mathematics, vol. 223, no. 1, pp. 230–239, 2009.

K. Su and Z. Yu, “A modified SQP method with nonmonotone technique and its global convergence,” Computers & Mathematics with Applications, vol. 57, no. 2, pp. 240–247, 2009.

N. I. M. Gould and P. L. Toint, “Global convergence of a non-monotone trust-region filter algorithm for nonlinear programming,” in Multiscale optimization methods and applications, vol. 82, pp. 125–150, Springer, New York, NY, USA, 2006.

Z. Chen and X. Zhang, “A nonmonotone trust-region algorithm with nonmonotone penalty parameters for constrained optimization,” Journal of Computational and Applied Mathematics, vol. 172, no. 1, pp. 7–39, 2004.

Z. W. Chen, “A penalty-free-type nonmonotone trust-region method for nonlinear constrained optimization,” Applied Mathematics and Computation, vol. 173, no. 2, pp. 1014–1046, 2006.

F. A. M. Gomes, M. C. Maciel, and J. M. Martínez, “Nonlinear programming algorithms using trust regions and augmented Lagrangians with nonmonotone penalty parameters,” Mathematical Programming A, vol. 84, no. 1, pp. 161–200, 1999.

M. J. D. Powell, “Convergence properties of a class of minimization algorithms,” in Nonlinear Programming, pp. 1–27, Academic Press, New York, NY, USA, 1974.

K. Schittkowski, More Test Examples for Nonlinear Programming Codes, vol. 282, Springer, Berlin, Germany, 1987.

W. Hock and K. Schittkowski, Test Examples for Nonlinear Programming Codes, vol. 187, Springer, Berlin, Germany, 1981.

M. Lalee, J. Nocedal, and T. Plantenga, “On the implementation of an algorithm for large-scale equality constrained optimization,” SIAM Journal on Optimization, vol. 8, no. 3, pp. 682–706, 1998.