New convergence properties of the proximal augmented Lagrangian method is established for continuous nonconvex optimization problem with both equality and inequality constrains. In particular, the multiplier sequences are not required to be bounded. Different convergence results are discussed dependent on whether the iterative sequence

In this paper, we consider the following nonlinear programming problem:

Augmented Lagrangian algorithms are very popular tools for solving nonlinear programming problems. At each outer iteration of these methods, a simpler optimization problem is solved, for which efficient algorithms can be used, especially when the problems are large. The most famous augmented Lagrangian algorithm based on the Powell-Hestenes-Rockafellar [

An indispensable assumption in the most existing global convergence analysis for augmented Lagrangian methods is that the multiplier sequence generated by the algorithms is bounded. This restrictive assumption confines applications of augmented Lagrangian methods in many practical situation. The important work on this direction includes [

In this paper, for the optimization problem (

This paper is organized as follows. In Section

The primal augmented Lagrangian function for (

Given

If

Recall that a vector

The multiplier algorithm based on the primal augmented Lagrangian

Select an initial point

Compute

Find

If

The iterative formula for

Let

This follows immediately from (

For establishing the convergence property of Algorithm

The following result shows that the perturbation value function is upper semicontinuous at zero.

The perturbation function

Since

Let

For any given

Let

We prove this result by the way of contradiction. Suppose that we can find an

Since

There exist an index

Using Lemma

There exist an index

Let

For an arbitrarily

With these preparation, the global convergence property of Algorithm

Let

According to the construction of Algorithm

The foregoing result is applicable to the case when

Let

We first show the sufficiency. According to the proof of Theorem

Note that in many practical cases, the set

A point

Suppose that

Noting that

Either

If

Both

We claim that

If

To give some insight into the behavior of our proposed algorithm presented in this paper, we solve the following nonlinar programming problems. The test was done at a PC of Pentium 4 with 2.8 GHz CPU and 1.99 GB memory, and the computer codes were written in MATLAB 7.0. Numerical results are reported in Tables

Result of Example

1 | 1.0000 | 6.2500 | (0.2424, 0.2424, 0.4661) | 0.3348 |

3 | 0.1667 | 40.0541 | (0.2299, 0.2299, 0.4438) | 0.3027 |

Result of Example

1 | 1 | 6.2500 | (0.1643, 0.1643, 1.0000) | 0.1581 |

3 | 1.667 | 42.6030 | (0.1350, 0.1350, 1.0000) | 0.1076 |

Result of Example

1 | 1 | 576 | (3.1094, 3.5073, −1.4777) | 957.2349 |

2 | 0.5000 | 1.9679 | (2.9094, 3.606, −1.8787) | 956.9680 |

Result of Example

1 | 1 | 3025 | (0.7101, 2.8042, 1.5490) | 978.2781 |

2 | 0.5000 | 8.4469 | (0.7208, 2.8092, 1.5579) | 978.1236 |

It holds that

Consider

It holds that

Consider

Augmented Lagrangian methods are useful tools for solving many practical nonconvex optimization problems. In this paper, new convergence property of proximal augmented Lagrangian algorithm is established without requiring the boundedness of multiplier sequences. It is proved that if the algorithm terminates in finite steps, then we obtain a KKT point of the primal problem; otherwise, the iterative sequence

The authors would like to thank the referees for their valuable comments, which greatly improved the presentation of the paper. Research of the first author was partly supported by the National Natural Science Foundation of China (11101248, 11026047) and Shandong Province Natural Science Foundation (ZR2010AQ026). Research of the second author was partly supported by the National Natural Science Foundation of China (11171247). Research of the forth author was partly supported by the National Natural Science Foundation of China (10971118).