Filter Trust Region Method for Nonlinear Semi-Infinite Programming Problem

We present a filter trust region method for nonlinear semi-infinite programming. Based on the discretization technique and motivated by the multiobjective programming, we transform the semi-infinite problem into a finite one. Together with the filter technique, we propose a modified method that avoids the merit function. Compared with the existing methods, our method is more flexible and easier to implement. Under somemild conditions, the convergent properties are proved.Moreover, the numerical results are reported in the end.

For convenience, consider only the case of Ω = [, ];  and  are real numbers.
The SIP problem arises in various applications such as approximation theory, optimal control, eigenvalue computation, mechanical stress of materials, pollution control, and statistical design.The research of SIP can be traced back to linear semi-infinite programming [1] and it was also proposed in researching Fritz-John conditions [2].Then the SIP was formally proposed by Charnes, Coopper, and Kortanek in 1962 [3].It is difficult to solve SIP because the constraints in SIP are infinite.Numerical methods for solving SIP may be divided into discretization methods and continuous methods.Discretization methods are based on nonlinear programming problems which are obtained by discretization of the original problem and incorporate some grid-refinement strategy [4,5].Our new method might be useful in the discretization context for semi-infinite programming since it drastically decreases the number of constraints.
For continuous methods, Hettich and Honstede present an iterative method which attempts to find a solution satisfying optimality conditions of the original problem.However, this method is only locally convergent and is very restrictive for practical use [6].Jian et al. present a Norm-Relaxed Method of Feasible Directions algorithm, in order to obtain global convergence [7].Coope and Watson have proposed a Sequential Quadratic Programming (SQP) method that utilizes an exact  1 penalty function and global convergence obtained [8].Then, a globally convergent SQP method for SIP is proposed which utilizes an exact  ∞ penalty function and trust region methods [9].For optimization problems with smooth inequality constraints, the penalty function method is, in general, recognized as an efficient method.In [10,11], in order to solve SIP, the smooth approximate functions in integral form are appended to the objective function by using the concept of the penalty function, but the error caused by taking the smooth approximation of the continuous inequality constraints cannot be avoided.Moreover, it is difficult to find a suitable penalty parameter.If the penalty parameter is too large, then any monotonic method would be forced to follow the nonlinear constraint manifold very closely, resulting in much shortened Newton steps and slow convergence.
To avoid using the above drawbacks, Fletcher and Leyyfer proposed a filter method in 2002 [12].In their method, instead of combining the objective and constraint violation into a single function, they view the original problem as a biobjective optimization problem that minimizes objective function and constraint violation function.Consequently, filter technique has been employed in many approaches, for instance, SQP methods [13], interior point approaches [14], bundle techniques [15], and SIP [16].But, in [16], the search direction is obtained in a linear equation.The structure of filter is complex.
Motivated by the above ideas, we propose a filter trust region algorithm.We transform the SIP into a finite problem based on a modified discretion method.Then, the search direction is obtained by the modified trust region quadratic subproblem.Moreover, without the merit function, we adopt the filter technique to decide whether a new iteration point is acceptable or not.Compared with the existing methods, there are two main advantages in our presented algorithm: (1) the modified quadratic subproblem is always feasible; (2) unlike the continuous method, the penalty parameter is avoided in our presented method, and actually the algorithm has a certain nonmonotonicity property.Under some wild conditions, the convergent properties of the proposed method are proved.The numerical results show the effectiveness of our approach.
The remainder of this paper is organized as follows.The algorithm is described in Section 2. In Section 3, we analyze the convergent properties of proposed algorithm.Numerical results are reported in the final section.

Description of Algorithm
For solving SIP, discretization of the original problem SIP, consider the following problem: where In order to improve the degree of accuracy of optimal solution of SIP, we observed that trust region method is exceedingly important for ensuring global convergence while retaining fast local convergence in optimization algorithms, so the trust region condition is added to the quadratic subproblem of (2).
For the current k-th iteration,   is obtained by the following modified trust region quadratic subproblem (QP): where  0 ,   > 0 are positive constants,   is a symmetric positive definite matrix, and Δ  ≥ 0 is the k-th trust region radius. Denote Define the actual reduction of () as  (  ) −  (  +   ) . ( The predict reduction of () as Here  () (  ,   ) =  0   + (1/2)       .Define The parameter   is used to decide whether the predicted reduction of () is close to the actual reduction of () or not.
In the proposed algorithm, the filter technique is used to decide whether a trial point is acceptable or not.To ensure the convergence, some additional conditions are required to decide whether to accept a trial point to the filter or not.The traditional acceptable criterion is as follows.Definition 3. A trial point  is called acceptable to the filter F if and only if either To avoid convergence to infeasible limit points, we add an envelope around the current filter.So criterion (8) can be changed to the following criterion.
where 0 <  <  < 1 are constants.In practice,  is close to 1 and  is close to 0.
Suppose that QP has local optimal solution (  ,   ); i.e., (  ,   ) is a Karush-Kuhn-Tucker (KKT) point of QP.So there exists multiplier   ∈   and    ∈  +1 ,  ∈ Ω  , satisfy: ‖‖ ≤ △  , (15) Similarly, if   is a KKT point for problem   , then there exists a multiplier vector   ∈   and   ∈  +1 such that the following formulas hold: The following lemma shows that our algorithm is well defined.
In addition, the subproblem (QP) is equal to the following constrained optimization: min Obviously, the first term of the objective function is strictly convex about variable   and the second term is convex about variable   ; thus the objective function of the problem above is strictly convex [18].Besides, the constrained function of (31) is a convex function.Combining the convexity of   , we obtain problem (31) which is a convex programming and its optimal solution   is unique.Therefore, the optimal solution of subproblem (QP) is unique.
(ii) If (  ,   ) is a KKT point for subproblem (QP), then it is an optimal solution of subproblem (QP) since subproblem (QP) is a convex programming.Conversely, if (  ,   ) is an optimal solution of subproblem (QP), note that Abadie's Constraint Qualification [19] holds automatically since the constraints of subproblem (QP) are linear; then (  ,   ) is a KKT point for subproblem (QP).The proof is completed.
Proof.In view of the definition of  and   ∉ , it follows that Ω +  ̸ = Ø.Now we prove by contradiction that the first conclusion of Lemma 7 holds.Suppose that there exists a vector  ∈   such that ∇   (, )   < 0, ∀ ∈ Ω 0 () . ( the following result holds if the positive parameter  is small enough Obviously, for  > 0 small enough, (, ) is a feasible solution of (3); furthermore,  0  + (1/2)()    () < 0. Therefore Proof.Suppose that   = 0, we can get   = 0,   ∈ ,   = 0 by Lemma 6, so Ω +  (  ) = Ø, Ω −  (  ) = Ω  , and the KKT conditions ( 10)-( 20) can be rewritten as In view of the formula  0 =    0 + ∑ ∈Ω       and the nonnegative properties of multipliers    ,   and parameter  0 > 0, we can obtain the boundedness of sequences {  } and {   }.We prove that   ̸ = 0.By contradiction, suppose that   = 0, then in view of the equation  0 =    0 + ∑ ∈Ω       one can get    ̸ = 0; i.e., (  , ) < 0, one knows that Choosing a vector  * satisfying MFCQ at the point   and multiplying the equality above and the constraints of subproblem for  ∈ Ω 0 () by  * , we obtain This is a contradiction and the conclusion that   ̸ = 0 is true.Let and then the KKT condition of   holds at   with the multiplier vector   , so   is a KKT point for   .Now, to prove the necessity of theorem, note that if   is a KKT point for   , namely, (37)-(41) holds, then (0, 0) satisfies KKT conditions (37)-(41) of   with multipliers From Lemma 5 we obtain the uniqueness of the KKT point.So the uniqueness of the KKT point shows that   = 0.The whole proof is completed.
From Lemma 10,   = 0 can be used as terminating condition.

Lemma 11. FTR algorithm cannot cycle infinitely many times among the inner cycle (Step 1󳨀→ Step 2󳨀→ Step 4󳨀→ Step 1).
Proof.Suppose the contrary, then it follows from the rules of FTR algorithm that one has Δ  → 0,   → 0, while   ≤  0 is maintained.Consequently, we have This indicates that we eventually have   >  0 .This is a contradiction and the desired conclusion follows.This proof is completed.

Convergent Properties
In this section, under appropriate conditions, we establish the convergence of FTR algorithm.For this purpose, the following basic assumption is necessary.
Assumptions A3.Assume the sequence {  } generated by FTR algorithm is bounded.
Assumptions A4.Assume there exists ,  > 0, and ,  ∈  + , such that To analyze the convergence property of FTR algorithm, we need the following lemma.Lemma 12. Assume Assumptions A3 and A4 hold; then, sequences {  } and {  } generated by FTR algorithm are bounded.
The following result is obtained.
Theorem 13.Assume Assumptions A3 and A4 hold,  is an infinite iterative index set; if   → 0,  ∈ , then any accumulation point of {  },  ∈ , generated by FTR algorithm is the KKT point of   .
Then, from Lemmas 9 and 10, the conclusion is obtained.
Because we use the filter technique in FTR algorithm, the following lemma is necessary.Lemma 14. Assume there are infinitely many points added to the filter.Then lim →∞ ℎ (  ) = 0. (51) Proof.If the theorem is not true, there would have infinite components in  1 , which is defined as follows: We assume that |  | ≤  for all  without loss of generality, where  is a positive constant.The following two cases are considered, respectively.(i) If min ∈ 1 {  ()} exists, let   = min ∈ 1 {  ()}, ℎ  = min ∈ 1 ℎ() = min ∈ 1 ‖ + ‖, and  + = max{0, (  , )}.Then, according to the definition of the filter, the other components, which lie behind   in the filter, satisfy Then, all the filter points, which enter the filter behind   can be covered with a square, whose area is no more than 2ℎ  .We consider the area lies to the southwest of the filter in this square.When a new point   enters the filter, the next point  +1 should lie to the southwest of the points in the filter   and the area which lies to the southwest of   in the square is smaller than that of  +1 .Therefore, we think that the area is reduced if a new point enters the filter.When a point is added to the filter, its ℎ is less than every point, which lies to the left of this point, to more than (1 − ).Its  is less than every point, which lies to the right of this point, to more than .Therefore, the area of this square, more than (1 − ) 2 , will be reduced.Thus, the area will be reduced to 0 after finite times.When the area is zero, it means that a point does not enter  1 , which is contradicted with the infiniteness of  1 .
(ii) If min ∈ 1 {  ()} does not exist, because () and (, ) are continuous and differentiable, let From the definition of inf(), there exists   such that   ≥   and   ≤   + .Then, according to the definition of the filter, the other components, which lie behind   in the filter, satisfy Using the same techniques as that in (i), the conclusion can also obtained which is contradicted with the infiniteness of  1 .So  1 is infinity; the result is gotten.Thus, the conclusion is obtained.
As for the case of finitely many points added to the filter, it is apparent that the following result is true.Lemma 15.Assume there are finitely many points added to the filter.Then ℎ(  ) = 0, where  is the number of points added to the filter.
Proof.From the terminating condition of Step 2 in FTR algorithm, one has   = 0 and also   = 0; furthermore ℎ(  ) = 0. Thus, if there are finitely many points added to the filter, the index of the last point is the number of points added to the filter.The proof is completed.
The following result is obtained.Theorem 16.Assume Assumptions A3 and A4 hold; if {  } generated by FTR algorithm is a bounded sequence, then {  } has an accumulation point satisfying the KKT conditions of   .

Numerical Results
In this section, we report our preliminary numerical test results.FTR algorithm described in Section 2 was then implemented in Matlab R2016a.We compared the performance of the FTR algorithm with [17].The following four problems were tested.Throughout the computational experiments, the parameters used in algorithm were  = 100,  = 0.95,  = 0.05,  = 10 −4 ,  0 = 0.75,  0 = 1, and   = 1.

Problem 1.
min  In Table 1, the problem is denoted by Pro; e.g., Problem 1 is denoted by Pro1, the dimension of  is denoted by n, the number of iterations is denoted by Iter, the optimal solution is denoted by  * , and the optimal value is denoted by ( * ).
In Table 1, the limited numerical results show that the proposed algorithm can give an accurate solution quickly (the iterate number is less than that in [17]), which indicates that the algorithm herein is effective.Moreover, it has been observed that the computational results are not very sensitive to the choice of the initial point and the parameter n, showing that the proposed algorithm is stable.

Conclusion
In this paper, we present a filter trust region method.Based on the discrete technique, we transform nonlinear semiinfinite programming into a finite problem.And the search direction is obtained by a modified trust region quadratic subproblem which is always feasible.Besides, we adopt the filter technique to decide whether a new iteration point is acceptable or not, so the penalty parameter which is always used in most of the existing methods is avoided.Under some wild conditions, the global convergence of the proposed method was obtained.The main advantage of the presented method is that our method is more flexible and easier to implement, the modified trust region quadratic subproblem is always feasible, and penalty parameter is avoided.The numerical results show that our presented method is effective and stable.

Definition 4 .
A trial point  is called acceptable to the filter F if and only if either
This is a contradiction to   = 0. Hence Lemma 7 follows.The proof is completed.∈Ω       and the nonnegative properties of multipliers    ,   and parameter  0 > 0, we can obtain the boundedness of sequences {  } and {   }.Assume Assumptions A1 and A2 hold and (  ,   ) is an optimal solution of subproblem (QP); then   = 0 if and only if   is a KKT point of   .

Table 1 :
[17]comparison between FTR algorithm and algorithm in[17]with same conditions.