A QP-Free Algorithm for Finite Minimax Problems

and Applied Analysis 3 guaranteed, since the properties of the nonnegativity and complementarity of multiplier vector cannot be guaranteed at the point xk with dk0 and at the accumulation points of the infinite sequence {xk} generated by the proposed algorithm. Thus, we consider computing the second direction dk by another linear system:

So, a point  is called the stationary point (see [2]) of (1), if there exists a vector  = (  ,  ∈ ) such that (4) holds, where  is called the multiplier vector.
1.1.Related Work.Some algorithms have been proposed to solve the minimax problems (1) and can be grouped into three classes.The first one is the direct nonsmooth method.The problem (1) is viewed as an unconstrained nonsmooth optimization problem, which can be solved by some nonsmooth methods, such as bundle methods, gradient sampling methods, and cutting plane methods; see, for example, [3][4][5].
Secondly, a variety of regularization approaches have been used to obtain smooth approximations to the problem (1); see, for example, [6][7][8][9][10].The main advantage of the smoothing techniques is that the minimax problems are converted into smooth unconstrained optimization problems that can be solved by a standard unconstrained minimization solver.However, when the approximation accuracy is high, the smooth approximating problems become significantly illconditioned.Hence, the unconstrained optimization solver may experience numerical difficulty and slow convergence.Consequently, the simple use of smoothing techniques is complicated by the need of trading-off accuracy of approximation against problem ill-conditioning.
It is well known that the SQP method is one of the efficient methods for solving the smooth nonlinear program due to its fast convergence rate.Jian et al. [14], Hu et al. [15], and Zhu et al. [16] present some new SQP type algorithms for unconstrained minimax problems, respectively.For an iterative point   , a new quadratic program (QP) subproblem is given by where   is a symmetric positive matrix and   is the  active constraints set.The descent direction of () can be obtained by solving (5), and the algorithms have global and superlinear convergence by introducing another correction direction.In these SQP algorithms or SQCQP method, QP subproblems or quadratically constrained quadratic programming (QCQP) subproblems with inequality constraints are also required to be solved which are computationally expensive compared with system of linear equations.In addition, the IP methods and the second class smoothing techniques do not need to solve QP and QCQP subproblems, but they need the penalty parameter or barrier parameter which is difficult to deal with and causes numerical difficulty when the penalty parameter or barrier parameter is too large.Therefore, it is necessary to construct a new algorithm without solving any QP and QCQP subproblems and using penalty parameters and barrier parameters.

Division of the Systems of Linear Equations.
In this paper, we intend to replace the QP subproblem (5) used in [14][15][16] by two systems of linear equations with the same matrix so that the computation effort per iteration is much less and propose a QP-free algorithm, which does not need any penalty parameters and barrier parameters.
For (5), in order to speed up the rate of convergence and construct the systems of linear equations conveniently, we select an index   ∈ (  ) and consider the following QP subproblem by introducing more parameters    ,  ∈   \ {  }, associated with the iterate   : Since ( 6) is a convex program with linear constraints, its optimal solution is the KKT point; that is, ( 6) is equivalent to the following KKT system with the multiplier vector (,   ,  ∈ Ĩ ): Motivated by the KKT conditions above, we present the following system of linear equations (SLE): The equation "   (  )   −  = 0" comes from   = (   ,  ∈ Ĩ ) → 0 (⇒  → 1).Obviously, the system above is equivalent to Let ( 0 ,  0 Ĩ ) be the solution of (9), and  0 is taken as the first direction of our algorithm, which is not entirely suitable as the main search direction although it is a descent direction of ().In fact, the global convergence cannot be guaranteed, since the properties of the nonnegativity and complementarity of multiplier vector cannot be guaranteed at the point   with  0 and at the accumulation points of the infinite sequence {  } generated by the proposed algorithm.Thus, we consider computing the second direction   by another linear system: where the right-hand parameters V   ,  ∈ Ĩ , are yielded by  0 Ĩ and (  ) −   (  ),  ∈ Ĩ as follows: Considering that the linear systems (10) and ( 9) have the same decomposed coefficient matrix, the computational cost is typically low.Lemma 8 shows that   is still a descent direction of (), so   can be taken as the main search direction to design the algorithm.The parameters    ,  ∈ Ĩ , need to be devised deliberately to guarantee the nonsingularity of the coefficient matrix and global convergence.It is a difficult work throughout the whole research.

Properties of Our Algorithm.
The proposed algorithm in this paper possesses the following properties.
(i) Only the constraints indexed by some subset   of  are considered which reduces the scale and computation cost of the subproblems to some extent.
(ii) At each iteration, only the solutions of two linear systems with the same coefficient matrix are required; that is, the new algorithm is completely QP-free.
(iii) It does not need any penalty parameters and barrier parameters.Therefore, the difficulty of choosing some suitable penalty parameters and barrier parameters is avoided.
(iv) It needs few parameters which are adjusted easily, and the algorithm is robust.
(v) It has weakly global convergence under some suitable assumptions.
We conclude this section by giving some notation which is used throughout this paper.The symbol ‖ ⋅ ‖ refers to the Euclidean norm.In addition, we denote by 0 an empty set, the cardinality of any finite set  by ||, and by det () the determinant of the matrix .Furthermore, the directional derivation of  at the point  along with the direction  is denoted by   (; ).It is easy to know that

Description of Algorithm
The new algorithm is based on the following assumption.
For a point  ∈   , an index  ∈ , and a given index set  such that {  (),  ∈ } are linearly independent, we introduce the following technique similar to [24] to generate the parameter   () in Lemma 1. Define where  is Napierian base and the parameter  ≥ 0.
Let   be a given iteration point.We denote   = min{ |  ∈ (  )} and use the following pivoting operation to generate the index set Ĩ such that   ≜  Ĩ (  ) has full column rank, so vectors {  (  ),  ∈ Ĩ } are linearly independent.
To describe some beneficial properties of the POP above, which is helpful for discussing the convergence of our algorithm, we have the following results.(i) The parameter   can be obtained in a finite number of steps in the POP.
(ii) If a sequence {  } is bounded, then there exist two constants , ε > 0 such that det ∀  ⊆ Ĩ . ( Proof.Based on the assumption (A1), it is easy to get that (i) and ( 18) hold, so it is omitted here.Now, we will prove that (19) holds.In view of Ĩ and   being the subsets of the fixed and finite set  and the boundedness of {  }, we assume by contradiction without loss of generality that there exist an infinite index set  and According to (14)  A detailed description of the algorithm for solving (1) is given below.
Step 1 (generating  active set).Set parameter  =  −1 , generate the set Ĩ by the POP, and let   be the corresponding termination parameter.

Ĩ𝑘
) of the following linear system: with Step 4. Compute the unique solution (  ,   Ĩ ) of the following linear system: where V  = (V   ,  ∈ Ĩ ) is yielded by If   = 0 and then   is a stationary point of (1), and stop.
For convenience of analysis in the rest of this paper, we give the equivalent forms of (LS1)-(LS2): Taking into account the inverse matrix of   ≜ (  ,   ,   ) that can be expressed as  −1  = ( ), with from (LS1)-(LS2) and (32), we have the following relations: (iii)   is a descent direction of () at the iterate   , so the proposed algorithm is well defined.
(ii) If  ∈ (  ) \ {  }, from (  ) \ {  } ⊆ Ĩ and ( 27), we have V   ≤ 0. From Remark 5, we can assume without loss of generality that  −1 ̸ = 0. Furthermore, it follows from (34) and ( 23) that   > 0. Therefore, from the second formula of (31) and assumption (A1), it is easy to get (iii) From ( 12) and the above results (i)-(ii), one knows that   (  ;   ) < 0, which implies that   is a descent direction of () at   .So the line search can be performed and the proposed algorithm is well defined.

Global Convergence
In this part, under mild assumptions, we show that Algorithm A is weak globally convergent; that is, there exists at least one accumulation point of the iterates {  } yielded by Algorithm A such that it is a stationary point of (1).To this end, in addition to (A1), the following two assumptions are necessary.
(A2) The sequence {  } generated by Algorithm A is bounded.
(A3) There exist two constants ,  > 0 such that The following lemma establishes the boundedness of the associated sequences generated by the algorithm, and its proof is similar to Lemma 3.1 in [25], so it is omitted here.
Proof.From Lemma 9, we know there exists an infinite subset Again, lim  → ∞   =  * follows since {  } is monotone and bounded.In view of Lemma 8 (i) and (A3), we have This along with (40)-(41) shows that So, passing to the limit for  ∈   ,  → ∞, in (30), one gets Furthermore, it follows from ( 23) and ( 27) that  +1 → 0,  ∈   .So,  * = 0 and  * 0 ,  ∈   ), and  * = ( *   , 0 \  ), then (44) implies that  * is a stationary point with the multiplier vector  * .Theorem 11.Suppose that (A1)-(A3) hold.Then Algorithm A either stops at a stationary point of (1) in a finite number of iterations or generates an infinite sequence {  }, of which at least one accumulation point is a stationary point of (1).In such sense, Algorithm A is said to possess weakly global convergence.

Numerical Results
In this section, some preliminary numerical tests on 5 typical problems from [26] are reported, and the computation results show that Algorithm A is efficient.All the numerical experiments were implemented on MATLAB 7.0, under Windows XP and 2.2 GHz CPU.The BFGS formula with Powell's modification [27] is adopted in the algorithm, and  0 is the identity matrix.The parameters were selected as  = 0.2,  = 0.6, and  −1 = 1.2.In addition, execution is terminated if one of the following termination criteria is satisfied: The computational results are reported in Table 1, and the columns of Table 1 have the following meanings: IP: the initial point; : the number of variables; : the number of functions   (); ALG: the type of algorithm; NI: the number of iterations."Algo A" represents Algorithm A in this paper, "J2006-1" and "J2006-2" represent the algorithms in [13], and "Hu2009" represents the algorithm in [15].
From Table 1, we can see that our algorithm can find the solutions of the test problems with a small number of iterations, and the computational results illustrate that our algorithm executes well for those problems.The numerical results are comparative with the algorithms in [13,15].Furthermore, we only need to solve two systems of linear equations with the same coefficient matrix per iteration.Considering that these linear systems have the same decomposed coefficient matrix, the computational cost per iteration of Algorithm A is typically low.This shows the potential advantage of our algorithm when applied to solving problems with large numbers of constraints.In addition, the parameters in the proposed algorithm are few, and the stability of the algorithm is very well.

Concluding Remarks
In this paper, a QP-free algorithm without solving any QP and QCQP subproblems is presented for unconstrained nonlinear finite minimax problems.At each iteration, only two systems of linear equations with the same coefficient matrix need to be solved.The proposed algorithm does not need any penalty parameters and barrier parameters which are difficult to deal with.Furthermore, under some mild assumptions, the global convergence is attained.As further work of this method, we think that there are still some problems worthy of discussing.For example, the assumption (A1) is different from the linearly independent assumption in common use, and any of them cannot derive the other.The discussion that the assumption (A1) is a constraint qualification needs further consideration.In addition, one should also take into account improving it to have superlinear convergence and generalizing it to solve minimax problems with inequality constraints.

Table 1 :
Numerical results of Algorithm A.