A Globally Convergent Filter-Type Trust Region Method for Semidefinite Programming

When using interior methods for solving semidefinite programming SDP , one needs to solve a system of linear equations at each iteration. For problems of large size, solving the system of linear equations can be very expensive. In this paper, based on a semismooth equation reformulation using Fischer’s function, we propose a filter method with trust region for solving large-scale SDP problems. At each iteration we perform a number of conjugate gradient iterations, but do not need to solve a system of linear equations. Under mild assumptions, the convergence of this algorithm is established. Numerical examples are given to illustrate the convergence results obtained.


Introduction
Semidefinite programming SDP is convex programming over positive semidefinite matrices.For early application, SDP has been widely used in control theory and combinatorial optimization see, e.g., 1-3 .Since some algorithms for linear optimization can be extended to many general SDP problems, that aroused much interest in SDP.In the past decade, many algorithms have been proposed for solving SDP, including interior-point methods IPMs 4-7 , augmented methods 8-10 , new Newton-type methods 11 , modified barrier methods 12 , and regularization approaches 13 .For small and medium sized SDP problems, IPMs are generally efficient.But for large-scale SDP problems, IPMs become very slow.In order to improve this shortcoming, 9, 14 proposed inexact IPMs using an iterative solver to compute a search direction at each iteration.More recently, 13 applied regularization approaches to solve SDP problems.All of these methods are first-order based on a gradient, or inexact second-order based on an approximation of Hessian matrix methods 15 .In this paper, we will extend filter-trust-region methods for solving linear or nonlinear programming 16 to large-scale SDP problems and use Lipschitz continuity.Furthermore, the accuracy of this method is controlled by a forcing parameter.It is shown that, under mild assumptions, this algorithm is convergent.
The paper is organized as follows.Some preliminaries are introduced in Section 2. In Section 3, we propose a filter-trust-region method for solving SDP problems, and we study the convergence of this method in Section 4. In Section 5, some numerical examples are presented to demonstrate the convergence results obtained in this paper.Finally, we give some conclusions in Section 6.
In this paper, we use the following common notation for SDP problems: X n and R m denote the space of n×n real symmetric matrices and the space of vectors with m dimensions, respectively; X 0 X 0 denotes that X ∈ X n is positive semidefinite positive definite , and X 0 X ≺ 0 is used to indicate that X ∈ X n is negative semidefinite negative definite .A superscript T represents transposes of matrices or vectors.For X, Y ∈ X n , the standard scalar product on the space of X n is defined by Let X be a p × q matrix.Then we denote by Vec X a pq vector made of columns of X stacked one by one, and the operator Mat • is the inverse of Vec • , that is, Mat Vec X X.We also denote that I is identity matrix.

Preliminaries
We consider a SDP problem of the form where The dual to the problem 2.1 is given by max b T y subject to A * y S C, S 0,

2.3
where A * is an adjoint operator of A : R m → X n given by Obviously, X ∈ X n and y, S ∈ R m × X n are the primal and dual variables, respectively.In addition, for τ > 0 and X, S ∈ X n , we define a mapping φ τ :

2.11
Then L C is strictly monotone and so has an inverse L −1 C .

The Algorithm
In this section, we will present a filter-trust-region method for solving SDP problems 2.1 and 2.3 .Firstly, for a parameter τ > 0, we construct a function: According to Lemmas 2.1, 2.3 and 2.4, the following theorem is obvious.
Theorem 3.1.Let τ > 0 and let H τ X, y, S be defined by 3.1 .If SDP problems 2.1 and 2.3 have strictly feasible points, then

3.2
In what follows, we will study properties of the function H τ X, y, S .For simplicity, in the remaining sections of this paper, we denote Z : X, y, S , Z k : X k , y k , S k and ΔZ : ΔX, Δy, ΔS .Theorem 3.2.Let H τ Z be defined by 3.1 .For any Z, ΔZ ∈ X n × R m × X n and τ > 0, then H τ Z is Fréchet-differentiable and where Δτ > 0 and C : Proof.For any Z ∈ X n × R m × X n , since A X − b and A * y S − C are linear functions and continuous differentiable, it follows that they are also locally Lipschitz continuous.Then, from Lemma 2.7, A X − b and A * y S − C are Fréchet-differentiable.Furthermore, X S − √ X 2 S 2 2τ 2 I is Fréchet-differentiable from Lemma 2.6.Thus, H τ Z is Fréchetdifferentiable and has the form of 3.3 .We complete the proof.
We endow the variable Z with the following norm:

3.4
In addition, we set where

3.6
We also define the function H τ Z and the vector h Z with the following norm:

3.8
Lemma 3.3.For any τ > 0 and Z ∈ X n × R m × X n , if X and S are nonsingular, then Ψ τ Z is locally Lipschitz continuous and twice Fréchet-differentiable at every Z ∈ X n × R m × X n .
Proof.For any τ > 0, since Ψ τ Z is convex and continuously differentiable, it follows that Ψ τ Z is also locally Lipschitz continuous.
In addition, for any Z ∈ X n ×R m ×X n , from 20, pages 173-175 , h 3 Z 2 is twice Fréchetdifferentiable.Furthermore, h 1 Z 2 , h 2 Z 2 , and h 4 Z 2 are continuous at every Z ∈ X n ×R m × X n when τ > 0, which, together with Lemma 2.7, Ψ τ Z is twice Fréchet-differentiable.The proof is completed.Lemma 3.4.Let H τ Z and Ψ τ Z be defined by 3.1 and 3.8 , respectively.For any τ > 0, we have 3.9 Proof.The proof can be immediately obtained from the definition of H τ Z and Ψ τ Z .
We follow the classical method for solving Ψ τ Z 0, which consists some norm of the residual.For any τ > 0, we consider min Ψ τ Z , 3.10 where Z ∈ X n × R m × X n .Thus, for any τ > 0, we want to find a minimizer Z * of Ψ τ Z .Furthermore, if Ψ τ Z * 0, then Z * is also a solution of H τ Z .In order to state our method for solving 3.10 , we consider using a filter mechanism to accept a new point.Just as 16, pages 19-20 , the notation of filter is based on that of dominance.Definition 3.5.For any τ > 0 and any 3.11 Thus, if iterate Z 1 dominates iterate Z 2 , the latter is of no real interest to us since Z 1 is at least as good as Z 2 for each of the components of h Z .All we need to do is remember iterates that are no dominated by other iterates by using a structure called a filter.Definition 3.6.Let F k be a set of 4-tuples of the following form:

3.12
We define F k as a filter if h Z k and h Z l belong to F k , when k / l, then h i Z k < h i Z l for at least one i ∈ {1, 2, 3, 4}.

3.13
Definition 3.7.A new point Z k is acceptable for the filter F k if and only if where α ∈ 0, 1/ √ 4 is a small constant.Now, we formally present our trust region algorithm by using filter techniques.

3.17
If ΔZ k < ε, stop.Otherwise, computer the trial point Z k Z k ΔZ k .
Step 3. Compute Ψ τ k Z k and define the following ratio: If r k < η 1 but Z k satisfies 3.14 , then add h Z k to the filter F k and remove all points from F k dominated by h Z k .At the same time, set Z k 1 Z k .
Else, set Z k 1 Z k .
Step 5. Update τ k by choosing and update trust-region radius Δ k by choosing

3.20
Step 6. Set k : k 1 and go to Step 1. Remark 3.9.Algorithm 3.8 can be started any τ > 0. In fact, in order to increase the convergent speed greatly, we always choose τ 0 X 0 , S 0 /2n.In addition, in this algorithm, we fix τ at first, then search Z for Ψ τ Z 0 to update Z.At last we update τ and repeat.
The following lemma is a generalized case of Proposition 3.1 in 23 .
For the purpose of our analysis, in the sequence of points generated by Algorithm 3.8, we denote Remark 3.11.Lemma 3.3 implies that there exists a constant 0 < M ≤ 1 such that for all k ∈ C and i ∈ {1, 2, 3, 4}.The second of above inequalities ensures that the constant 0 < M ≤ 1 can also be chosen such that

Convergence of Analysis
In this section, we present a proof of global convergence of Algorithm 3.8.First, we make the following assumptions.Some lemmas will be presented to be used in the subsequent analysis.
where ΔZ k is a solution of 3.16 .

S2
The iterations generated by Algorithm 3.8 remain in a close, bounded domain.
Proof.Suppose that ∇Ψ τ k Z k / 0 for all k ∈ C. Then there exists ω 0 > 0 such that From Lemma 4.1, there exists ω 1 > 0 such that On the other hand, |C| < ∞, let N be the last successful iteration, then where we obtain that

4.11
From 4.10 , we know that Ψ τ k Z k is monotone decreasing and bounded below, which implies that Ψ As a result, we have By the update rule of Δ k , there exists an infinite subsequence K ⊆ K, and we have that which contradicts k ∈ K ⊆ A. This completes the proof.
Proof.First let {τ k } be the sequence generated by Algorithm 3.8.From Lemma 4.2, we have which, together with assumption S2 , the desired result follows from 16, Lemma 3.1 .

Numerical Experiments
In this section, we describe the results of some numerical experiments with the Algorithm 3.8 for the random sparse SDP considered in 13 .All programs are written in Matlab code and all computations are tested under Matlab 7.1 on Pentium 4. In addition, in the computations, the following values are assigned to the parameters in the Algorithm: η 1 0.1, η 2 0.5, η 3 0.8, μ 0.1, γ 0.2, γ 1 0.5, and γ 2 2. We also use the stopping criteria is being of ε 10 −8 .
In the following Table 1, the first two columns give the size of the matrix C and the dimension of the variable y.In the middle columns, "F-time" denotes the computing time in seconds , "F-it."denotes the numbers iteration, and "F-obj."defines the value of Ψ τ k Z k when our stopping criteria is satisfied.Some numerical results of 13 are shown in the last two columns.
As shown in Table 1, all test problems have been solved just few iterations compared with 13 .Furthermore, this algorithm is less sensitive to the size of SDP problems.Comparatively speaking, our method is attractive and suitable for solving large-scale SDP problems.

Conclusions
In this paper, we have proposed a filter-trust-region method for SDP problems.Such a method offers a trade-off between the accuracy of solving the subproblems and the amount of work for solving them.Furthermore, numerical results show that our algorithm is attractive for large-scale SDP problems.

Theorem 4 . 3 .
Let |C| < ∞, assumptions (S1) and (S2) hold.Then there exists k ∈ C such that are unsuccessful iterations.From Steps 4 and 5 of Algorithm 3.8, r N j < η 1 , for sufficiently large N, we have lim We now consider what happens if the set A is infinite in the course of Algorithm 3.8.× R m × X n , if X and S are nonsingular, then each accumulation point of the infinite sequences generated by Algorithm 3.8 is a stationary point of Ψ τ Z .Proof.The proof is by contradiction.Suppose that {Z k } is an infinite sequence generated by Algorithm 3.8, and any accumulation point of {Z k } is not a stationary point of Ψ τ Z .Suppose furthermore that Z * and τ * are the accumulation points of {Z k } and {τ k }, respectively.Since Z * is not a stationary point of Ψ τ Z , then For some * > 0, let N Z * , * be a neighborhood of Z * .From 4.8 , there exists {Z k } k∈K ∈ N Z * , * such that

Table 1
In what follows, we investigate the case where the number of iterations added to the filter F k in the course of Algorithm 3.8 is infinite.Suppose that |C| |B| ∞ but |A| < ∞, SDP problems 2.1 and 2.3 have strictly feasible points.Suppose furthermore that assumptions (S1) and (S2) hold.For any τ > 0 and Z