An Alternating Direction Method for Convex Quadratic Second-Order Cone Programming with Bounded Constraints

An alternating direction method is proposed for convex quadratic second-order cone programming problems with bounded constraints. In the algorithm, the primal problem is equivalent to a separate structure convex quadratic programming over secondorder cones and a bounded set. At each iteration, we only need to compute the metric projection onto the second-order cones and the projection onto the bound set. The result of convergence is given. Numerical results demonstrate that our method is efficient for the convex quadratic second-order cone programming problems with bounded constraints.


Introduction
In this paper, we consider a convex quadratic second-order cone programming (CQSOCP) problem with bounded constraints which is defined by minimizing a convex quadratic function over the intersection of an affine set, a bounded set, and the product of second-order cones.The primal convex quadratic second-order cone programming problem is defined as where Ω = { |  ≤  ≤ } is a bounded set,  is an  ×  symmetric positive semidefinite matrix,  ∈ R × ,  ∈ R  ,  ∈ R  , and  = [ 1 , . . .,   ] ∈ R  1 × ⋅ ⋅ ⋅ × R   is viewed as a column vector in R  1 +⋅⋅⋅+  with ∑  =1   = .In addition,  =   1 ×   2 × ⋅ ⋅ ⋅ ×    and   ∈    , where    is the   -dimensional second-order cone given by where ‖ ⋅ ‖ 2 is the standard Euclidean norm.
Convex quadratic second-order cone programming problem with bounded constraints is a nonlinear programming problem, which can be seen as a trust region subproblem in the trust region method for the nonlinear second-order cone programming [1,2].Since  is symmetric positive semidefinite, we can compute its positive semidefinite square root  1/2 by the Cholesky method.Then problem (1) can be equivalently transformed as the following mix linear and second-order cone programming (MLSOCP) [3]: 1/2 In paper [1,2], the authors use those well developed and publicly available softwares, based on interior-point methods, such as SeDuMi [4] and SDPT3 [5] to solve the equivalent MLSOCP (3).
Interior-point methods have been well developed for linear symmetric cone programming [6][7][8].However, at each 2 Mathematical Problems in Engineering iteration these solvers require to formulate and solve a dense Schur complement matrix, which for the CQSOCP problem with bounded constraints amounts to a linear system of dimension ( + 3 + 2) × ( + 3 + 2).In addition, the transformed method needs to compute the square root of semidefinite matrix .When  is large, because of the very large size and ill-conditioning of the linear system of equations, interior-point methods are difficult to solve the transformed MLSOCP problem efficiently [3].
The alternating direction method (ADM) has been an effective first-order approach for solving large optimization problems, such as linear programming [9], linear semidefinite programming (LSDP) [10,11], nonlinear convex optimization [12], and nonsmooth  1 minimization arising from compressive sensing [13,14].A modified alternating direction method is proposed for convex quadratically constrained quadratic semidefinite programs in paper [15].In the thesis [3], a semismooth Newton-CG augmented Lagrangian method is proposed for large scale convex quadratic symmetric cone programming.In paper [16], an alternating direction dual augmented Lagrangian method for solving linear semidefinite programming problems in standard form is presented and extended to the SDP with inequality constraints and positivity constraints.
In the paper, an alternating direction method for the CQSOCP problem with bounded constraints is proposed.Firstly, the primal problem is equivalent to a separate structure convex quadratic programming over second-order cones and a bounded set.Then the alternating direction method is proposed to solve the separate structure convex quadratic programming.In the alternating direction method, we only need to compute the metric projection onto the second-order cones and projection onto the bounded set at each iteration.We also give the convergence results and the numerical results.

The Projection on the Second-Order Cone and the Bounded Set
In this section, we will give the projection results on the second-order cones and the bounded set. Let , . . ., ; then the spectral decomposition of   associate with second-order cone    can be described as [17][18][19] where with Next we introduce the projection lemma over the secondorder cone [17][18][19].
Let  = [ 1 , . . .,   ] ∈ R  1 ×⋅ ⋅ ⋅×R   ; then the projection   () of  over the cone  is described as Let  ∈ R  ; then the projection on the bounded set Ω is easy to carry out, namely, through an element by element method:

An Alternating Direction Method for CQSOCP Problems with Bounded Constraints
In this section, we give an alternating direction method for convex quadratic second-order cone programming problems with bounded constraints.Firstly, we give an equivalent separate structure convex quadratic programming over second-order cone and bounded set as follows: The Lagrangian function for the separate structure convex quadratic programming problem is written as where  ∈ R  ,  ∈   .Under mild constraint qualifications (e.g., Slater condition), strong duality holds for problem (9), and hence,  * is an optimal solution of (9) if and only if there exists Mathematical Problems in Engineering 3 ( * ,  * ,  * ,  * ) ∈  × Ω × R  × R  satisfying the following KKT system in variational inequality form: The augmented Lagrangian function for the the separate structure convex quadratic programming problem is defined as where  1 ,  2 > 0.
The variational inequality form of alternating direction method for ( 12) is as follows.
Lemma 2 (see [20]).Let Θ be a closed convex set in a Hilbert space and let  Θ () be the projection of  onto Θ.Then ⟨ − ,  − ⟩ ≥ 0, ∀ ∈ Θ ⇐⇒  =  Θ () . ( Taking (17), we see that ( 13) is equivalent to the following nonlinear equation: where  1 can be any positive number. Taking )) and  =  +1 in ( 17), we see that ( 14) is equivalent to the following nonlinear equation: where  2 can be any positive number.Due to the existence of the terms  +1 and    +1 in (18), we can not compute  +1 directly.We therefore use the following approximate approach which is similar to the one in paper [15].For certain constants  1 and  2 , let be the residual between  +1 ,    +1 and their linearization at   , respectively.
Instead of computing (18), we compute We choose  1 ,  2 so that  1 >  max (),  2 >  max (  ), where  max () and  max (  ) are the largest eigenvalues of  and   , respectively. Setting in (21), we have which will be used as an approximation to the solution of variational inequality (13).
Let  2 =  2 in (19); we have In summary, the modified alternating direction method is given as follows.

The Convergence Result
In this section, we extended and modified the convergence results of the alternating direction methods for convex quadratically constrained quadratic semidefinite programs in paper [15] and then give the convergence analysis of the alternating direction method for CQSOCP problems with bounded constraints.Lemma 3. The sequence {  ,   ,   ,   } generated by the modified alternating direction method satisfies where { * ,  * ,  * ,  * } is a KKT point of system (11).
Proof.Let  =  +1 in the second inequality in system (11); we have Let  =  * in ( 14), and coupled with ( 16), we have Adding (30) and (31) together, we have In addition, from ( 14) and ( 16), we have Adding the two inequality above, we have Note that (21) can be written equivalently as Setting  =  * , we have Let  =  +1 in the first inequality in system (11); we have Adding ( 36) and (37) together, we have From first part at the left side of (38) and the third equation in system (11), we have From ( 16), (36), the last equation in system (11), and the second part at the left side of (38), we have In addition, from the third part at the left side of (38), we have It follows from (32)-( 34) and ( 38)-(41) that Now, we give the convergent conclusion.
Theorem 4. The sequence {  } generated by the modified alternating direction method converges to a solution point  * of problem (9).
Proof.We denote where   denotes the -dimensional unit matrix and  is positive definite.Here, we define the -inner product of  and  as and the associated -norm as where Observe that, by Lemma 2, solving the optimal condition (11) for problem (9) From ( 15), ( 16), and the first equation in (21), we have that From ( 19) and ( 16), we have Based on ( 47)-( 48), ( 15)-( 16), and the nonexpansion property of the projection operator, we have (50) From Lemma 3, we can write (29) as which implies that From the above inequality, we have That is, the sequence {  } is bounded.Thus there exists at least one cluster point of {  }.
It also follows from (53) that and thus lim Let  be a cluster point of {  } and the subsequence {   } converges to .We have so  satisfies system (11).Setting  * = , we have The sequence

Simulation Experiments
In this section we present computational results by comparing the modified alternating direction method with the interior-point method.The interior-point method is used to solve the transformed mix linear and second-order cone programming problems (3).All the algorithms are run in the MATLAB 7.0 environment on an Inter Core processor 1.80 GHz personal computer with 2.00 GB of Ram.
The first set of test problems includes 16 small scale CQSOCP problems with bounded constraints, which is shown in Table 1.In Tables 1 and 3, an entry of the form "20 × 5" in the "SOC" column means that there are 20 5dimensional second-order cones, and the "ratio" denotes the ratio between the number of the second-order cones and the value of .As is known to all, the interior-point methods have proved to be one of the most efficient class of methods for SOCP.Here the Matlab program codes for the interior-point method are designed from the software package by SeDuMi [4].In the SeDuMi software, we set the desired accuracy parameter .= 10 −6 .
For the first set of test problems, the iteration number and average CPU time are used to evaluate the performances of the modified alternating direction method and the interiorpoint method by SeDuMi.The test results are shown in Table 2.In the Tables 2 and 4, "Time" represents the average CPU time (in seconds), and "Iter." denotes the average number of iteration.In addition, "MADM" represents the modified alternating direction method.In Table 4, "/" denotes that the method does not work in our personal computer because the method is "out of memory." Table 2 shows that the modified alternating direction method costs less CPU time than the interior-point method by SeDuMi.But, the iteration number of the interior-point method is less than that of the modified alternating direction method.
In addition, Table 1 gives different kinds of test problem, including the problems with only one large second-order cone, such as P01, P05, P09, and P13, the problems with many small second-order cones, such as P04, P08, P12, and P16, and the problems with one large second-order cone and some small second-order cones, such as P02, P06, P10, and P14.The test results in Table 2 show that the modified alternating direction method can solve different kinds of convex quadratic second-order cone programming problems within appropriate CPU time and accuracy.
The second set of test problems includes 15 medium scale problems, which is shown in Table 3.For the second set of test problems, the test results are shown in Table 4.
The results in Table 4 show the interior point method by SeDuMi does not work for the transformed problem (3) because of being "out of memory" in our personal computer when  > 2000, but the modified alternating direction method is still efficient because the modified alternating direction method needs less memory space than the interiorpoint method.
In addition, we add test results of P04 and P12 in smaller criteria and with random initial points.The smaller criteria of our method is 10 −10 .In addition, we do one hundred experiments with the random initial point.The test results are shown in Table 5.In the SeDuMi software, we set the desired accuracy parameter .= 10 −8 .
Table 5 shows that the performances of MADM with random initial points are a bit better than that of MADM with fixed initial points in two different stop criteria.In addition, the number of iteration of MADM with  = 10 −10 is more than that of MADM with  = 10 −6 , and the CPU time of MADM with  = 10 −10 is longer than that of MADM with  = 10 −6 .

Conclusion
In the paper, a modified alternating direction method is proposed for solving convex quadratic second-order cone programming problems with bounded constraints.The proposed method does not require solving subvariational inequality problems over the second cones and the bounded set.At each iteration, we only need to compute the metric projection onto the second-order cones and a projection onto the bounded set.The proposed modified method does not require secondorder information and it is easy to implement.The random simulation results show that our method can efficiently solve some convex quadratic second-order cone programming problems of vector size up to 5000 within reasonable time and accuracy by using a desktop computer.

Table 1 :
The test problems with small scale.

Table 2 :
The results for the test problems with small scale.

Table 4 :
The results for the test problems with small scale.

Table 5 :
The results in smaller criteria and with random initial points.