Journal of Applied Mathematics

Volume 2013 (2013), Article ID 963563, 7 pages

http://dx.doi.org/10.1155/2013/963563

Research Article

A Rank-Two Feasible Direction Algorithm for the Binary Quadratic Programming

Xuewen Mu¹ and Yaling Zhang1,2

¹School of Mathematics and Statistics, Xidian University, Xi’an 710071, China

²School of Computer Science, Xi’an Science and Technology University, Xi’an 710054, China

Received 16 March 2013; Accepted 3 October 2013

Academic Editor: Luigi Muglia

Copyright © 2013 Xuewen Mu and Yaling Zhang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Based on the semidefinite programming relaxation of the binary quadratic programming, a rank-two feasible direction algorithm is presented. The proposed algorithm restricts the rank of matrix variable to be two in the semidefinite programming relaxation and yields a quadratic objective function with simple quadratic constraints. A feasible direction algorithm is used to solve the nonlinear programming. The convergent analysis and time complexity of the method is given. Coupled with randomized algorithm, a suboptimal solution is obtained for the binary quadratic programming. At last, we report some numerical examples to compare our algorithm with randomized algorithm based on the interior point method and the feasible direction algorithm on max-cut problem. Simulation results have shown that our method is faster than the other two methods.

1. Introduction

In this paper, we consider the following binary quadratic programming:

where is real symmetric matrices and is a real -dimensional column vector.

The binary quadratic programming is a fundamental problem in optimization theory and practice. Some combinatorial optimization problems and engineering problems can be modeled as binary quadratic programming, such as VLSI design, statistical physics, max-cut problem [1], the optimal multiuser detection [2–5], image processing [6], and the design of FIR filters with discrete coefficients [7]. These problems are known to be -hard [1]. One typical approach to solve these problems is to construct lower bounds for approximating the optimal value. Now, the semidefinite programming (SDP) relaxation approach had been studied and proven to be quite powerful for finding approximate optimal solutions. Based on solving its semidefinite programming relaxation, Goemans and Williamson [8] developed a randomized algorithm for the max-cut problem, which provides an approximate solution guaranteed to be within a factor of 0.87856 of its optimal value. Interior point method is a powerful method for SDP with small and moderate scale. But the interior point method is limited to problems of moderate size, which cannot solve SDP with large scale efficiently [1]. So Goemans and Williamson’s method based on the interior point method is not adapted to solve the large scale max-cut problems.

Some efficient nonlinear programming algorithms only based on gradient for solving the SDP relaxation of the max-cut problem have been developed. Homer and Peinado [9] proposed a parallel and distributed approximation algorithms for max-cut problem. In the algorithm, the author transformed the max-cut SDP relaxation into a constrained nonlinear programming problem in the new variable for using the change of variables , , where is the primal matrix variable of the SDP relaxation. Burer and Monteiro [10] proposed a projected gradient algorithm for solving the max-cut SDP relaxation by using the change of variable , where is a lower triangular matrix. The rank-two relaxation heuristics algorithm in [11] relaxed the max-cut problem to form an unconstrained optimization problem by replacing each binary variable with one unit vector in space and then using polar coordinates. In [12], the rank-two SDP relaxation model is proposed for maximal independent set problem. Based on the low-rank decomposition of the semidefinite matrix, Liu et al. [13] proposed a feasible direction method to solve a nonlinear programming model of binary quadratic programming.

In the paper, we propose a rank-two feasible direction method for the binary quadratic programming. we restrict the rank of matrix variable to be two in the semidefinite programming relaxation and obtain a quadratic objective function with simple quadratic constraints. A feasible direction method is used to solve the nonlinear programming. We also give the analysis of the convergence and the complexity of the method. The randomized algorithm is used to obtain the suboptimal solution of the binary quadratic programming. At last, we compare our method with the randomized algorithm based on the interior point method and the feasible direction method on max-cut problem. Simulation results show that our method costs less CPU time than the two methods.

2. The SDP Relaxation Method for Binary Quadratic Programming

In this section, we introduce the SDP relaxation of binary quadratic programming problem [1].

In problem (1), let , , and ; then problem (1) can be formulated as

It is well known that problem (2) is also -hard [1].

Let , where denotes the largest eigenvalue of the matrix and denotes the unit matrix; then is a negative definite matrix. Problem (2) is equivalent to the following problem below:

Letting and ignoring the constant term, then problem (3) is equivalent to the following problem:

where and is the diagonal elements of matrix . In addition, denotes that matrix is semidefinite. Ignoring the nonconvex “rank one” constraint, the SDP relaxation is given as follows [1]:

Interior point methods have been proved to be quite efficient for small and moderate scale SDP. In the 0.878 randomized method by Goemans and Williamson [8], the author solved the SDP relaxation problem (5) by interior point methods. However, interior point methods are second-order method, so they are quite time and memory intensive and not adapted to large scale binary quadratic problems. The complexity of the primal-dual interior point method based on AHO search direction for the SDP relaxation (5) of the max-cut is [14, 15].

3. The Rank-Two SDP Relaxation for Binary Quadratic Programming

In [11], the rank-two SDP relaxation model is proposed for max-cut problem based on the polar direction. In [12], the rank-two SDP relaxation model is proposed for maximal independent set problem. In this section, we present a rank-two SDP relaxation based on the rank-two approximate matrix for binary quadratic programming.

In SDP relaxation problem (5), let ; we have

Obviously, matrix is positive definite.

Let , [12]; then and . We obtain the rank-two SDP relaxation of binary quadratic programming as follows:

where and are the elements of vector and . Obviously, matrix satisfies that , , so problem (7) is a rank-two SDP relaxation problem.

Problem (7) is also a nonlinear programming with quadratic objective function and constraints. Compared to the variables of SDP relaxation, the rank-two relaxation has only variables, so this approach possesses scalability for solving large-scale binary quadratic programming problems.

Let

then the gradients of the function and are

The KKT condition for problem (7) is given here. If the variable in problem (7) satisfies the following condition:

then is a KKT point for problem (7).

It is simple to obtain that

Then we have the equivalent KKT condition for problem (7) as follows:

4. The Rank-Two Feasible Direction Algorithm for Binary Quadratic Programming

Feasible direction algorithm is an efficient algorithm for some special nonlinear programming problems. In [13], the feasible direction algorithm is applied to solve the low-rank nonlinear programming relaxation for binary quadratic programming problems. In [16], the feasible direction algorithm is applied to solve a rank one nonlinear programming relaxation for max-cut problem.

In this section, we extend the feasible direction algorithm to solve problem (7). The algorithm employs only gradient evaluations of the objective function in problem (7), and no calculations on any matrices and no line searches, thus greatly reduces the calculation costs and increases the efficiency of the algorithm.

In the rank-two feasible direction algorithm, we give the following iteration for problem (7):

where denotes the element pair of matrix variable .

The iteration (13) is very simple and has the following characteristics.

(1) No matrix calculations and no line searches are required, and only one gradient evaluation is needed to get the new iteration.(2) The new iteration point is feasible to problem (7).(3) If the sequence converges to , then is feasible to problem (7).

Define direction as follows:

as a search direction, where is the elements pair of . Then the iteration (13) can be written as

The following lemmas show that if , then is a KKT point of problem (7), and if , then is a feasible increasing direction for problem (7).

Lemma 1. If , then is a point of (7).

Proof. It is clear that satisfies the constraint in problem (7). Since , then

Let ; by the KKT condition (12), we have that is a KKT point of (7). This completes the proof of the lemma.

Lemma 2. Suppose that ; then is a feasible increasing direction for problem (7), and iteration point is feasible to problem (7).

Proof. The feasibility of the iteration point directly comes from definition (13). Using the fact that , we have

So direction is a feasible increasing direction for problem (7).

The convergence of the feasible direction method is concluded by the following lemmas.

Lemma 3. Suppose that ; then any accumulation point is a point of (7).

Proof. Let be an accumulation point of the sequence ; it is simple to obtain the result by Lemma 1.

Lemma 4 (see [1]). Let matrixes and be positive definite; then is bounded by

where and denote the smallest and the largest eigenvalues of the matrix .

Lemma 5. If for all , then .

Proof. Since

from Lemma 4, we have

where denotes the Frobenius norm matrix .

From Lemma 2 and Lemma 4, for any , we have

This shows that is convergent, and hence holds.

In view of Lemmas 1 and 5, the termination criterion used in the rank-two feasible algorithm is , where is a prespecified constant.

Lemma 6. For all initial point and , the rank-two feasible direction algorithm terminates in iterations, where is an integer which does not exceed .

Proof. Based on Lemma 5, the number of iterations of the rank-two feasible direction algorithm is finite. Let be the number of iterations; then

So we obtain

Now we conclude that is an upper bound on the number of iterations.

Since problem (7) is nonconvex, there is no guarantee that the solution generated from the feasible direction method is a global solution. However, numerical experiments in Section 5 show that the proposed algorithm always converges to the optimal solution set of problem (7).

Now, we derive the complexity of our algorithm.

The complexity of computation of the gradient is . Each norm of the gradient of the objective function can be computed in , so we conclude that the overall complexity to evaluate the next iteration point is . Together with Lemma 6, we get that the overall complexity of the rank-two feasible direction algorithm is . Here we can choose satisfying ; then the complexity does not exceed . We know that the complexity of the primal-dual interior point method based on AHO direction is [14, 15]. It is obvious that the complexity of the primal-dual method is higher than that of our algorithm, so our algorithm is faster than the interior-point method for the large-scale SDP relaxation of binary quadratic programming problems. In addition, the complexity of low rank feasible direction method is [13], which is higher than that of our method.

Let the KKT point of problem (7) be ; then we can obtain the rank-two solution . Since the rank-two relaxation has the same form as Goemans and Williamson’s relaxation [8], except that ours has variables in rather than , the same analysis as Goeman and Williamson, with minimal changes, can be applied. By the randomized cut generation scheme, the suboptimal solution of binary quadratic programming problem is obtained.

5. Numerical Results

In this section we present computational results by comparing our method with GW randomized algorithm [8] based on interior point method and low-rank feasible direction algorithm to find approximate solutions to the max-cut problem.

In interior point method, we solve the SDP relaxation by three SDP solvers, which include SDPpack software [17], SeDuMi [18], and DSDP [19]. SeDuMi is one of state-of-the-art SDP solvers. The code DSDP uses a dual-scaling interior-point algorithm and an iterative linear-equation solver. It is currently one of the fastest interior-point codes for solving SDP problems. Low-rank feasible direction algorithm is one of the efficient methods for the max-cut problem, and the algorithm is faster than the projected gradient algorithm [13]. The projected gradient algorithm [10] is faster than Homer and Peinado algorithm [9].

All the algorithms are run in the MATLAB 7.0 environment on an Inter Core2 D2.0 GHz personal computer with 2.0 GB of RAM.

5.1. Max-Cut Problem

The max-cut problem is one of the standard -complete problems defined on graphs [8]. Let denote an edge-weighted undirected graph without loops or multiple edges. We use , for an edge with endpoints and and for the weight of an edge . For the cut is the set of edges that have one endpoint in and the other endpoint in . The max-cut problem asks for the cut maximizing the sum of the weights of its edges. Here, we only work with the complete graph . In order to model a graph in this setting, define is referred to as the weighted adjacency matrix of the graph. An algebraic formulation can be obtained by introducing cut vectors . The max-cut problem can be formulated as the integer quadratic program as follows:

The matrix is called the Laplace matrix of the graph , where is the unit vector whose every component is 1 and is the diagonal matrix whose diagonal elements are . Let ; the max-cut problem may be interpreted as a special case of the problem (1).

5.2. Numerical Results for the Random Graphs

The first set of test problems contains random graphs with two different edge densities 0.8 and 0.2, which denotes the dense random graphs and sparse random graphs, respectively. The weight on each edge is 1. We select problems in size from to for comparing the suboptimal value of max-cut problem and the CPU time of the four methods.

For the interior point method, we use the codes by two SDP solvers, which include SDPpack software [17] and SeDuMi [18]. In our algorithm, the iteration stops once is found. The result is shown in Table 1.

Table 1: Comparison results for the random graphs.

Size

Density

SDPpack
SeDuMi
FD
R2FD

Values
CPU
Values
CPU
Values
CPU
Values
CPU

50
0.2
319
0.73
319
0.59
319
0.05
319
0.04

50
0.8
781
0.67
781
0.59
781
0.06
779
0.04

100
0.2
1185
9.27
1185
1.64
1185
0.21
1182
0.09

100
0.8
3074
10.00
3074
1.69
3075
0.28
3071
0.11

150
0.2
2521
54.46
2521
4.23
2524
0.92
2525
0.19

150
0.8
6811
41.99
6811
4.17
6813
1.15
6811
0.22

200
0.2
4404
121.33
4404
8.71
4407
2.19
4413
0.35

200
0.8
12046
152.46
12046
8.02
12051
2.98
12052
0.39

250
0.2
6729
301.64
6729
12.32
6739
4.64
6738
0.47

250
0.8
18703
336.20
18703
15.15
18701
6.87
18711
0.62

300
0.2
9521
618.06
9521
27.17
9527
10.83
9551
0.86

300
0.8
26501
687.51
26501
26.44
26513
13.85
26541
1.05

350
0.2
12934
1076.24
12934
39.13
12945
18.60
12964
1.33

350
0.8
36067
1345.38
36067
40.92
36064
25.31
36094
1.43

In Table 1, “SDPpack” stands for randomized algorithm based on interior point method solved by SDPpack software, “SeDuMi” for randomized algorithm based on interior point method solved by SeDuMi software, “FD” for feasible direction algorithm coupled with the randomized method, “R2FD” for our rank-two feasible direction algorithm coupled with the randomized method, “CPU” for the CPU time, “Values” for the suboptimal value of the max-cut problem based these methods, and “Density” for edge density of the random graphs.

The SDPpack and SeDuMi provide the currently best conclusion on its performance guarantee in theory. The results in Table 1 show that the approximate solutions obtained by R2FD are as good as those generated by SDPpack, SeDuMi, and FD. In addition, the CPU time of our method is less than that of SDPpack, SeDuMi, and FD. In particular, with the increase of the size of the max-cut problem, the ratios of the CPU time between our methods to the three methods decrease quickly.

5.3. Numerical Results for the G-Set Graphs

The second set of test problems are from the so-called G-set graphs, which are randomly generated by the procedure rudy, a machine independent graph generator written by Rinaldi [20], Helmberg and Rendl [21], and Alperin and Nowak [22]. [20–22]. The test problems include 14 randomly generated large size test problems with nodes from 800 to 2000. Recently, Choi and Ye [19] reported computational results on a subset of G-set graphs that were solved as max-cut problems using their SDP code COPL-DSDP, or simply DSDP. The code DSDP uses a dual-scaling interior-point algorithm and an iterative linear-equation solver. The SDPpack software does not work when the size of the max-cut problems is larger than 350, so we give the results by the randomized method based on the dual-scaling algorithm solved by the DSDP software [19].

Table 2 gives the results of comparison among our R2FD method, the FD method, and the randomized method based on DSDP and SeDuMi on 14 large size test problems in the second set. In Table 2, “DSDP” presents the randomized method based on the dual-scaling algorithm by the DSDP software.

Table 2: Comparison results for G-set graphs.

Graph

Size

DSDP
SeDuMi
FD
R2FD

Values
CPU
Values
CPU
Values
CPU
Values
CPU

G01
800
11404
66
11389
590
11446
258
11469
4

G03
800
11403
28
11413
626
11431
72
11444
4

G13
800
552
22
552
542
554
384
540
3

G14
800
2979
45
2979
645
2986
336
2998
4

G15
800
2972
32
2974
728
2975
55
2974
4

G22
2000
12978
817
12979
11213
12995
5630
13146
35

G23
2000
12971
898
12976
12546
12969
6637
13074
34

G24
2000
12959
802
12960
10859
12993
5236
13062
37

G35
2000
7438
1387
7438
13236
7446
7397
7504
38

G36
2000
7421
1717
7422
14078
7427
7104
7480
32

G37
2000
7441
1390
7445
13569
7449
6891
7495
35

G43
1000
6504
91
6497
1192
6525
487
6528
8

G44
1000
6470
90
6479
1195
6507
496
6520
7

G53
1000
3731
70
3738
1135
3747
738
3753
7

The results in Table 2 show that the approximate solutions by our method is nearly as good as those of the DSDP cuts. But our method which reaches solutions of the problems is at least 10 times faster than the FD method, 7 times faster than DSDP, and 100 times faster than SeDuMi. In particular, for G35, G36, and G37, the CPU time of our method is almost 40 times less than that of DSDP. Furthermore, We observe that R2FD took less than 5 minutes to return approximate solutions to all the 14 test problems, which required more than 2 hours of computation by the DSDP, more than 11 hours of computation by the FD, and more than 22 hours of computation by the SeDuMi.

6. Conclusion

Because the interior-point method and feasible direction method increase the dimension of a problem from to and ( is the function of ), so the two methods cost more CPU time than our method for solving large size binary quadratic programming problems, especially for problems with a large number of edges. The rank-two feasible direction method only increases the dimension of a problem from to , so it is efficient for solving large size binary quadratic programming.

Acknowledgments

The work of Xuewen Mu is supported by the National Science Foundation for Young Scientists of China (Grant nos. 11101320 and 61201297) and the Fundamental Research Funds for the Central Universities (Grant no. K50511700007). The work of Yaling Zhang is supported by the Xian University of Science and Technology Cultivation Foundation in Shaan Xi Province of China (Program no. 2010032).

References

C. Helmberg, Semidefinite Programming for Combinatorial Optimization, Konrad-Zuse-Zentrum fur Information-Stechnik, Berlin, Germany, 2000.

F. Barahona, M. Groetschel, M. Juenger, and G. Reinelt, “An application of combinatorial optimization to statiscal optimization and circuit layout design,” Operations Research, vol. 36, no. 3, pp. 493–513, 1988.

P. H. Tan and L. K. Rasmussen, “The application of semidefinite programming for detection in CDMA,” IEEE Journal on Selected Areas in Communications, vol. 19, no. 8, pp. 1442–1449, 2001.

F. Hasegawa, J. Luo, K. Pattipati, and P. Willett, “Speed and accuracy comparation of techniques to solve a binary programming problem with application to syschronous CDMA,” IEEE Transaction on Communication, vol. 52, pp. 2775–2780, 2004.

X. Mu and Y. Zhang, “A new rank-two semidefinite programming relaxation method for multiuser detection problem,” Wireless Personal Communications, vol. 65, pp. 223–233, 2012.

J. Keuchel, C. Schnörr, C. Schellewald, and D. Cremers, “Binary partitioning, perceptual grouping, and restoration with semidefinite programming,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 11, pp. 1364–1379, 2003.

S.-P. Wu, S. Boyd, and L. Vandenberghe, “FIR filter design via semidefinite programming and spectral factorization,” in Proceedings of the 35th IEEE Conference on Decision and Control, pp. 271–276, Kobe, Japan, December 1996.

M. X. Goemans and D. P. Williamson, “Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming,” Journal of the Association for Computing Machinery, vol. 42, no. 6, pp. 1115–1145, 1995.

S. Homer and M. Peinado, “Design and performance of parallel and distributed approximation algorithms for maxcut,” Journal of Parallel and Distributed Computing, vol. 46, no. 1, pp. 48–61, 1997.

S. Burer and R. D. C. Monteiro, “A projected gradient algorithm for solving the maxcut SDP relaxation,” Optimization Methods and Software, vol. 15, no. 3-4, pp. 175–200, 2001.

S. Burer, R. D. C. Monteiro, and Y. Zhang, “Rank-two relaxation heuristics for max-cut and other binary quadratic programs,” SIAM Journal on Optimization, vol. 12, no. 2, pp. 503–521, 2001.

S. Burer, R. D. C. Monteiro, and Y. Zhang, “Maximum stable set formulations and heuristics based on continuous optimization,” Mathematical Programming A, vol. 94, no. 1, pp. 137–166, 2002.

H. Liu, X. Wang, and S. Liu, “Feasible direction algorithm for solving the SDP relaxations of quadratic programming problems,” Optimization Methods & Software, vol. 19, no. 2, pp. 125–136, 2004.

F. Alizadeh, P. A. Haeberly, and M. L. Overton, “Primal-dual interior-point methods for semidefinite programming: convergence rates, stability and numerical results,” SIAM Journal on Optimization, vol. 8, no. 3, pp. 746–768, 1998.

M. J. Todd, “Semidefinite optimization,” Acta Numerica, vol. 10, pp. 515–560, 2001.

C.-x. Xu, X.-l. He, and F.-m. Xu, “An effective continuous algorithm for approximate solutions of large scale max-cut problems,” Journal of Computational Mathematics, vol. 24, no. 6, pp. 749–760, 2006.

F. Alizadeh, J. P. Haeberly, M. V. Nayakkankuppam, M. L. Overton, and S. Schmieta, “SDPpack user’s guide-version 0.9Beta,” Tech. Rep. TR1997-737, Courant Institute of Mathematical Science, New York, NY, USA, June 1997.

J. F. Sturm, “Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones,” Optimization Methods and Software, vol. 11-12, no. 1–4, pp. 625–653, 1999.

C. Choi and Y. Ye, “Solving Sparse Semidefinite Programs Using the Dual Scaling Algorithm with an Iterative Solver,” Working Paper, Department of Management Science, University of Iowa, Iowa City, Iowa, USA, 2000.

G. Rinaldi, “Rudy graph generator,” http://www-user.tu-chemnitz.de/ helmberg/rudy.tar.gz

C. Helmberg and F. Rendl, “A spectral bundle method for semidefinite programming,” SIAM Journal on Optimization, vol. 10, no. 3, pp. 673–696, 2000.

H. Alperin and I. Nowak, “Lagrangian smoothing heuristics for max-cut,” Journal of Heuristics, vol. 11, no. 5-6, pp. 447–463, 2005.