AAA Abstract and Applied Analysis 1687-0409 1085-3375 Hindawi Publishing Corporation 598563 10.1155/2014/598563 598563 Research Article Proximal Alternating Direction Method with Relaxed Proximal Parameters for the Least Squares Covariance Adjustment Problem Xu Minghua 1 Zhang Yong 1 Huang Qinglong 1 Yang Zhenhua 2 Bnouhachem Abdellah 1 School of Mathematics and Physics Changzhou University, Jiangsu 213164 China cczu.edu.cn 2 College of Science Nanjing University of Posts and Telecommunications Jiangsu 210003 China njupt.edu.cn 2014 2112014 2014 13 06 2013 27 07 2013 21 1 2014 2014 Copyright © 2014 Minghua Xu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

We consider the problem of seeking a symmetric positive semidefinite matrix in a closed convex set to approximate a given matrix. This problem may arise in several areas of numerical linear algebra or come from finance industry or statistics and thus has many applications. For solving this class of matrix optimization problems, many methods have been proposed in the literature. The proximal alternating direction method is one of those methods which can be easily applied to solve these matrix optimization problems. Generally, the proximal parameters of the proximal alternating direction method are greater than zero. In this paper, we conclude that the restriction on the proximal parameters can be relaxed for solving this kind of matrix optimization problems. Numerical experiments also show that the proximal alternating direction method with the relaxed proximal parameters is convergent and generally has a better performance than the classical proximal alternating direction method.

1. Introduction

This paper concerns the following problem: (1)minX{12X-CF2XS+nSB}, where CRn×n is a given symmetric matrix, (2)S+n={XRn×nXT=X,X0},SB={XRn×nTr(AiX)=bi,i=1,2,,p,Tr(GjX)dj,j=1,2,,m}, matrices AiRn×n and GjRn×n are symmetric and scalars, bi and dj are the problem data, X0 denotes that X is a positive semidefinite matrix, Tr denotes the trace of a matrix, and ·F denotes the Frobenius norm; that is, (3)XF=(Tr(XTX))1/2=(i,j=1nXij2)1/2, and S+nSB is nonempty. Throughout this paper, we assume that the Slater’s constraint qualification condition holds so that there is no duality gap if we use Lagrangian techniques to find the optimal solution to problem (1).

Problem (1) is a type of matrix nearness problem, that is, the problem of finding a matrix that satisfies some properties and is nearest to a given one. Problem (1) can be called the least squares covariance adjustment problem or the least squares semidefinite programming problem and solved by many methods . In a least squares covariance adjustment problem, we make adjustments to a symmetric matrix so that it is consistent with prior knowledge or assumptions and a valid covariance matrix [2, 5, 6]. The matrix nearness problem has many applications especially in several areas of numerical linear algebra, finance industry, and statistics in . A recent survey of matrix nearness problems can be found in . It is clear that the matrix nearness problem considered here is a convex optimization problem. It thus follows from the strict feasibility and coercivity of the objective function that the minimum of (1) is attainable and unique.

In the literature of interior point algorithms, S+n is called the semidefinite cone and the related problem (1) belongs to the class of semidefinite programming (SDP) and second-order cone programming (SOCP) . In fact, it is possible to reformulate problem (1) into a mixed SDP and SOCP as in [3, 9]: (4)mints.t.Ai,X=bi,i=1,2,,p,s.t.Gj,Xdj,j=1,2,,m,s.t.tX-CF,s.t.XS+n, where X,Y=  Tr(XTY).

Thus, problem (1) can be efficiently solved by standard interior-point methods such as SeDuMi  and SDPT3  when the number of variables (i.e., entries in the matrix X) is modest, say under 1000 (corresponds to n around 32) and the number of equality and inequality constraints is not too large (say 5,000) [2, 3, 12].

Specially, let (5)SB={XRn×nDiag(X)=e}, where Diag(X) is the vector of diagonal elements of X and e is the vector of 1s. Then problem (1) can be viewed as the nearest correlation matrix problem. For the nearest correlation matrix problem, a quadratically convergent Newton algorithm was presented recently by Qi and Sun , and improved by Borsdorf and Higham . For problem (1) with equality and inequality constraints, one difficulty in finding an efficient method for solving this problem is the presence of the inequality constraints. In , Gao and Sun overcome this difficulty by reformulating the problem as a system of semismooth equations with two level metric projection operators and then design an inexact smoothing Newton method to solve the resulting semismooth system. For the problem (1) with large number of equality and inequality constraints, the numerical experiments in  show that the alternating direction method (hereafter alternating direction method is abbreviated as ADM) is more efficient in computing time than the inexact smoothing Newton method which additionally requires solving a large system of linear equations at each iteration. The ADM has many applications in solving optimization problems [15, 16]. Papers written by Zhang, Han, Li, Yuan, and Bauschke and Borwein show that the ADM can be applied to solve convex feasibility problems .

The proximal ADM is a class of ADM type methods which can also be easily applied to solve the matrix optimization problems. Generally, the proximal parameters (i.e., the parameters r and s in (14) and (15)) of the proximal ADM are greater than zero. In this paper, we will show that the restriction on the proximal parameters can be relaxed while the proximal ADM is used to solve problem (1). Numerical experiments also show that the proximal ADM with the relaxed proximal parameters generally has a better performance than the classical proximal ADM.

The paper is organized as follows. In Section 2, we give some preliminaries about the proximal alternating direction method. In Section 3, we convert the problem (1) to a structured variational inequality and apply the proximal ADM to solve it. The basic analysis and convergent results of the proximal ADM with relaxed proximal parameters are built in Section 4. Preliminary numerical results are reported in Section 5. Finally, we give some conclusions in Section 6.

2. Proximal Alternating Direction Method

In order to introduce the proximal ADM, we first consider the following structured variational inequality problem which includes two separable subvariational inequality problems: find (x,y)Ω such that (6)(x-x)Tf(x)0,(y-y)Tg(y)0,(x,y)Ω, where (7)Ω={(x,y)Ax+By=b,x𝒳,y𝒴},f:Rn1Rn1 and g:Rn2Rn2 are monotone; that is, (8)(x~-x)T(f(x~)-f(x))0,x~,xRn1,(y~-y)T(g(y~)-g(y))0,y~,yRn2,ARl×n1, BRl×n2, and bRl; 𝒳Rn1 and 𝒴Rn2 are closed convex sets. Studies of such variational inequality can be found in Glowinski , Glowinski and Le Tallec , Eckstein and Fukushima , He and Yang , He et al. , and Xu .

By attaching a Lagrange multiplier vector λRl to the linear constraint Ax+By=b, problem (6)-(7) can be explained as the following form (see [20, 21, 24]): find w=(x,y,λ)𝒲 such that (9)(x-x)T[f(x)-ATλ]0(y-y)T[g(y)-BTλ]0,w=(x,y,λ)𝒲,Ax+By-b=0, where (10)𝒲=𝒳×𝒴×Rl. For solving (9)-(10), Gabay  and Gabay and Mercier  proposed the ADM method. In the classical ADM method, the new iterate wk+1=(xk+1,yk+1,λk+1)𝒲 is generated from a given triple wk=(xk,yk,λk)𝒲 via the following procedure.

First, xk+1 is found by solving the following problem: (11)(x-x)T{f(x)-AT[λk-β(Ax+Byk-b)]}0,x𝒳, where x𝒳. Then, yk+1 is obtained by solving (12)(y-y)T{g(y)-BT[λk-β(Axk+1+By-b)]}0,y𝒴, where y𝒴. Finally, the multiplier is updated by (13)λk+1=λ-β(Axk+1+Byk+1-b), where β>0 is a given penalty parameter for the linearly constraint Ax+By-b=0. Most of the existing ADM methods require that the subvariational inequality problems (11)-(12) should be solved exactly at each iteration. Note that the involved subvariational inequality problem (11)-(12) may not be well-conditioned without strongly monotone assumptions on f and g. Hence, it is difficult to solve these subvariational inequality problems exactly in many cases. In order to improve the condition of solving the subproblem by the ADM, some proximal ADMs were proposed (see, e.g., [26, 27, 3034]). The classical proximal ADM is one of the attractive ADMs. From a given triple wk=(xk,yk,λk)𝒲, the classical proximal ADM produces the new iterate wk+1=(xk+1,yk+1,λk+1)𝒲 by the following procedure.

First, xk+1 is obtained by solving the following variational inequality problem: (14)(x-x)T{f(x)-AT[λk-β(Ax+Byk-b)]+r(x-xk)}0,x𝒳, where r>0 is the given proximal parameter and x𝒳. Then, yk+1 is found by solving (15)(y-y)T{(y-yk)g(y)-BT[λk-β(Axk+1+By-b)]+s(y-yk)}0,y𝒴, where s>0 is the given proximal parameter and y𝒴. Finally, the multiplier is updated by (16)λk+1=λk-β(Axk+1+Byk+1-b).

In this paper, we will conclude that problem (1) can be solved by the proximal ADM and the restriction on the proximal parameters r>0, s>0 can be relaxed as r>-1/2, s>-1/2 when the proximal ADM is applied to solve problem (1). Our numerical experiments later also show that the numerical performance of the proximal ADM with smaller value of proximal parameters is generally better than the proximal ADM with comparatively larger value of proximal parameters.

3. Converting Problem (<xref ref-type="disp-formula" rid="EEq1">1</xref>) to a Structured Variational Inequality

In order to solve the problem (1) with proximal ADM, we convert problem (1) to the following equivalent one: (17)minX,Y12X-CF2+12Y-CF2s.t.X-Y=0,s.tXS+n,YSB. Following the KKT condition of (17), the solution to (17) can be found by finding w=(X,Y,Λ)𝒲 such that (18)X-X,(X-C)-Λ0,Y-Y,(Y-C)+Λ0,w=(X,Y,Λ)𝒲,X-Y=0, where (19)𝒲=S+n×SB×Rn×n.

It is easy to see that problem (18)-(19) is a special case of the structured variational inequality (9)-(10) and thus can be solved by proximal ADM. For given wk=(Xk,Yk,Λk)𝒲, it is fortunate that the wk+1=(Xk+1,Yk+1,Λk+1) can be exactly obtained by the proximal ADM in the following way: (20)Xk+1=PS+n{11+β+r(C+rXk+βYk+Λk)},(21)Yk+1=PSB{11+β+s(C+βXk+1+sYk-Λk)},(22)Λk+1=Λk-β(Xk+1-Yk+1), where the projection of v on a nonempty closed convex set S of Rm×n under Frobenius norm, denoted by PS(v), is the unique solution to the following problem; that is, (23)PS(v)=  argminu{u-vF2uS}. It follows that the solution to (24)min{12Z-XF2ZS+n} is called the projection of X on S+n and denoted by PS+n(X). Using the fact that matrix Frobenius norm is invariant under unitary transform, it is known (see ) that (25)PS+n(X)=QΛ~QT, where (26)QTXQ=diag(λ1,,λn) is the symmetric Schur decomposition of X (Q=(q1,,qn) is an orthogonal matrix whose column vector qi,   i=1,,n, is the eigenvector of X, and λi,   i=1,,n, is the related eigenvalue), (27)Λ~=diag(λ~1,,λ~n),λ~i=max(λi,0). In order to obtain the projection PSB(X), we need to solve the following quadratic program:(28)minZ12Z-XF2s.t.  Tr(AiZ)=bi,i=1,2,,p,s.t.  Tr(GjZ)dj,j=1,2,,m. The dual problem of (28) can be written as (29)minv12vTHv+qTvs.t.vRp×R+m, where H is positive semidefinite and H and q have the following form, respectively:(30)H=(Tr(A1A1T)Tr(A1ApT)Tr(A1G1T)Tr(A1GmT)Tr(ApA1T)Tr(ApApT)Tr(ApG1T)Tr(ApGmT)Tr(G1A1T)Tr(G1ApT)Tr(G1G1T)Tr(G1GmT)Tr(GmA1T)Tr(GmApT)Tr(GmG1T)Tr(GmGmT)),q=(b1-Tr(A1X)bp-Tr(ApX)d1-Tr(G1X)dm-Tr(GmX)).

Problem (29) is often a medium-scale quadratic programming (QP) problem. A variety of methods for solving the QP are commonly used, including interior-point methods and active set algorithm (see [36, 37]).

Particularly, if SB is the following special case: (31)SB={XRn×nXT=X,HLXHU}, where H0 expresses that each element of H is nonnegative, HL and HU are given n×n symmetric matrices, and XHU means that HU-X0; then PSB(X) is easy to be carried out and is given by (32)PSB(X)=min(max(X,HL),HU), where max(X,Y) and min(X,Y) compute the element-wise maximum and minimum of matrix X and Y, respectively.

4. Main Results

Let {wk} be the sequence generated by applying the procedure (14)–(16) to problem (18)-(19); then for any w=(X,Y,Λ)𝒲, we have that (33)X-Xk+1,Xk+1-C-Λk+1-β(Yk-Yk+1)+r(Xk+1-Xk)0,Y-Yk+1,Yk+1-C+Λk+1+s(Yk+1-Yk)0,Λk+1=Λk-β(Xk+1-Yk+1). Further, letting (34)F(wk+1)=(Xk+1-C-Λk+1Yk+1-C+Λk+1Xk+1-Yk+1),d1(wk,wk+1)=(rIn000(s+β)In0001βIn)(Xk-Xk+1Yk-Yk+1Λk-Λk+1), where InRn×n is the unit matrix, and (35)d2(wk,wk+1)=F(wk+1)-β(In-In0)(Yk-Yk+1), then we can get the following lemmas.

Lemma 1.

Let {wk} be the sequence generated by applying the proximal ADM to problem (18)-(19) and let w*𝒲* be any solution to problem (18)-(19); then one has (36)wk+1-w*,d2(wk,wk+1)-Λk-Λk+1,Yk-Yk+1+Xk+1-X*F2+Yk+1-Y*F2.

Proof.

From (22) and (35), we have (37)wk+1-w*,d2(wk,wk+1)=-Λk-Λk+1,Yk-Yk+1+wk+1-w*,F(wk+1). Since (9) and w* are a solution to problem (18)-(19) and Xk+1S+n,  Yk+1SB, we have (38)wk+1-w*,F(w*)0. From (38), it follows that (39)wk+1-w*,F(wk+1)-F(wk+1)+F(w*)0. Thus, we have (40)wk+1-w*,F(wk+1)wk+1-w*,F(wk+1)-F(w*)=Xk+1-X*,Xk+1-X*-(Λk+1-Λ*)+Yk+1-Y*,Yk+1-Y*+(Λk+1-Λ*)+Λk+1-Λ*,Xk+1-X*-(Yk+1-Y*)=Xk+1-X*,Xk+1-X*+Yk+1-Y*,Yk+1-Y*=Xk+1-X*F2+Yk+1-Y*F2. Substituting (40) into (37), we get the assertion of this lemma.

Lemma 2.

Let {wk} be the sequence generated by applying the proximal ADM to problem (18)-(19) and let w*𝒲* be any solution to problem (18)-(19); then one has (41)wk-w*,G0(wk-wk+1)wk-wk+1,G0(wk-wk+1)-Λk-Λk+1,Yk-Yk+1+Xk+1-X*F2+Yk+1-Y*F2, where (42)G0=(rIn000(s+β)In0001βIn).

Proof.

It follows from (33) that (43)w-wk+1,d2(wk,wk+1)-d1(wk,wk+1)0,w𝒲. Thus, we have (44)wk+1-w*,d1(wk,wk+1)wk+1-w*,d2(wk,wk+1)-Λk-Λk+1,Yk-Yk+1+Xk+1-X*F2+Yk+1-Y*F2. From the above inequality, we get (45)wk-w*,G0(wk-wk+1)wk-wk+1,G0(wk-wk+1)-Λk-Λk+1,Yk-Yk+1+Xk+1-X*F2+Yk+1-Y*F2. Hence, (41) holds and the proof is completed.

Theorem 3.

Let {wk} be the sequence generated by applying the proximal ADM to problem (18)-(19) and let w*𝒲* be any solution to problem (18)-(19); then one has (46)wk+1-w*G2wk-w*G2-wk-wk+1,M(wk-wk+1), where (47)G=((r+1)In000(1+s+β)In0001βIn),M=((12+r)In000(12+s+β)In-In0-In1βIn), and wG2=w,Gw.

Proof.

From (41), we have (48)wk+1-w*G02=wk-w*-(wk-wk+1)G02wk-w*G02-2wk-wk+1G02+2Λk-Λk+1,Yk-Yk+1-2Xk+1-X*F2-2Yk+1-Y*F2+wk-wk+1G02=wk-w*G02-wk-wk+1G02+2Λk-Λk+1,Yk-Yk+1-2Xk+1-X*F2-2Yk+1-Y*F2. Rearranging the inequality above, we find that(49)wk+1-w*G2wk-w*G2-wk-wk+1,(rIn000(s+β)In-In0-In1βIn)(wk-wk+1)-(Xk+1-X*F2+Xk-X*F2)-(Yk+1-Y*F2+Yk-Y*F2).Using the Cauchy-Schwarz Inequality on the last term of the right-hand side of (49), we obtain (50)Xk+1-X*F2+Xk-X*F212Xk+1-XkF2,Yk+1-Y*F2+Yk-Y*F212Yk+1-YkF2. Substituting (50) into (49), we get (51)wk+1-w*G2wk-w*G2-wk-wk+1,M(wk-wk+1). Thus, the proof is completed.

Based on the Theorem 3, we get the following lemma.

Lemma 4.

Let {wk} be the sequence generated by applying proximal ADM to problem (18)-(19), w*𝒲* any solution to problem (18)-(19), r>-1/2, and s>-1/2; then one has the following.

The sequence {wk-w*G2} is nonincreasing;

The sequence {wk} is bounded;

limkwk+1-wkF2=0;

G and M are both symmetric positive-definite matrices.

Proof.

Since (52)|(12+s+β)In-In-In1βIn|=((1/2)+s)β, it is easy to check that if r>-1/2, s>-1/2, then G and M are symmetric positive-definite matrices.

Let τ>0 be the smallest eigenvalue of matrix M. Then, from (46), we have (53)wk+1-w*G2wk-w*G2-τwk-wk+1F2. Following (53), we immediately have that wk-w*G2 is nonincreasing and thus the sequence {wk} is bounded. Moreover, we have (54)wk+1-w*G2w0-w*G2-τj=0kwj-wj+1F2. So, we get (55)j=0kwj-wj+1F2<,k>0, then (56)limkwk-wk+1F2=0. Thus, the proof is completed.

Following Lemma 4, now we are in the stage of giving the main convergence results of proximal ADM with r>-1/2 and s>-1/2 for problem (18)-(19).

Theorem 5.

Let {wk} be the sequence generated by applying proximal ADM to problem (18)-(19), r>-1/2, and s>-1/2; then {wk} converges to a solution point of (18)-(19).

Proof.

Since the sequence {wk} is bounded (see point (2) of Lemma 4), it has at least one cluster point. Let w be a cluster point of {wk} and the subsequence {wkj} converges to w. It follows from (33) that (57)limjX-Xkj+1,Xkj+1-C-Λkj+1-β(Ykj-Ykj+1)+r(Xkj+1-Xkj)0,limjY-Ykj+1,Ykj+1-C+Λkj+1+s(Ykj+1-Ykj)0,w𝒲,limjΛkj+1=Λkj-β(Xkj+1-Ykj+1). Following point (3) of Lemma 4, we have (58)X-X,X-C-Λ0,Y-Y,Y-C+Λ0,w𝒲,X-Y=0. This means that w is a solution point of (18)-(19). Since {wkj} converges to w, we have that, for any given ε>0, there exists an integer N>0 such that (59)wkj-wG2<ε,kjN. Furthermore, using the inequality (53), we have (60)wk-wG2<wkj-wG2,kkj. Combining (59) and (60), we get that (61)wk-wG2<ε,k>N. This implies that the sequence {wk} converges to w. So the proof is completed.

5. Numerical Experiments

In this section, we implement the proximal ADM to solve the problem (1) and show the numerical performances of proximal ADM with different proximal parameters. Additionally, we compare the classical ADM (i.e., the proximal ADM with proximal parameters r=0 and s=0) with the alternating projections method proposed by Higham  numerically and show that the alternating projections method is not equivalent to proximal ADM with zero proximal parameters. All the codes were written in Matlab 7.1 and run on IBM notebook PC R400.

Example 6.

In the first numerical experiment, we set the C1 as an n×n matrix whose entries are generated randomly in [-1,1]. Let C=(C1+C1T)/2 and further let the diagonal elements of C be 1 that is, Cii=1, i=1,2,,n. In this test example, we simply let SB be in the form of (31) and (62)HL=(lij)Rn×n,lij={-0.5,ij1,i=j,i,j=1,2,,n,HU=(uij)Rn×n,uij={0.5,ij1,i=j,i,j=1,2,,n. Moreover, let X0=eye(n), Y0=eye(n), Λ0=zeroes(n), β=4, and ε=10-6, where eye(n) and zeroes(n) are both the Matlab functions. For different problem size n and different proximal parameters r and s, Table 1 shows the computational results. There, we report the number of iterations (It.) and the computing time in seconds (CPU.) it takes to reach convergence. The stopping criterion of the proximal ADM is (63)wk+1-wkmax<ε, where Xmax=max(max(abs(X))) is the maximum absolute value of the elements of the matrix X.

Numerical results of Example 6.

n r = - 0.3 , s=-0.3 r = 0 , s=0 r = 3 , s=3
It. CPU. It. CPU. It. CPU.
100 31 0.292 34 0.331 72 0.764
200 33 1.346 39 1.570 84 3.364
300 38 4.265 41 5.746 90 9.991
400 40 9.872 43 9.919 94 22.03
500 39 15.83 45 18.39 98 39.91
Remark 7.

Note that if the proximal parameters are equal to zero, that is, r=0 and s=0, then the proximal ADM is the classical ADM.

Example 8.

All the data are the same as in Example 6 except that C1 is an n×n matrix whose entries are generated randomly in [-1000,1000], (64)HL=(lij)Rn×n,lij={-500,ij1000,i=j,i,j=1,2,,n,HU=(uij)Rn×n,uij={500,ij1000,i=j,i,j=1,2,n. The computational results are reported in Table 2.

Numerical results of Example 8.

n r = - 0.3 , s=-0.3 r = 0 , s=0 r = 3 , s=3
It. CPU. It. CPU. It. CPU.
100 49 0.476 54 0.551 116 1.837
200 51 2.197 57 2.334 128 5.430
300 59 6.614 61 8.108 136 15.25
400 56 12.74 63 14.51 140 31.65
500 58 23.90 66 26.90 147 59.98
Example 9.

Let SB be in the form of (31) and lij=0, uij=+, i,j=1,2,,n. Assume that C, X0, Y0, Λ0, β, ε, and the stopping criterion are the same as those in Example 6, but the diagonal elements of matrix C are replaced by (65)Cii=α+(1-α)×rand,i=1,2,,n, where α(0,1) is a given number, rand is the Matlab function generating a number randomly in [0,1]. In the following numerical experiments, we let α=0.2. For different problem size n and different proximal parameters r and s, Table 3 shows the number of iterations and the computing time in seconds it takes to reach convergence.

Numerical results of Example 9.

n r = - 0.3 , s=-0.3 r = 0 , s=0 r = 3 , s=3
It. CPU. It. CPU. It. CPU.
100 32 0.282 35 0.288 70 0.566
200 33 1.295 36 1.397 72 4.006
300 34 3.745 37 4.156 73 8.285
400 34 7.885 37 8.571 73 16.73
500 34 14.07 37 15.42 74 29.87
Example 10.

All the data are the same as in Example 9 except that α=0. The computational results are reported in Table 4.

Numerical results of Example 10.

5 r = - 0.3 , s=-0.3 r = 0 , s=0 r = 3 , s=3
It. CPU. It. CPU. It. CPU.
100 32 0.259 35 0.300 70 0.557
200 33 1.306 36 1.424 72 2.880
300 33 3.750 37 4.087 72 7.958
400 34 7.799 37 8.546 74 16.98
500 34 13.96 37 16.10 74 30.77
Example 11.

Let C1 be an n×n matrix whose entries are generated randomly in [-0.5,0.5], C=(C1+C1T)/2, and let the diagonal elements of C be 1. And let(66)SB={XRn×nX=XT,Xij=eij,(i,j)e,Xijlij,(i,j)l,Xijuij,(i,j)uRn×n}, where e,l,u are subsets of {(i,j)1i,jn} denoting the indexes of such entries of X that are constrained by equality, lower bounds, and upper bounds, respectively. In this test example, we let the index sets e,l, and u be the same as in Example 5.4 of ; that is, e={(i,i)i=1,2,,n} and l,u{(i,j)1i<jn} consist of the indices of min(n^r,n-i) randomly generated elements at the ith row of X, i=1,2,,n with n^r=5 and n^r=10, respectively. We take eii=1 for (i,i)e, lij=-0.1 for (i,j)l, and uij=0.1 for (i,j)u.

Moreover, let X0, Y0, Λ0, β, ε, and the stopping criterion be the same as those in Example 6. For different problem size n, different proximal parameters r and s, and different values of n^r, Tables 5(a) and 5(b) show the number of iterations and the computing time in seconds it takes to reach convergence, respectively.

Numerical experiments show that the proximal ADM with relaxed parameters is convergent. Moreover, we draw the conclusion that the proximal ADM with smaller value of proximal parameters generally converges more quickly than the proximal ADM with comparatively larger value of proximal parameters to solve the problem (1).

(a) Numerical results of Example 11 with n^r=5. (b) Numerical results of Example 11 with n^r=10.

n    r = - 0.3 , s=-0.3 r = 0 , s=0 r = 1 , s=1
It. CPU. It. CPU. It. CPU.
100 22 0.293 25 0.354 34 0.448
200 25 2.119 28 2.425 40 3.436
300 27 7.141 30 8.024 44 11.64
400 29 17.40 31 18.59 46 27.32
500 30 34.17 33 37.45 48 53.84
n r = - 0.3 , s=-0.3 r = 0 , s=0 r = 1 , s=1
It. CPU. It. CPU. It. CPU.
100 23 0.309 25 0.342 33 0.439
200 24 2.029 27 2.305 38 3.162
300 27 7.150 29 7.801 42 11.29
400 28 16.68 31 18.47 45 26.60
500 29 32.73 32 36.37 47 53.06
Example 12.

In this test example, we apply the proximal ADM with r=0, s=0 (i.e., the classical ADM) to solve the nearest correlation matrix problem, that is, problem (1) with SB in the form of (5), and compare the classical ADM numerically with the alternating projections method (APM) . The APM computes the nearest correlation matrix to a symmetric CRn×n by the following process:

ΔS0=0, Y0=C;

for k=1,2,

Rk=Yk-1-ΔSk-1;

Xk=PS+n(Rk);

ΔSk=Xk-Rk;

Yk=PSB(Xk);

end.

In this numerical experiment, the stopping criterion of the APM is (67)max{Xk-Xk-1max,Yk-Yk-1max,Xk-Ykmax}<ε.

Let the matrix C and the initial parameters of classical ADM be the same as those in Example 6. Table 6(a) reports the numerical performance of proximal ADM and the APM for computing the nearest correlation matrix to C.

Further, let C1 be an n×n matrix whose entries are generated randomly in [0,1] and C=(C1+C1T)/2. The other data are the same as above. Table 6(b) reports the numerical performance of the classical ADM and the APM for computing the nearest correlation matrix to the matrix C. Numerical experiments show that the classical ADM generally exhibits a better numerical performance than the APM for the test problems above.

(a) Numerical results of Example 12. (b) Numerical results of Example 12.

It. CPU. It. CPU.
100 28 0.381 47 0.743
200 33 2.878 59 5.443
300 36 9.462 70 20.68
400 38 22.50 81 54.38
500 39 43.32 89 114.7
It. CPU. It. CPU.
100 27 0.634 42 0.582
200 30 2.590 59 5.428
300 32 8.524 65 19.36
400 34 20.34 75 50.79
500 35 39.43 86 111.6
6. Conclusions

In this paper, we apply the proximal ADM to a class of matrix optimization problems and find that the restriction of proximal parameters can be relaxed. Moreover, numerical experiments show that the proximal ADM with relaxed parameters generally has a better numerical performance in solving the matrix optimization problem than the classical proximal alternating direction method.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

The authors thank the referees very sincerely for their valuable suggestions and careful reading of their paper. This research is financially supported by a research Grant from the Research Grant Council of China (Project no. 10971095).

Borsdorf R. Higham N. J. A preconditioned Newton algorithm for the nearest correlation matrix IMA Journal of Numerical Analysis 2010 30 1 94 107 10.1093/imanum/drn085 MR2580548 ZBL1188.65055 Boyd S. Xiao L. Least-squares covariance matrix adjustment SIAM Journal on Matrix Analysis and Applications 2005 27 2 532 546 10.1137/040609902 MR2179687 ZBL1099.65039 Gao Y. Sun D. Calibrating least squares semidefinite programming with equality and inequality constraints SIAM Journal on Matrix Analysis and Applications 2009 31 3 1432 1457 10.1137/080727075 MR2587785 Gravel S. Elser V. Divide and concur: a general approach to constraint satisfaction Physical Review E 2008 78 036706 Higham N. J. Computing a nearest symmetric positive semidefinite matrix Linear Algebra and its Applications 1988 103 103 118 10.1016/0024-3795(88)90223-6 MR943997 ZBL0649.65026 Higham N. J. Computing the nearest correlation matrix—a problem from finance IMA Journal of Numerical Analysis 2002 22 3 329 343 10.1093/imanum/22.3.329 MR1918653 ZBL1006.65036 Higham N. J. Gover M. Barnett S. Matrix nearness problems and applications Applications of Matrix Theory 1989 22 Oxford, UK Oxford University Press 1 27 MR1041063 ZBL0681.65029 Boyd S. Vandenberghe L. Convex Optimization 2004 Cambridge, UK Cambridge University Press xiv+716 MR2061575 Vandenberghe L. Boyd S. Semidefinite programming SIAM Review 1996 38 1 49 95 10.1137/1038003 MR1379041 ZBL0845.65023 Sturm J. F. Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones Optimization Methods and Software 1999 11/12 1–4 625 653 10.1080/10556789908805766 MR1778433 ZBL0973.90526 Tütüncü R. H. Toh K. C. Todd M. J. Solving semidefinite-quadratic-linear programs using SDPT3 Mathematical Programming 2003 95 2 189 217 10.1007/s10107-002-0347-5 MR1976479 ZBL1030.90082 Malick J. A dual approach to semidefinite least-squares problems SIAM Journal on Matrix Analysis and Applications 2004 26 1 272 284 10.1137/S0895479802413856 MR2112861 ZBL1080.65027 Qi H. Sun D. A quadratically convergent Newton method for computing the nearest correlation matrix SIAM Journal on Matrix Analysis and Applications 2006 28 2 360 385 10.1137/050624509 MR2255334 ZBL1120.65049 He B. Xu M. Yuan X. Solving large-scale least squares semidefinite programming by alternating direction methods SIAM Journal on Matrix Analysis and Applications 2011 32 1 136 152 10.1137/090761549 MR2811295 ZBL1243.49039 Pardalos P. M. Resende M. G. C. Handbook of Applied Optimization 2002 Oxford, UK Oxford University Press xx+1095 MR2055796 Pardalos P. M. Rassias T. M. Khan A. A. Nonlinear Analysis and Variational Problems 2010 35 New York, NY, USA Springer xxviii+490 Springer Optimization and Its Applications In honor of George Isac, Edited by Panos M. Pardalos, Themistocles M. Rassias and Akhtar A. Khan 10.1007/978-1-4419-0158-3 MR2590601 Bauschke H. H. Borwein J. M. On projection algorithms for solving convex feasibility problems SIAM Review 1996 38 3 367 426 10.1137/S0036144593251710 MR1409591 ZBL0865.47039 Zhang W. Han D. Li Z. A self-adaptive projection method for solving the multiple-sets split feasibility problem Inverse Problems 2009 25 11 10.1088/0266-5611/25/11/115001 MR2545996 ZBL1185.65102 Zhang W. Han D. Yuan X. An efficient simultaneous method for the constrained multiple-sets split feasibility problem Computational Optimization and Applications 2012 52 3 825 843 10.1007/s10589-011-9429-8 MR2950507 ZBL06115315 Glowinski R. Numerical Methods for Nonlinear Variational Problems 1984 New York, NY, USA Springer xv+493 MR737005 Glowinski R. Le Tallec P. Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics 1989 9 Philadelphia, Pa, USA SIAM x+295 SIAM Studies in Applied Mathematics 10.1137/1.9781611970838 MR1060954 Eckstein J. Some saddle-function splitting methods for convex programming Optimization Methods and Software 1994 4 75 83 Eckstein J. Fukushima M. Hager W. W. Hearn D. W. Pardalos P. M. Some reformulations and applications of the alternating direction method of multipliers Large Scale Optimization: State of the Art 1994 Dordrecht, The Netherlands Kluwer Academic Publishers 115 134 MR1307168 ZBL0816.90109 Fukushima M. Application of the alternating direction method of multipliers to separable convex programming problems Computational Optimization and Applications 1992 1 1 93 111 10.1007/BF00247655 MR1195631 ZBL0763.90071 He B. Yang H. Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities Operations Research Letters 1998 23 3-5 151 161 10.1016/S0167-6377(98)00044-3 MR1677664 ZBL0963.49006 He B. Liao L.-Z. Han D. Yang H. A new inexact alternating directions method for monotone variational inequalities Mathematical Programming 2002 92 1 103 118 10.1007/s101070100280 MR1892298 ZBL1009.90108 Xu M. H. Proximal alternating directions method for structured variational inequalities Journal of Optimization Theory and Applications 2007 134 1 107 117 10.1007/s10957-007-9192-2 MR2341497 ZBL1129.49040 Gabay D. Fortin M. Glowinski R. Applications of the method of multipliers to variational inequalities Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems 1983 Amsterdam, The Netherlands North-Holland 299 331 Gabay D. Mercier B. A dual algorithm for the solution of nonlinear variational problems via finite element approximations Computer and Mathematics with Applications 1976 2 17 40 Güler O. New proximal point algorithms for convex minimization SIAM Journal on Optimization 1992 2 4 649 664 10.1137/0802032 MR1186167 ZBL0778.90052 Hager W. W. Zhang H. Asymptotic convergence analysis of a new class of proximal point methods SIAM Journal on Control and Optimization 2007 46 5 1683 1704 10.1137/060666627 MR2361989 ZBL1154.90009 Hager W. W. Zhang H. Self-adaptive inexact proximal point methods Computational Optimization and Applications 2008 39 2 161 181 10.1007/s10589-007-9067-3 MR2373244 ZBL1147.90414 Rockafellar R. T. Monotone operators and the proximal point algorithm SIAM Journal on Control and Optimization 1976 14 5 877 898 MR0410483 10.1137/0314056 ZBL0358.90053 Teboulle M. Convergence of proximal-like algorithms SIAM Journal on Optimization 1997 7 4 1069 1083 10.1137/S1052623495292130 MR1479615 ZBL0890.90151 Glunt W. K. An alternating projections method for certain linear problems in a Hilbert space IMA Journal of Numerical Analysis 1995 15 2 291 305 10.1093/imanum/15.2.291 MR1323742 ZBL0827.65064 Nocedal J. Wright S. J. Numerical Optimization 1999 New York, NY, USA Springer xxii+636 10.1007/b98874 MR1713114 Narendra N. A new polynomial time algorithm for linear programming Combinatorica 1987 4 373 395