JAM Journal of Applied Mathematics 1687-0042 1110-757X Hindawi Publishing Corporation 503242 10.1155/2012/503242 503242 Research Article A Proximal Analytic Center Cutting Plane Algorithm for Solving Variational Inequality Problems Shen Jie 1 Pang Li-Ping 2 Yao Jen Chih 1 School of Mathematics Liaoning Normal University Dalian 116029 China lnnu.edu.cn 2 School of Mathematical Sciences Dalian University of Technology Dalian 116024 China dlut.edu.cn 2012 30 12 2012 2012 26 07 2012 19 12 2012 2012 Copyright © 2012 Jie Shen and Li-Ping Pang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Under the condition that the values of mapping F are evaluated approximately, we propose a proximal analytic center cutting plane algorithm for solving variational inequalities. It can be considered as an approximation of the earlier cutting plane method, and the conditions we impose on the corresponding mappings are more relaxed. The convergence analysis for the proposed algorithm is also given at the end of this paper.

1. Introduction

According to , the history of algorithms for solving finite-dimensional variational inequalities is relatively short. A recent development of such methods is the analytic center method based on cutting plane methods. It combines the feature of the newly developed interior point methods with the classical cutting plane scheme to achieve polynomial complexity in theory and quick convergence in practice. More details can be found in [2, 3]. Specifically, Goffin et al.  developed a convergent framework for finding a solution x* of the variational inequality VIP(F,X) associated with the continuous mapping F from X to Rn and the polyhedron X={xRn  |  Axb} under an assumption slightly stronger than pseudomonotonicity. Again, Marcotte and Zhu  extended this algorithm to quasimonotone variational inequalities that satisfy a weak additional assumption. Such methods are effective in practice.

Note that the facts in optimization problems, see , some functions from Rn to R are themselves defined through other minimization problems. For example, consider the Lagrangian relaxation, see , the primal problem is (1.1)max{q(ξ)ξP,h(ξ)=0}, where P is a compact subset of Rm and q:RmR, h:RmRn are two functions. Lagrangian relaxation in this problem leads to the problem min{f(x)  |  xRn}, where (1.2)f(x)=maxξP{q(ξ)+x,h(ξ)} is the dual function. Trying to solve problem (1.1) by means of solving its dual problem min{f(x)  |  xRn} becomes more difficult since in this case evaluating the function value f(x) requires solving exactly another optimization problem (1.2). Let us see another example: consider the problem (1.3)min{f(x)xC}, where f is convex (not necessarily differentiable), CRn is a nonempty closed convex set, F is called the Moreau-Yosida regularization of f on C, that is, (1.4)F(x)=minzC{f(z)+12αz-x2}, where α is a positive parameter. A point xC is a solution to (1.3) if and only if it is a solution to the problem: (1.5)minxRnF(x). The problem (1.5) is easier to deal with than (1.3), see . But in this case, computing the exact function value of F at an arbitrary point x is difficult or even impossible since F itself is defined through a minimization problem involving another function f. Intuitively, we consider the approximate computation of function F.

The above-mentioned phenomenon also exists for mappings from X to Y, where X and Y are two subspaces of any two finite-dimensional spaces, respectively. Once a mapping, and more specifically, a continuous mapping is defined implicitly rather than explicitly, the approximation of the mapping becomes inevitable, see . In this paper we try to solve VIP(F,X) by assuming the values of the mapping F from X to Rn can be only computed approximately. Under the assumption, we construct an algorithm for solving the approximate variational inequality problem AVIP(F,X) and we also prove that there exists a cluster point of the iteration points generated by the proposed algorithm, it is a solution to the original problem VIP(F,X).

This paper is organized as follows. Some basic concepts and results are introduced in Section 2. In Section 3, a proximal analytic center cutting plane algorithm for solving the variational inequality problems is given. The convergence analysis of the proposed algorithm is addressed in Section 4. In the last section, we give some conclusions.

2. Basic Concepts and Results

Let X={xRnAxb} be a polyhedron and F a continuous mapping from X to Rn. A vector x* is a solution to the variational inequality VIP(F,X) if and only if it satisfies the system of nonlinear inequalities: (2.1)F(x*),x-x*0,xX. The vector x* is a solution to the dual variational inequality VID(F,X) of VIP(F,X) if and only if it satisfies (2.2)F(x),x-x*0,xX. We denote by XP* the solution set of VIP(F,X), and XD* the solution set of VID(F,X), respectively. Whenever F is continuous, we have XD*XP*, see . If F is pseudomonotone on X, then XD*=XP*, see . If F is quasimonotone at x*XP* and F(x*) is not normal to X at x*, then XD* is nonempty, see Proposition 1 in . For the definitions of monotone, pseudomonotone and quasimonotone, see [5, 15].

Definition 2.1.

The gap functions gP(x) and gD(x) of VIP(F,X) and VID(F,X) are, respectively, defined by (2.3)gP(x)=maxyXF(x),x-y,gD(x)=maxyXF(y),x-y. Note that gP(x)0,gD(x)0, and gP(x*)=0 if and only if x* is a solution to VIP(F,X), gD(x*)=0 if and only if x* is a solution to VID(F,X). Thus, XP*={xXgP(x)=0},XD*={xXgD(x)=0}.

Definition 2.2.

A point x~X is called an ε-solution to VIP(F,X) if gP(x~)ε for given ε>0.

Definition 2.3.

For x,yX, we say F(x)F(y) if and only if Fi(x)Fi(y), for i=1,2,,n, where F(x)=(F1(x),F2(x),,Fn(x))T.

Assumptions.

Throughout this paper, we make the following assumptions: for each x,yX, given any ε-=(ε,ε,,ε), δ-=(δ,δ,,δ), where ε,δ(0,1), we can always find a F-xRn and a F-yRn such that (2.4)(a)      F(x)F-xF(x)+ε-,  F(y)F-yF(y)+δ-,that  is,wecancomputetheapproximatevalueofFwithanyprecision;(b)        F-yF-xifyx,nomattertherelationshipbetweenε-andδ-;(c)        F-y-F-xLy-x,whereL>0isaconstant. These assumptions are realistic in practice, see [16, 17]. By using the given architecture in [16, 17], we can approximate the mapping F arbitrary well since neural networks are capable of approximating any function from one finite-dimensional real vector space to another one arbitrary well, see . Specifically, let us consider the case of univariate function. If f is a min-type function of the form (2.5)f(x)=inf{Nz(x)zZ}, where each Nz(x) is convex and Z is an infinite set, then it may be impossible to calculate f(x). However, we may still consider two cases. In the first case of controllable accuracy, for each positive ε>0 one can find an ε-minimizer of (2.5), that is, an element zxZ satisfying Nzx(x)f(x)+ε; in the second case, this may be possible only for some fixed (any possibly unknown) ε<. In both cases, we may set f-x=Nzx(x)f(x)+ε. A special case of (2.5) arises from Lagrangian relaxation , where the problem max{f(x)xS} with S=R+n is the Lagrangian dual of the primal problem (2.6)infψ0(x)s.t.ψj(x)0,j=1,2,,n,xX, with Nz(x)=ψ0(z)+x,ψ(z) for ψ=(ψ1,ψ2,,ψn). Then, for each multiplier x0, we need only to find zxZ such that f-x=Nzx(x)f(x)+ε, see .

Under the above assumptions (2.4), we introduce an approximate problem AVIP(F,X) associated with VIP(F,X): finding x*X such that (2.7)F-x*,x-x*0,xX, where F-x* satisfies F(x*)F-x*F(x*)+ε- for arbitrary ε-0. Its dual problem AVID(F,X) is to find x*X such that (2.8)F-x,x-x*0,xX, where F-x satisfies F(x)F-xF(x)+ε- for arbitrary ε-0.

Definition 2.5.

The gap function of AVIP(F,X) is defined by g-P(x)=maxyXF-x,x-y.

Definition 2.6.

A point x~X is called an ε-solution to AVIP(F,X) if g-P(x~)ε for given ε>0.

The optimal solution sets of AVIP(F,X) and AVID(F,X) are denoted by AXP* and AXD*, respectively. The following proposition ensures that AXD* is nonempty.

Proposition 2.7.

If there exists a point x*AXP* such that (2.9)F-x*,y-x*>0F-y,y-x*0,yX, and F-x* is not normal to X at x*, then AXD* is nonempty.

Proof.

Since F-x* is not normal to X at x*, there exists a point x0X such that F-x*,x0-x*>0.   xX, set xt=tx0+(1-t)x for t(0,1], then we have F-x*,x-x*0 and F-x*,xt-x*>0. Note the condition (2.9), we obtain F-xt,xt-x*0. Letting t0, it follows from the condition (b) in (2.4) that F-x,x-x*0, that is, x*AXD*.

In the following part, we focus our attention on solving AVIP(F,X). Let Γ(y,x):Rn×RnRn denote an auxiliary mapping, continuous in x and y, strongly monotone in y, that is, (2.10)Γ(y,x)-Γ(z,x),y-zβy-z2,y,zX, for some β>0. We consider the auxiliary variational inequality associated with Γ, whose solution w(x) satisfies (2.11)Γ(w(x),x)-Γ(x,x)+F-x,y-w(x)0,yX. In view of the strong monotonicity of Γ(y,x) with respect to y, this auxiliary variational inequality has a unique solution w(x).

Proposition 2.8.

The mapping w:XX is continuous on X. Furthermore, x is a solution to AVIP(F,X) if and only if x=w(x).

Proof.

The first part of the proposition follows from Theorem 5.4 in . To prove the second part, we first suppose that x=w(x). This yields F-x,y-x0,  yX, that is, x solves AVIP(F,X). Conversely, suppose that x solves AVIP(F,X), then (2.12)F-x,w(x)-x0, and from (2.11), we have (2.13)Γ(w(x),x)-Γ(x,x)+F-x,x-w(x)0. Adding the two preceding inequalities, one obtains (2.14)Γ(w(x),x)-Γ(x,x),x-w(x)0, and we conclude, from the strong monotonicity of Γ with respect to y, that x=w(x).

Let ρ<1,α<β be two positive numbers. Let l be the smallest nonnegative integer satisfying (2.15)F-x+ρl(w(x)-x),x-w(x)αw(x)-x2, where F-x+ρl(w(x)-x) satisfies F(x+ρl(w(x)-x))F-x+ρl(w(x)-x)F(x+ρl(w(x)-x))+ε- for arbitrary ε-0. The existence of a finite l will be proved in Proposition 2.9. The composite mapping G is defined, for every xX, by (2.16)G(x)=G-x=F-x+ρl(w(x)-x). If x* is a solution to AVIP(F,X), then we have w(x*)=x*, l=0 and G-x*=F-x*.

Proposition 2.9.

The operator G is well defined for every xX. Moreover, we have (2.17)lln((β-α)/L)lnρ, where L is the number given in (2.4)-(c).

Proof.

From the definition of w(x), we have (2.18)F-x,x-w(x)Γ(w(x),x)-Γ(x,x),w(x)-xβx-w(x)2. Suppose (2.15) does not hold for any finite integer l, that is, (2.19)F-x+ρl(w(x)-x),x-w(x)<αx-w(x)2. Note the assumption (2.4)-(b), we obtain (2.20)F-x+ρl(w(x)-x)F-xasl+, therefore, (2.21)F-x,x-w(x)αx-w(x)2. Since α<β, (2.21) is in contradiction with (2.18). To prove the second part, we notice that (2.22)F-x+ρl(w(x)-x),x-w(x)=F-x,x-w(x)+F-x+ρl(w(x)-x)-F-x,x-w(x)βw(x)-x2-Lρlw(x)-x2αw(x)-x2 if αβ-Lρl, which means the second conclusion of Proposition 2.9 holds.

Proposition 2.10.

If xAXP*, then y*AXD*, we have (2.23)G-x,x-y*>0.

Proof.

Let y(x)=x+ρl(w(x)-x), then G-x=F-y(x) and (2.24)F-y(x),w(x)-x-αw(x)-x2. Since xAXP*, so w(x)x. Therefore, (2.25)F-y(x),y(x)-x=ρlF-y(x),w(x)-x<0. For all y*AXD*, there holds (2.26)F-y(x),y(x)-y*0. By combining (2.25) with (2.26), we obtain F-y(x),x-y*>0, that is, G-x,x-y*>0.

3. A Proximal Analytic Center Cutting Plane Algorithm

Algorithm 3.1 offered in this section is a modification of the algorithm in . Algorithm 3.1 is described as follows.

Algorithm 3.1.

Let β be the strong monotonicity constant of Γ(x,y) with respect to y, and let α(0,β), ε(0,1) be two constants. Set k=0,  Ak=ARm×n,  bk=b, and εk=ε.

Step 1 (computation of the center). Find an approximate analytic center xk of Xk={xRnAkxbk}.

Step 2 (stopping criterion). If gP(xk)ε, stop.

Step 3 (solving the approximate auxiliary variational inequality problem). Find w(xk), such that (3.1)F-xk+Γ(w(xk),xk)-Γ(xk,xk),y-w(xk)0,yX, where F-xk satisfies F(xk)F-xkF(xk)+ε-xk, ε-xk=(εk,εk,,εk)T.

Step 4 (construction of the approximate cutting plane). Let yk=xk+ρlk(w(xk)-xk) and G-xk=F-yk, where lk is the smallest integer that satisfies (3.2)F-xk+ρlk(w(xk)-xk),xk-w(xk)αw(xk)-xk2, where F-xk+ρlk(w(xk)-xk) satisfies F(xk+ρlk(w(xk)-xk))F-xk+ρlk(w(xk)-xk)F(xk+ρlk(w(xk)-xk))+ε-xk+ρlk(w(xk)-xk),ε-xk+ρlk(w(xk)-xk)=(εk,εk,,εk)T.

Let Hk={xG-xk,x-xk=0}, (3.3)Ak+1=(AkG-xk),bk+1=(bkG-xkTxk).

Increase k by one and go to Step 1.

End of Algorithm 3.1

4. Convergence Analysis

In , the authors proposed a column generation scheme to generate the polytope Xk, and they proved if k satisfies the following inequality (4.1)ε2m1/2+2mln(1+(k+1)/8m2)2m+k+1e-2α((k+1)/(2m+k+1)), where ε<1/2 is a constant, the scheme will stop with a feasible solution, that is, they can find a vector ak+1 such that {yak+1Tyak+1Tyk}Γ with ||ak+1||=1, Γ contains a full-dimensional closed ball with ε<1/2 radius. In other words, there exists the smallest k(ε) such that Xk(ε) generated by the column generation scheme does not contain the ball with ε<1/2 radius, and it is known as the finite cut property. It is easy to know that the result of Theorem 6.6 in  also holds without much change for our Algorithm 3.1 using the approximate centers. That is, by using the row generation scheme, there exists the smallest k(ρ) such that Xk(ρ) generated in Step 4 in Algorithm 3.1 does not contain the ball with ρ radius lying inside the polytope X. This result plays an important role in proving the convergence of the described Algorithm 3.1 in Section 3.

Theorem 4.1.

Let the polyhedron X have nonempty interior, and let AXD* be nonempty. Assumption (2.4) holds. Then either Algorithm 3.1 stops with a solution to AVIP(F,X) after a finite number of iterations, or there exists a subsequence of the infinite sequence {xk} that converges to a point in AXP*.

Proof.

Assume that xkAXP* for every iteration k, and let y*AXD*. From Proposition 2.10, we know that y*Xk never lies on Hk for any k. Let {y-i}iN be an arbitrary sequence of point in the interior of X converging to y* and δi a sequence of positive numbers such that limi+δi=0 and that the sequence of closed balls {B(y-i,δi)}iN lies in the interior of X. Note that limi+B(y-i,δi)={y*}. From finite cut property, we know that there exists the smallest index k(i) and a point y~iB(y-i,δi) such that y~i satisfies (4.2)G-xk(i),xk(i)-y~i<0. As G-xk(i),xk(i)-y*>0, there exists a point y^i on the segment [y~i,y*] such that G-xk(i),xk(i)-y^i=0. Since X is compact, we can extract from {xk(i)}iN a convergent subsequence {xk(i)}iS. Denote by x^ its limit point, we have (4.3)G-xk(i),y^i-xk(i)=0,iS. From Proposition 2.9, we know that lk(i) is bounded. Consequently, we can extract form {lk(i)}iS a constant subsequence lk*(i)=k*. Now from the continuity of w(x) for fixed k and the relations (2.15) and (4.3), it follows by taking the limit in (4.3) that (4.4)G-x^,y*-x^=0. By Proposition 2.10, we conclude that x^AXP*.

Theorem 4.2.

Under the conditions of Theorem 4.1, either Algorithm 3.1 stops with a solution to AVIP(F,X) after a finite number of iterations, or there exists a subsequence of the infinite sequence {xk} that converges to a point in XP*.

Proof.

Since ε(0,1), εk(0,1). At the end of Step 4 in Algorithm 3.1 we increase k by one, so we have εk+1<εk, εk0 as k. Moreover, ε-xk0- as k in Algorithm 3.1, where 0- denotes the zero vector. This means F-x^=F(x^) as k. Therefore, from the second result of Theorem 4.1, we know x^ is the solution to the problem VIP(F,X).

5. Conclusions

In , the authors proposed a cutting plane method for solving the quasimonotone variational inequalities, but throughout the paper they employed the exact information of the mapping F from X to Rn. Just like the discussion in the first part of our paper, sometimes, it is not so easy or even impossible to compute the exact values of the mapping F. Motivated by this fact, we consider constructing an approximate problem AVIP(F,X) of VIP(F,X), and try out a proximal analytic center cutting plane algorithm for solving AVIP(F,X). In contrast to , our algorithm can be viewed as an approximation algorithm, and it is easier to implement than  since it only requires the inexact information of the corresponding mapping. At the same time, the conditions we impose on the corresponding mappings are more relaxed, for example,  requires the mapping F satisfies the Lipschitz condition, but we only require that the so-called approximate Lipschitz condition (2.4)-(c) holds.

Acknowledgments

This work is partially supported by the National Natural Science Foundation of China (Grant nos. 11171138, 11171049) and Higher School Research Project of Educational Department of Liaoning Province (Grant no. L2010235).

Harker P. T. Pang J.-S. Finite-dimensional variational inequality and nonlinear complementarity problems: a survey of theory, algorithms and applications Mathematical Programming B 1990 48 2 161 220 10.1007/BF01582255 MR1073707 ZBL0734.90098 Goffin J.-L. Vial J.-P. Convex nondifferentiable optimization: a survey focused on the analytic center cutting plane method Optimization Methods & Software 2002 17 5 805 867 10.1080/1055678021000060829 MR1953822 ZBL1065.90060 Luo Z. Q. Sun J. An analytic center based column generation algorithm for convex quadratic feasibility problems SIAM Journal on Optimization 1999 9 1 217 235 10.1137/S1052623495294943 MR1662587 Goffin J. L. Marcotte P. Zhu D. An analytic center cutting plane method for pseudomonotone variational inequalities Operations Research Letters 1997 20 1 1 6 10.1016/S0167-6377(96)00029-6 MR1428382 ZBL0899.90157 Marcotte P. Zhu D. L. A cutting plane method for solving quasimonotone variational inequalities Computational Optimization and Applications 2001 20 3 317 324 10.1023/A:1011219303531 MR1857061 ZBL1054.90073 Hintermüller M. A proximal bundle method based on approximate subgradients Computational Optimization and Applications 2001 20 3 245 266 10.1023/A:1011259017643 MR1857057 ZBL1054.90053 Kiwiel K. C. An algorithm for nonsmooth convex minimization with errors Mathematics of Computation 1985 45 171 173 180 10.2307/2008055 MR790650 ZBL0584.65034 Solodov M. V. On approximations with finite precision in bundle methods for nonsmooth optimization Journal of Optimization Theory and Applications 2003 119 1 151 165 10.1023/B:JOTA.0000005046.70410.02 MR2028443 ZBL1094.90046 Bertsekas D. P. Nonlinear Programming 1995 Belmont, Mass, USA Athena Scientific Bertsekas D. P. Tsitsiklis J. N. Parallel and Distributed Computation 1989 Englewood Cliffs, NJ, USA Prentice-Hall Bonnans J. F. Gilbert J. C. Lemaréchal C. Sagastizábal C. Optimization Numerique: Aspects Theoriques et Pratiques 1997 Berlin, Germany Springer MR1613833 Lemaréchal C. Lagrangian decomposition and nonsmooth optimization: bundle algorithm, prox iteration, augmented Lagrangian Nonsmooth optimization: Methods and Applications 1992 Philadelphia, Pa, USA Gordon and Breach 201 216 MR1263502 ZBL1050.90553 He Y. R. Minimizing and stationary sequences of convex constrained minimization problems Journal of Optimization Theory and Applications 2001 111 1 137 153 10.1023/A:1017575415432 MR1850682 ZBL0987.90067 Pinkus A. Density in approximation theory Surveys in Approximation Theory 2005 1 1 45 MR2221565 ZBL1067.41003 Auslender A. Optimization, Méthodes 1976 Masson, Paris Numériques Hammer B. Universal approximation of mappings on structured objects using the folding architecture Technical Report. Reihd P, Heft 1996 183 Universitat Osnabruck, Fachbereich Informatic Hammer B. Sperschneider V. Wang P. P. Neural networks can approximate mappings on structured objects Proceedings of the International Conference on Computational Inteliigence and Neural Network 1997 211 214 Hornik K. Stinchcombe M. White H. Multilayer feedforward networks are universal approximators Neural Networks 1989 2 5 359 366 2-s2.0-0024880831 Kiwiel K. C. A proximal bundle method with approximate subgradient linearizations SIAM Journal on Optimization 2006 16 4 1007 1023 10.1137/040603929 MR2219129 ZBL1104.65055 Goffin J. L. Luo Z. Q. Ye Y. Complexity analysis of an interior cutting plane method for convex feasibility problems SIAM Journal on Optimization 1996 6 3 638 652 10.1137/S1052623493258635 MR1402198 ZBL0856.90088