A Regularization SAA Scheme for a Stochastic Mathematical Program with Complementarity Constraints

To reflect uncertain data in practical problems, stochastic versions of the mathematical program with complementarity constraints (MPCC) have drawn much attention in the recent literature. Our concern is the detailed analysis of convergence properties of a regularization sample average approximation (SAA) method for solving a stochastic mathematical program with complementarity constraints (SMPCC). The analysis of this regularization method is carried out in three steps: First, the almost sure convergence of optimal solutions of the regularized SAA problem to that of the true problem is established by the notion of epiconvergence in variational analysis. Second, under MPCC-MFCQ, which is weaker than MPCC-LICQ, we show that any accumulation point of Karash-Kuhn-Tucker points of the regularized SAA problem is almost surely a kind of stationary point of SMPCC as the sample size tends to infinity. Finally, some numerical results are reported to show the efficiency of the method proposed.

In this paper, we focus on the detailed analysis of convergence properties of the regularized SAA problem (3) to the true problem (1) as the sample size tends to infinity.The main contributions of this paper can be summarized as follows: by the notion of epiconvergence in [18], we establish the almost sure convergence of optimal solutions of smoothed SAA problem as the sample size tends to infinity.Under MPCC-MFCQ, we show that any accumulation point of Karash-Kuhn-Tucker points of the regularized SAA problem is a kind of stationary point almost surely.The obtained results can be seen an improvement of [17,Theorem 3.1] for solving SMPCC under weaker constraint qualification conditions.Moreover, under the MPCC strong second-order sufficient condition (MPCC-SSOSC) in [16], we investigate sufficient conditions under which the smoothed SAA problem possesses a Karash-Kuhn-Tucker point when the sample size is large enough, and the sequence of those points converges exponentially to a kind of stationary point of SMPCC almost surely as the sample size tends to infinity.This paper is organized as follows: Section 2 gives preliminaries needed throughout the whole paper.In Sections 3 and 4, we establish the almost sure convergence of optimal solutions and stationary points of the regularized SAA problem as the sample size tends to infinity respectively.In Section 5, existence and exponential convergence rate of stationary points of the regularized SAA problem are investigated.We also report some preliminary numerical results in Section 6.

Preliminaries
Throughout this paper we use the following notations.Let ‖⋅‖ denote the Euclidean norm of a vector or the Frobenius norm of a matrix.For a  ×  matrix ,   denotes the element of the th row and th column of .We use   to denote the  ×  identity matrix, B denotes the closed unite ball, and B(, ) denotes the closed ball around  of radius  > 0. For a extended real-valued function  : R  → R ∪ {±∞}, epi , ∇(), and ∇ 2 () denote their epigraph that is, the set {(, ) | () ≤ }, the gradient of  at , and the Hessian matrix of  at , respectively.For a mapping  : R  → R  , J() denotes the Jacobian of  at . R ++ stands for the positive real numbers.
In the following, we introduce some concepts of the convergence of set sequences and mapping sequences in [18] which will be used in the next section.Define where N denotes the set of all positive integer numbers.
Definition 1.For sets  ] and  in R  with  closed, the sequence { ] } ]∈N is said to converge to  (written with lim sup The continuous properties of a set-valued mapping  can be developed by the convergence of sets.
if the sequence of sets epi  ] converges to epi  in R  × R as ] → ∞.
Definition 4. Given a clos set Ξ ⊆ R  and a point  ∈ Ξ.The cone is called the Fréchet normal cone to Ξ at .Then the limiting normal cone (also known as Mordukhovich normal cone or basic normal cone) to Ξ at  is defined by If Ξ ⊆ R  is a closed convex set, the limiting normal cone  Ξ () is the normal cone in the sense of convex analysis.
Next, we recall some basic concepts that are often employed in the literature on optimization problems with complementarity constraints.
Let  be a feasible point of problem (1) and for convenience we define the index sets The constraint qualifications for SMPCC is as follows.
and E[(⋅, )] are continuously differentiable at .We say the MPCC linear independence constraint qualification (MPCC-LICQ) holds at  if the set of vectors are linearly independent.
As in [16], we use the following two stationarity concepts for SMPCC.
and V ∈ R ||+|| such that  satisfies the following conditions: The following upper level strict complementarity condition was used in [16] in the context of sensitivity analysis for MPCC.
Definition 8. We say that the upper level strict complementarity condition (ULSC) holds at  if   and V  , the multipliers correspondence to E[  (, )], and E[  (, )], respectively, satisfy   V  ̸ = 0 for all  ∈ .
It is well known that a point (, ) satisfies the lower level strict complementarity condition (LLSC) if E[  (, )]+ E[  (, )] > 0 hold for all  ∈ {1, . . ., }, we can see from an example from [16] that ULSC condition is considerably weaker than the LLSC condition, and in practice, it may make more sense than the latter one.
We use the following second-order condition based on the MPCC-Lagrangian: of ().
Definition 9 (see [16]).Let  be a -stationary point of ( 1) and (, , , V) is the corresponding multiplier at . Suppose and E[(⋅, )] are twice continuously differentiable at .We say that the MPCC strong second-order sufficient condition (MPCC-SSOSC) holds at  if for every nonvanishing  with Assume  is a -stationary point of ( 1) and (, , , V) is the corresponding multiplier.Then we know from [16, Theorem 7] that if MPCC-SSOSC holds at , it is a strict local minimizer of the SMPCC (1).
Throughout the paper, we assume the sample  1 , . . .,   of the random vector  is iid and give the following assumptions to make (1) more clearly defined and to facilitate the analysis.
(A1) For every  ∈ R  , the moment generating function

Almost Sure Convergence of Optimal Solutions
In this section, by the notion of epiconvergence in [18], we establish the almost convergence of optimal solutions of regularized SAA problem (3) to those of SMPCC (1) as the sample size tends to infinity.
Let us introduce some notions: (25) Proof.We at first show that lim sup  → ∞ Z  ⊆ Z 0 w.p.
Then, by Lemma That is,   ∈ P  0 , w.p.1, where By Lemma 13, we obtain for sufficiently large ,   ∈   due to  ∈  0 , where which means that   ∈   ∩ P  0 ⊆ Z  .As a result,  belongs to lim inf  → ∞ Z  w.p.1 because of the almost sure convergence of   to  as  → ∞.We complete the proof.
By Definition 3, similarly to the proof of [15,Lemma 4.3], we obtain the following lemma.

Lemma 15. Under the conditions of Proposition 14, we have
The following result is directly from [

Almost Sure Convergence of Stationary Points
In practice, finding a global minimizer might be difficult and in some cases we might just find a stationary point.As a result, we want to know whether or not an accumulation point of the sequence of stationary points of regularized SAA problem ( 3) is almost surely a kind of stationary point of SMPCC (1).Notice that (3) is a standard nonlinear programming with smooth constraints.If   is a local optimal solution of the regularized SAA problem (3), then under some constraint qualifications,   is a stationary point of (3); namely, there exists Lagrange multipliers   ∈ R  ,   ∈ R  ,   ∈ R  ,   ∈ R  , and   ∈ R  such that the vector (  ,   ,   ,   ,   ,   ) satisfies the following Karash-Kuhn-Tucker (KKT) condition for problem (3): We now prove the almost sure convergence of the regularization SAA method for SMPCC (1).(ii) If, in addition, the multipliers   = 0 and V  = 0 for all  ∈  ∩  0 , where then  is a -stationary point of SMPCC (1) almost surely.
Proof.Since   is a stationary point of problem (3), there exist multipliers and Then ( 43) can be reformulated as with Next we show that Ψ  is almost surely bounded under the MPCC-MFCQ.We assume by contradiction that Ψ  is unbounded, then there exists a number sequence {  } ↘ 0 such that   Ψ  → Ψ ̸ = 0, where Since and by outer semicontinuousness of normal cone Notice that   = 0,  ∈   0 ∩ , and   = 0,  ∈   0 ∩ ; then by multiplying   to both sides of (45) and taking limit, we have with Ψ ̸ = 0, where Then we know from (46) that for  ∈ , in the case when  ∈   0 , ã b ≥ 0 due to (  )  ≤ 0 and (  )  ≤ 0 for each .In the case when  ∈  0 , since we have ũ Ṽ ≥ 0. As a result, by Definition 7,  is a stationary point.If Ψ = 0 for  ∈  0 ∩ , then we know from Definition 7 that  is a -stationary point.The proof is completed.
Remark 18.For a deterministic MPCC problem, Scholtes [17] studied the properties of the limit point of a sequence of stationary points generated by the same regularization method under MPCC-LICQ.Notice that MPCC-MFCQ in Theorem 17 is weaker than MPCC-LICQ.Thus this theorem can be seen as an improvement of [17, Theorem 3.1] for solving SMPCC under weaker constraint qualification conditions.

Existence and Exponential Convergence Rate
In this section, we discuss the conditions ensuring existence and exponential convergence of stationary points of regularized SAA problem satisfying (40) when the sample size is sufficiently large.We need the following lemma.
We now state our existence and exponential convergence results.The proof relies on an application of Robinson's standard NLP stability theory in [20].for  sufficiently large.
and compare the test results.
In our experiments, we employed the random number generator unifrnd, exprnd, and normrnd in MATLAB 7.1 to generate independently and identically distributed random samples { 1 ,  2 , . . .,   }.We solved problem (3) with  and  by the solver fmincon in MATLAB 7.1 to obtain the approximated optimal solution   .Throughout the tests, we recorded number of iterations of fmincon (Iter) and the values of the objective function at   (Obj) and these quantities are displayed in the tables of test results.
In the tables below, "REG" and "CHKS" denote regularization SAA method and the smoothing SAA method based on the CHKS smoothing function, respectively.
The examples below varied from examples in Shapiro and Xu [6].(91) Therefore, substituting the above ( 4 ,  5 ,  6 ) into the objective function, we obtain that (1.75, 0, 0.5, 0.25, 1, 0) is the exact optimal solution and 0.75 is the optimal value.The test results are displayed in Table 2.
Our preliminary numerical results shown in Tables 1 and 2 reveal that our proposed method yields a reasonable solution of the problems considered.To compare with the smoothing SAA method, the regularization SAA method may need fewer iteration numbers.
Definition 2. A set-valued mapping  : R   R  is continuous at , symbolized by lim  →  () = (), if Consider now a family of functions  ] : R  → R, where R = R ∪ {±∞}.One says that  ] epiconverges to a function  : R  → R as ] → ∞ and is written as of random variable [∇  (, )]  − [E(∇  (, ))]  is finite valued for all  in a neighborhood of zero.
18, Theorem 7.31].Suppose   solves (3) for each  and  is almost surely an accumulate point of the sequence {  }.If the conditions in Proposition 14 hold and  0 is finite, then  is almost surely an optimal solution of the true problem (1).

Table 1 :
The computational results for Example 1.