A Differential Evolution with Two Mutation Strategies and a Selection Based on an Improved Constraint-Handling Technique for Bilevel Programming Problems

Two mutation operators are used in the differential evolution algorithm to improve the diversity of population. An improved constraint-handling technique based on a comparisonmechanism is presented, and then it is combined with the selection operator in the differential evolution algorithm to fulfill constraint handling and selection simultaneously. A differential evolution with two mutation strategies and a selection based on this improved constraint-handling technique is developed to solve bilevel programming problems.The simulation results on some linear and nonlinear bilevel programming problems show the effectiveness and efficiency of the proposed algorithm.


Introduction
Bilevel programming problem (BLPP) deals with mathematical programming whose feasible set is implicitly determined by another embedded optimization problem [1].As a hierarchical system, BLPP is widely used to lots of fields such as economy, control, engineering, and management (see [2][3][4]).In such a problem, there is a leader that makes the decision first and a follower that makes its decision based on the leader's.More and more practical applications can be formulated as bilevel programming models; so it is important to design some effective algorithms to solve different types of BLPPs in order to satisfy the needs of real-world problems.
Various approaches have been developed in different branches of numerical optimization to solve BLPPs under various assumptions, including extreme point methods, branch and bound algorithms, cutting plane techniques, mixed integer programming, parametric complementary pivoting methods, penalty function methods, trust region methods, and heuristic algorithms.A detailed list of references can be found in [2][3][4].
As it is known, BLPPs being intrinsically difficult, it is not surprising that most algorithmic research to date has focused on the simplest cases of BLPPs, that is, problems having nice properties such as linear, quadratic or convex objective, and/or constraint functions.In particular, the most studied instance of BLPPs has been for a long time the linear version [5].Linear BLPP was shown to be a strongly NP-hard problem [6].Furthermore, it is a NP-hard problem to find the local optimal solution of BLPP [7].Linear BLPP has the important property that at least one optimal solution is attained at an extreme point of the constraint region.Based on this property, the th-best approach was used successfully to compute global solutions of linear BLPPs by enumerating the extreme points of the constraint region [1].A heuristic algorithm was proposed, in which classical extreme point enumeration techniques were combined with genetic algorithms by associating chromosomes with extreme points of the polyhedron [8].Another customary way to solve linear BLPPs is to use the Karush-Kuhn-Tucker conditions to reduce the problem to a one-level complementary slackness problem.To solve this nonlinear slackness problem, many distinct approaches were
The follower's rational reaction set for  ∈  is defined as The main feature of BLPP is that it involves two mathematical programming, the solution of one being part of the constraints of the other.That is to say, the follower's solution depends implicitly on the leader's variable; in return, the leader's objective optimal value is restricted by the follower's solution.So it is quite evident that problem (1) can be converted into the following equivalent single-level implicit programming problem: It is also rewritten as follows: min ∈  (,  ()) We suppose that (, ) and (, ) may be continuous, yet nonconvex, even nondifferentiable, and (, ) and (, ) are differentiable and convex in  for  fixed.Under these assumptions, the constraint region Ω of problem ( 1) is bounded and closed set.
If the lower-level problem is linear programming, we can use available linear programming solver to obtain () for the given ; if the lower-level problem is quadratic programming, we can use available quadratic programming solver to obtain () for the given ; for the generalized lower-level problem, we must turn to some available effective algorithms for solving this constrained optimization problem, such as trust region algorithm, interior algorithm, and active-set algorithm.In this paper, we adopt the optimization toolbox of MATLAB to solve lower-level problems.Certainly, the lowerlevel problem can be solved by evolutionary algorithms, but it is over time-consuming.

The Proposed Differential Evolution Algorithm
Differential evolution (DE), introduced by [21], is one of the most prominent evolutionary algorithms (EAs) for solving real-valued optimization problems.DE has exclusive advantages, such as a simple and easy-to-understand concept, ease of implementation, powerful search capability, and robustness.Using only a few control parameters, DE has gradually become more popular and has been used in many practical cases [33].

A Constraint-Handling Technique Based on a Comparison
Mechanism.A constraint-handling technique is developed by improving Deb's feasibility-based comparison method [34], which is based on the following criteria.
(i) Between two feasible solutions, the one with the lowest objective function value wins for minimization problem.(ii) If one solution is feasible and the other one is infeasible, the feasible solution wins.(iii) If both solutions are infeasible, the one with the lowest sum of constraint violations is preferred.
(3) If one individual is feasible and another one is infeasible, the following cases are considered.
The second subpopulation with NP/2 individuals is generated by the following mutation operator: where where   ∈ (0, 1) is a uniform random number, () ∈ {1, 2, . . ., } is the randomly selected index chosen once for each , and CR is a real-valued crossover rate constant in the range (0, 1).

Selection Operation Based on the Constraint-Handling
Technique.The above-mentioned constraint-handling technique is introduced to the selection operator to decide the population for the next generation.
For  = 1, 2, . . ., NP, the trial vector   () and the parent vector   () are compared.The individual of the next generation   ( + 1) is decided according to Definition 3: 3.7.The Proposed Algorithm.Now we present the differential evolution algorithm with two mutation strategies and a selection based on an improved constraint-handling technique as follows.
Step 1 (parameter setting).Set population size NP, scaling factor SF, crossover rate CR, and maximum number of generations Max .
For  = NP/2 + 1 to NP, select randomly The second subpopulation is generated according to (12).
Step 4 (crossover operation).Crossover individuals are produced by (13).For each crossover individual   (), obtain lower-level optimal solution by using available optimization solver.Evaluate the constraint violations and fitness values of all crossover individuals and determine the best individual according to Definition 4.
Step 5 (selection operation).According to selection rule, form the next population.
Step 6 (stopping criterion).If the stopping criterion is met, then stop and output the result.Otherwise, set  =  + 1 and go to Step 3.

Computational Results and Comparison
To evaluate the performance of the proposed algorithm, we test two types of benchmark problems: one type is linear BLPP and another type is nonlinear BLPP, which includes linear-quadratic programming, quadratic-quadratic programming, linear-linear fractional programming, general nonlinear BLPP, and complex nonlinear BLPP with nondifferentiable leader's objective function.The results of the proposed algorithm are compared with other algorithms in the literature.
All experiments are performed on a notebook computer with 2.53 GHz Dual-core Processor and 1.94 GB of RAM in MATLAB software.Each test problem implemented 50 independent runs.For solving the lower-level optimization problems, we use corresponding function calls in the optimization toolbox of MATLAB to solve linear programming, quadratic programming, or other nonlinear programming.
Performance of the proposed algorithm is evaluated by the following criteria: (i) success rate (SR), that is, the percentage of convergence to the optimal solution in 50 independent runs, which can be used to evaluate the reliability of the proposed algorithm; (ii) mean number of fitness evaluations (MNFE) when the optimal solution is reached first in 50 independent runs, which is assessed the computational cost; In order to demonstrate the effectiveness of the improved constraint-handling technique, we compare it with Deb's feasibility-based method [34] in the same framework of algorithm and same parameters.Also, we compare the obtained results with those presented in the corresponding references.

Computational Results for Linear BLPPs.
To examine the effectiveness of the proposed algorithm for linear BLPPs, we first test 9 linear bilevel programming problems and compare the results of the proposed algorithm with other algorithms in the literature.The linear test problems are provided in Appendix A.
The comparative results obtained by two constrainthandling methods are shown in Table 1.As shown in Table 1, the improved constraint-handling technique is slightly better than Deb's method in terms of SR, MNFE, and SD.Furthermore, the proposed algorithm has a good performance according to stability, efficiency, and quality of solution obtained.Therefore it can be concluded that the improved constraint-handling technique is effective.
Table 2 presents the best results obtained by the proposed algorithm for linear BLPPs, including best solution, upperlevel and lower-level objective function values.
In order to demonstrate the improvement in algorithm performance by the use of hybrid mutation operator, we compare hybrid mutation operator with single mutation operator in the same framework of algorithm and same parameters.The comparative results are shown in Table 3. From this table, it can be seen that the reliability of algorithm is enhanced by using hybrid mutation operator in terms of SR.Although hybrid mutation operator requires a larger computational cost than single mutation operator (11) except for Problems A.4 and A.7, the stability of single mutation operator (11) is worse than that of hybrid mutation operator according to SR and SD.The computational cost of single mutation operator (12) is larger than that of hybrid mutation operator, but the quality of solution obtained is better than hybrid mutation operator for Problem A.7 in terms of SR and SD.Therefore the algorithmic performance is improved by the use of those two mutation operators.
The proposed algorithm is compared with the algorithms in the literature according to the best values of upperlevel and lower-level objective functions.The comparative results are shown in Table 4. From this table, the proposed algorithm exhibits the better results than other algorithms.For Problems A.1, A.8, and A.9, the best results obtained by the proposed algorithm are better than those of other algorithms in the literature.For remaining problems, they have same optimization results.

Computational Results for Nonlinear BLPPs.
To validate the effectiveness of the proposed algorithm for nonlinear BLPPs, we test two sets of nonlinear bilevel programming problems.The first set has 10 examples selected from different sources of the literature, and the second set consists in solving the test collection (SMD1 to SMD12) proposed in [28].Finally, we compare the results of the proposed algorithm with the other algorithms in the literature.The first set of nonlinear BLPPs is provided in Appendix B, and the second set of nonlinear BLPPs can be obtained in [28].
With same algorithm's framework and parameter setting, the improved constraint-handling method is compared with Deb's feasibility-based method [34] for first set of nonlinear BLPPs to illuminate the effectiveness of the improved constraint-handling method again.Comparative results of two constraint-handling methods are shown in Table 5.From this table, it can be seen that the improved constraint-handling method is better than the Deb's method in terms of success rate, mean number of fitness evaluations, and standard deviation of fitness values.Moreover, this table shows that the proposed algorithm with the improved constraint-handling method is effective and can find the global optimal solutions with a 100% success rate, except for Problem B.4 with a 76% success rate.
In order to display the computational results obtained by the proposed algorithm for first set of nonlinear BLPPs, the best solution ( * ,  * ), the upper-level objective value ( * ,  * ), and the lower-level objective value ( * ,  * ) for all problems are provided in Table 6.This table shows that the best solutions obtained by the proposed algorithm are same as the global optimal solutions of all problems.
Table 7 illustrates the comparative results of the proposed algorithm with available algorithms in the literature for the first set of nonlinear BLPPs.As shown in Table 7, the proposed algorithm is able to find the global optimal solutions for the first set of nonlinear BLPPs.Compared against the other algorithms in the literature, the proposed algorithm can obtain equal optimal results for all problems.
In order to investigate further the performance of the proposed algorithm, the second set of nonlinear BLPPs is tested, which is a complex test collection with different dimensions in [28].We performed 50 runs for each of the test problems with 5 and 10 dimensions and 11 runs for each of the test problems with 20 dimensions.In the case with five dimensions,  = 1,  = 2, and  = 1 are used in SMD1 to  , where , , , and  are parameters of test problems in [28].Abovementioned parameters of the proposed algorithm continue to be used here.When the accuracy of best upper-level function value becomes less than 10 −4 , the algorithm terminates.The statistical results for 5-, 10-, and 20-dimensional test problems of second set are provided in Table 8, including success rate (SR), mean number of fitness evaluations (MNFE), and standard deviation of fitness values (SD).This table shows that the proposed algorithm can locate the optimal solution for 5-and 10-dimensional test problems.Except for SMD10-SMD12 with 20 dimensions, it can also obtain the optimal solution for other 20-dimensional test problems.The proposed algorithm fails to find the optimal solution within a given accuracy (less than 10 −4 ) for SMD10-SMD12 with 20 dimensions.
Table 9 presents the best results obtained by the proposed algorithm for 5-and 10-dimensional test problems of second set, including the best solution ( * ,  * ), the best upper-level objective value ( * ,  * ), and the best lower-level objective value ( * ,  * ).
The comparison of median accuracy achieved by the proposed algorithm and those reported in [28] is provided in Table 10.For 20-dimensional test problems, the accuracy for the upper level (UL) and lower level (LL) is not given in [28].From this table, it is obvious that the proposed algorithm is better than the nested bilevel evolutionary algorithm in [28] according to median value of the accuracy at the upper level and lower level for 5-and 10-dimensional test problems, except for SMD5.Furthermore, the proposed algorithm can obtain the good accuracy for 20-dimensional test problems except for SMD10-SMD12.The nested bilevel evolutionary algorithm in [28] cannot find feasible solutions for 10dimensional SMD9-SMD12, which are constrained problems, and 20-dimensional SMD7-SMD12.This also shows that the improved constraint-handling method in this paper performs better than the constraint handling in [28], which belongs to Deb's method [34].
Tables 11, 12, and 13 provide the best, median, and worst numbers of function evaluations at upper and lower levels for SMD1-SMD12 with 5, 10, and 20 dimensions, respectively.In these tables, we also compare the proposed differential evolution algorithm (DE) with the nested bilevel evolutionary algorithm (NBEA) in [28] for SMD1-SMD12 with 5, 10, and 20 dimensions, respectively.
From Table 11, it can be seen that for 5-dimensional test problems, DE has smaller "best FE" for the upper level than NBEA for 8 out of 12 test problems (SMD2-SMD4, SMD6-SMD9, and SMD11) and smaller "worst FE" for the upper level than NBEA for 6 out of 12 test problems (SMD1, SMD2, SMD5, SMD7, SMD8, and SMD11).But DE has larger "median FE" for the upper level than NBEA except for SMD4,  12, it is obvious that for 10-dimensional test problems, DE requires smaller "best FE" for the upper level than NBEA for 4 out of 8 test problems (SMD2, SMD4, SMD5, and SMD7), smaller "median FE" for the upper level than NBEA for 7 out of 8 test problems (SMD1-SMD3 and SMD5-SMD8), and smaller "worst FE" for the upper level than NBEA for 6 out of 8 test problems (SMD1-SMD3, SMD5, SMD6, and SMD8).DE has much smaller FE for lower level than NBEA in terms of best FE, median FE, and worst FE for SMD1-SMD8.In addition, NBEA cannot obtain a feasible solution for each of 10-dimensional SMD9-SMD12.
From Table 13, it can be seen that for 20-dimensional test problems, DE has smaller "median FE" for the upper level than NBEA for 5 out of 6 test problems (SMD1-SMD3, SMD5, and SMD6), and smaller "worst FE" for the upper level than NBEA for 6 test problems (SMD1-SMD6).But DE has larger "best FE" for upper level than NBEA except for SMD2 and SMD5.DE has much smaller FE for lower level than NBEA in terms of best FE, median FE, and worst FE for SMD1-SMD6.In addition, NBEA cannot obtain a feasible solution for each of 20-dimensional SMD7-SMD12.
From Tables 11,12, and 13, we can conclude that it is expensive that the lower-level programming is solved by evolutionary algorithms.

Conclusion
This paper presents a differential evolution with two mutation strategies and a selection operator based on an improved constraint-handling technique for BLPPs.This algorithm avoids the use of penalty function to deal with the constraints, because the right penalty parameter is hard to select.Especially, this algorithm can solve general nonlinear BLPP which may have a nondifferentiable upper-level objective function.The BLPP's natural structure is considered in this paper to develop algorithm in order to solve a wide class of BLPPs with weak features.The numerical results on linear and nonlinear BLPPs show the proposed algorithm is effective and robust.

( 2 )Definition 1 .Definition 3 .
The infeasible individuals in certain range (close to the boundary of the feasible region) with the good values of the objective function are selected for the next generation.This improved constraint-handling technique is presented detailedly as follows.An individual  is called a feasible individual, if and only if (, ) ∈ .Definition 2. For a given , obtain  ∈ ().Let   () = max{  (, ), 0} for  = 1, . . ., ,   () = max{  (, ), 0} for  = 1, . . ., . () = max{  (),   () for all , } is called constraint violation of an individual .An individual  1 is better than another individual  2 , if one of the following criteria is satisfied.

Table 1 :
Comparative results of two constraint-handling methods for linear BLPPs over 50 independent runs.
(iii) standard deviation of fitness values (SD), which is used to measure the quality of the solution obtained;(iv) best values of upper-level objective functions ( * ,  * ) and lower-level objective functions ( * ,  * ), as well as best solution obtained ( * ,  * ).

Table 2 :
Best results obtained by the proposed algorithm for linear BLPPs.

Table 3 :
Comparative results of hybrid mutation operator and single mutation operator for linear BLPPs over 50 independent runs.

Table 4 :
Comparison of the proposed algorithm with the algorithms in the literature for linear BLPPs.

Table 5 :
Comparative results of two constraint-handling methods for first set of nonlinear BLPPs over 50 independent runs.

Table 6 :
Best results obtained by the proposed algorithm for first set of nonlinear BLPPs.

Table 7 :
Comparison of the proposed algorithm with the algorithms in the literature for the first set of nonlinear BLPPs.

Table 8 :
Statistical results obtained by the proposed algorithm for second set of nonlinear BLPPs.

Table 9 :
Best results obtained by the proposed algorithm for 5 and 10 dimensions in second set of nonlinear BLPPs.

Table 10 :
[28]arison of accuracy for the upper level (UL) and lower level (LL) by the proposed algorithm and the algorithm in[28]for the test problems of the second set.A dash denotes that a feasible solution could not be obtained for the test problem.NA means the result is not available.

Table 11 :
[28]arison of function evaluations (FE) for the upper level (UL) and lower level (LL) by the proposed differential evolution algorithm (DE) and the nested bilevel evolutionary algorithm (NBEA) in[28]for 5-dimensional test problems of the second set.