^{1}

Evolution strategies are successful global optimization methods. In many practical numerical problems constraints are not explicitly given. Evolution strategies have to incorporate techniques to optimize in restricted solution spaces. Famous constraint-handling techniques are penalty and multiobjective approaches. Past work has shown that in particular an ill-conditioned alignment between the coordinate system of Gaussian mutation and the constraint boundaries leads to premature convergence. Covariance matrix adaptation evolution strategies offer a solution to this alignment problem. Last, metamodeling of the constraint boundary leads to significant savings of constraint function calls and to a speedup by repairing infeasible solutions. This work gives a brief overview over constraint-handling methods for evolution strategies by demonstrating the approaches experimentally on two exemplary constrained problems.

Many continuous optimization problems in practical applications are subject to constraints [

The remainder of this section gives a brief introduction to evolution strategies, constrained problems, and a taxonomy of constraint-handling techniques. Section

Evolution strategies (ES) are a family of strong stochastic methods for global optimization. Developed by Rechenberg [

In the field of evolutionary computation the constraints typically are not considered available in their explicit formal form. Rather, the constraints are assumed to be black boxes: a vector

The two following test functions excellently demonstrate the phenomenon of premature fitness stagnation that will be discussed in the following sections and that is a challenge for most constraint-handling techniques. The two functions will be used for the discussion of the methods reviewed in the current paper. Problem 2.40—taken from Schwefel's artificial test problems [

The second problem is called tangent problem (TR). It is based on the sphere model subject to one linear constraint:

A variety of constraint-handling methods for evolutionary algorithms have been developed in the last decades. Most of them can be classified into five main types of concepts.

(

(

(

(

(

Of course, constraint-handling methods exist that do not fit into the taxonomy. Montes and Coello Coello [

In the following section we will compare various approaches from different fields and compare them, in particular with regard to the mentioned premature step-size problem. The next section shows this problem experimentally.

Evolutionary search is guided by the quality of its candidate solutions. Consequently, an obvious solution to constraint-handling is to deteriorate the fitness of infeasible methods [

First of all, we will analyze the behavior of death penalty, that is, simply discarding infeasible offspring solutions [

Experimental results of the death penalty method.

Death penalty | best | mean | dev | ffe | cfe |
---|---|---|---|---|---|

TR2 | 11,720 | 20,447 | |||

2.40 | 51.9 | 227.6 | 65.2 | 50,624 | 96,817 |

We can summarize the behavior of death penalty mentioning the advantage that

The question arises whether dynamic penalty functions also suffer from premature convergence. To answer this question we tested the penalty function by Joines and Houck [

Experimental results of the dynamic penalty function by Joines and Houck [

best | mean | dev | ffe | cfe | |
---|---|---|---|---|---|

TR2 | 13,100 | 13,100 | |||

2.40 | 219.4 | 440.8 | 85.0 | 31,878 | 31,878 |

The most obvious modification to prevent premature step-size reduction is the introduction of a minimum step-size

Experimental results of the death penalty step control evolution strategy.

DSES | Type | best | Mean | dev | ffe | cfe |
---|---|---|---|---|---|---|

TR2 | 1,253,394 | 2,315,574 | ||||

2.40 | 89,832 | 1,118,490 |

Again, we summarize the following results:

A familiar variant to handle constraints is to treat each constraint—or an aggregated sum of all constraints—and the objective function as separate objectives in a multiobjective formulation. Similar approaches have been introduced in the past [

Table

Experimental results of the two-sex evolution strategy on TR2 and 2.40.

TSES | Type | best | mean | dev | ffe/cfe | |
---|---|---|---|---|---|---|

TR2 | (8+8,10+90) | 200 | 521,523 | |||

2.40 | (8+8,13+87) | 498,594 |

We can summarize that the

In real-valued optimization the coordinate system plays an important role. If the coordinate system of the mutation operators, for example, of Gaussian mutation, is not aligned to the coordinate system of the objective function—and this is frequently the case in black-box optimization—undesirable effects may occur like premature step-size reduction.

The phenomenon of premature step-size reduction at the constraint boundary has been analyzed in [

Illustration of the success probabilities at the constraint boundary. In this simplified model we assume that mutations are produced uniformly on the boundary of the circles. Both solutions

Arnold and Brauer [

The shape of the standard mutation ellipsoid is Gaussian. The best modification to improve the success rate situation would be a more flexible mutation distribution function. Later, we will see that a rotation of the mutation ellipsoid is a reasonable undertaking. But is a deformation also an adequate solution to low success rates? Biased mutation aims at biasing the mean of the Gaussian distribution into beneficial directions self-adaptively [

Experimental results of the biased mutation variants BMO and DMO.

BMO | best | mean | dev | ffe | cfe |
---|---|---|---|---|---|

TR2 | 26,832 | 25,479 | |||

2.40 | 459,774 | 508,387 | |||

DMO | best | mean | dev | ffe | cfe |

TR2 | 31,506 | 29,196 | |||

2.40 | 358,954 | 359,545 |

We can conclude that

Correlated mutation by Schwefel [

a self-adaptive rotation—in this case the

a rotation with the help of a coevolutionary approach,

with a metamodel of the constraint boundary that delivers the orientation of the constraint boundary.

Table

A comparison of correlated mutation, metaevolution, and the metamodel-based ellipsoid rotation on TR2.

SA-ES | MA-ES | MM-ES ( | MM-ES (30) | |
---|---|---|---|---|

Best | 0 | 0.0 | ||

Mean | 0 | 0.0 | ||

Dev | 0.0 | |||

Ffe | 22,445 | 927,372 | 18,736 | 11,998 |

Cfe | 39,921 | 1,394,023 | 32,960 | 20,183 |

Obviously, the coordinate system alignment problem is solved with the mutation ellipsoid rotation. But the self-adaptive rotation does not lead to satisfying results, while the metaevolutionary approach is inefficient. In the following paragraph we will investigate whether the covariance matrix adaptation techniques, which are designed to align coordinate systems, are able to adapt their covariance matrix to constrained problems automatically without a metamodel.

Past research on constraint-handling missed to concentrate on covariance matrix adaptation techniques. It is an astonishing fact that no sophisticated constraint-handling techniques for these algorithms have been introduced so far. Nevertheless, we will now analyze whether the coordinate system alignment problem can be solved with covariance matrix adaptation using death penalty. The idea of covariance matrix adaptation techniques is to adapt the distribution of the mutation operator such that the probability to reproduce steps that led to the actual population increases. This idea is similar to the estimation of distributions approaches. The covariance matrix adaptation evolution strategy (CMA-ES) was introduced by Hansen [

Experimental analysis of the CMA-ES with death penalty.

CMA-ES (DP) | best | mean | dev | ffe | cfe |
---|---|---|---|---|---|

TR2 | 0.0 | 0.0 | 6,754 | 12,019 | |

2.40 | 0.0 | 0.0 | 19,019 | 71,241 |

We can conclude that the CMA-ES is able to align the coordinate system automatically without a metamodel. Recent results have shown that an acceleration can be achieved if the covariance matrix is rotated with the help of a metamodel exactly at the time when the constraint boundary is reached [

In black-box scenarios the constraint boundaries are not explicitly given. Metamodeling of constraints allows advanced constraint-handling methods. Metamodels can be used for various purposes, for example, for checking the feasibility and for repair of infeasible mutations, and—like we have seen in the previous section—for control of mutation ellipsoids and covariance matrices. Metamodeling of objective functions has developed to a successful standard in evolutionary optimization [

For constraint metamodeling various classification and regression methods can be applied. For the case of linear constraints a metamodel that is based on sampling

In a next step the binary search procedure is applied to identify

Procedure to estimate the constraint boundary

In the last step we calculate the normal vector

The search (i.e., the center of the mutation ellipsoid)

The search

We use

For nonlinear constraints other regression or classification techniques may be taken into account like support vector regression or support vector machines [

A metamodel can be used to check the feasibility of new solutions in order to reduce constraint function evaluations [

The model predicts that

The model predicts that

Exemplarily, we take the linear constraint metamodel of the previous paragraph into account and test the feasibility check approach. A

Results of the CMA-ES with feasibility check based on the linear metamodel.

CMA-ES (check) | best | mean | dev | ffe | cfe |
---|---|---|---|---|---|

TR2 | 0.0 | 0.0 | 6,780 | 7,781 | |

2.40 | 0.0 | 0.0 | 19,386 | 34,254 |

The repair approach projects infeasible mutations onto the constraint boundary

The elongation of the projection of infeasible solution

The elongation of the projection into the potentially feasible region guarantees feasibility of the repaired individuals. Nevertheless, it might prevent fast convergence, in particular in regions far away from the hyperplane support point

Results of the CMA-ES with repair mechanism based on the linear metamodel.

CMA-ES (repair) | best | mean | dev | ffe | cfe |
---|---|---|---|---|---|

TR2 | 0.0 | 0.0 | 3,432 | 5,326 | |

2.40 | 0.0 | 0.0 | 16,067 | 75,705 |

Many constraint-handling methods exist for evolution strategies, at the head penalty functions. Due to low success rates at the constraint boundary, ES without coordinate alignment techniques often fail to find the optima in the vertex of the feasible solution space. The death penalty step control approach and the multiobjective biologically inspired two-sex ES prevent a premature step-size reduction on some problems, but its success depends on proper parameter settings. Low success rates at the constraint boundary can be increased with coordinate system alignment techniques. A first step into this direction is biased mutation techniques, that is, biased mutation and descent biased mutation. Much better results can be achieved with metamodel-based mutation ellipsoid rotation. This rotation cannot be achieved self-adaptively, but automatically with covariance matrix adaptation mechanisms. The latter shows excellent results, even on hard problems like TR2. Further improvements of the CMA-ES can be achieved with metamodeling: constraint boundary surrogates can be used for prediction of feasibility of mutations and for repair of infeasible solutions. At last, Table

Results of the CMA-ES with covariance matrix rotation, feasibility check, and repair mechanism [

CMA-ES (all) | best | mean | dev | ffe | cfe |
---|---|---|---|---|---|

TR2 | 0.0 | 0.0 | 3,249 | 3,650 | |

2.40 | 0.0 | 0.0 | 11,216 | 30,069 |

Metamodeling of constraints will probably become more and more important for future research. Nonlinear models will increase the accuracy of feasibility prediction. Advanced regression methods will improve the accuracy of repaired infeasible solutions. Further constraint-handling methods are imaginable like adaptation of mutation probability distributions and covariance matrices—also with non-linear metamodels.