Many-Objective Optimization Using Adaptive Differential Evolution with a New Ranking Method

Pareto dominance is an important concept and is usually used inmultiobjective evolutionary algorithms (MOEAs) to determine the nondominated solutions. However, for many-objective problems, using Pareto dominance to rank the solutions even in the early generation, most obtained solutions are often the nondominated solutions, which results in a little selection pressure of MOEAs toward the optimal solutions. In this paper, a new rankingmethod is proposed formany-objective optimization problems to verify a relatively smaller number of representative nondominated solutionswith a uniform andwide distribution and improve the selection pressure ofMOEAs. After that, amany-objective differential evolutionwith the new rankingmethod (MODER) for handlingmanyobjective optimization problems is designed. At last, the experiments are conducted and the proposed algorithm is compared with several well-known algorithms.The experimental results show that the proposed algorithm can guide the search to converge to the true PF and maintain the diversity of solutions for many-objective problems.


Introduction
Multiobjective evolutionary algorithms (MOEAs) are a kind of effective methods for solving multiobjective problems [1,2].Almost all well-known and frequently used MOEAs, which have been proposed in the last twenty years [3][4][5], are based on Pareto dominance.Such Pareto dominancebased algorithms usually work well on problems with two or three objectives but its searching ability is often severely degraded by the increased number of objectives [6].This is due to the fact that most solutions in a population could be nondominated solutions under many objectives even in the early stages of evolution for many-objective optimization problems.When almost all solutions in a population are nondominated solutions, Pareto dominance-based fitness evaluation almost cannot generate any selection pressure toward the Pareto front (PF).Therefore, how to increase the selection pressure toward the PF is critical for the manyobjective optimization algorithm.
In the literature, there are mainly three categories which are used to MOEAs to deal with many-objective optimization problems.The first category uses an indicator function, such as the hypervolume [7][8][9], as the fitness function.This kind of algorithms is also referred to as IBEAs (indicator-based evolutionary algorithms), and their high search ability has been demonstrated in the literature [10].Bader and Zitzler [11] propose a fast hypervolume-based many-objective optimization algorithm which uses Monte Carlo simulation to quickly approximate the exact hypervolume values.However, one of their main drawbacks is the computation time for the hypervolume calculation which exponentially increases with the number of objectives.
The second category utilizes the scalarizing functions to deal with the many-objective problem.According to the literatures [12][13][14], scalarizing function-based algorithms could better deal with many-objective problems than the Pareto dominance-based algorithms.The main advantage of scalarizing function-based algorithms is the simplicity of their fitness evaluation which can be easily calculated even if the number of objectives is large.The representative MOEA in this category is MOEA/D [15] (multiobjective evolutionary

Objective function Domain
DTLZ1 algorithm based on decomposition), which works well on a wide range of multiobjective problems with many objectives, discrete decision variables, and complicated Pareto sets [16][17][18].In MOEA/D [15], the uniformity of the used weighted vectors determines the uniformity of the obtained nondominated optimal solutions; however, the used weighted vectors in MOEA/D are not very uniform and the size  of these weighted vectors should satisfy the restriction  =   +−1 .Thus  cannot be freely assigned and it will increase nonlinearly with the increase of , where  is the number of objectives and  is an integer.This restricts the application of MOEA/D to many-objective optimization problems (i.e.,  is large).Therefore, for many-objective problems, how to set weight vectors is a very difficult but critical task.
The third category makes use of solution ranking methods.Specifically, solution ranking methods are used to discriminate among solutions in order to enhance the selection pressure toward the PF, which make sure the solutions are able to converge to the PF.Bentley and Wakefield [19] proposed ranking composition methods which extract the separated fitness of every solution into a list of fitness values for each objective.Ikeda et al. [20] proposed a relaxed form of dominance (RFD) to deal with what they called dominance resistant solutions, that is, solutions that are extremely inferior to others in at least one objective but are hardly dominated.Farina and Amato [21] proposed a dominance relation which takes into account the number of objectives where a solution is better, equal, and worse than another solution.Sato et al. [22] proposed a method to strengthen or weaken the selection process by expanding or contracting the solutions' dominance area.
In this paper, a new ranking approach for many-objective problems is proposed to rank the obtained solutions.In this approach, for each solution, a small number of virtual objective vectors which are evenly distributed around the objective vector of the solution are generated to determine whether the solution is a nondominance solution.The selection pressure, the diversity of solutions, and time consumption are taken into consideration by the ranking method.Moreover, a multiobjective DE algorithm based on the new ranking method is designed to solve many-objective optimization problems.
The remainder of this paper is organized as follows.Section 2 introduces the new ranking method.The proposed many-objective algorithm is proposed in Section 3. Section 4 shows the experimental results of the proposed algorithm.Finally, Section 5 gives the conclusions and future works.
For many-objective problems, to determine whether a solution is a nondominated one by using Pareto dominance, all other solutions should be used to make the comparisons of objectives one by one among all objectives, and a lot of comparisons (thus a large amount of computation) will be needed.Also, Pareto dominance will result in a lot of nondominated solutions.In this work, a new ranking method is designed to determine whether a solution is a nondominated solution by only using a small number of virtual objective vectors.For each solution ,  points in objective space are generated on the following surface: where  > 0 and it controls the shape and size of the surface;  = min{‖()‖  |  ∈ POP}, where POP is the set of the current solutions.Before the  points are generated,  vectors (  ,  = 1, . . ., ) are firstly generated.The th vector   = ( 1 , . . .,   ) ( = 1, . . ., ) is generated by optimizing the following formula: where  = ( 1 , . . .,   ) = ()/ ∑  =1   () and  is an integer.After   is obtained, the th virtual vector is generated by the following formula: Note that these  virtual vectors have the same and the smallest norm (-norm) among all obtained solutions and distribute evenly around the objective vector of the solution .Moreover, for any virtual vector, for example,   ( = 1, . . ., ), its th component is not larger than   () and the th component of   is not larger than   , which indicate that the virtual vector   more likely dominates the solution () than other virtual vectors whose corresponding th component of   is larger than   .Thus, if () is not dominated by these  virtual vectors, the solution  can be regarded as a nondominated solution (of course, it may be a dominated one, but it is more possible to be a nondominated one); otherwise, the solution  can be regarded as a dominated solution.
The parameter  plays a very important role in this method which controls the convergence and diversity of solutions.The larger the value of  is, the better the convergence of solutions is.However, if the surface and the true PF are in great difference, a large value of  will lead to the poor diversity.Therefore, the value of  should be traded off between the convergence and diversity of solutions.
The proposed ranking method has the following advantages.
(1) For each solution  of the set POP whose size is , this method only needs to make comparisons among +1 points, instead of  points ( ≪ ), which can reduce much time consumption.(2) The value of th component of   is not more than   (), and the angle of   and () is very small, which make () easily dominated by   .Thus, the number of nondominated solutions generated by the method is not too large, and the selection pressure can be enhanced.
(3) In this method, maximum  points need additional space to store, which only takes a small amount of storage space.(4) This method can balance the convergence and diversity by adjusting the parameter .

Algorithm
In this section, a many-objective optimization evolutionary algorithm is presented.The algorithm uses DE [24,25] operator to generate offspring and the (+) selection scheme to generate the next population.DE is a good optimizer for continuous optimization.The trial vector generation comprises two operators: mutation and crossover operators.
The mutation operator which is used in our algorithm is performed as follows: where  is the generation number;   1 and   2 are two random nondominated solutions among the current population POP();   3 is a random solution among POP() except for   1 and   2 ;    is th mutation offspring;  is the scale factor which is usually in (0, 1].
The crossover operator is described as follows: where CR is the crossover rate which is usually in (0 1);   and   are the lower bound and upper bound of the variable   , respectively; V   = (V  ,1 , . . ., V  , ) is the offspring.In classical DE, the control parameters  and CR highly affect its performance.Different settings of those parameters influence the quality of offspring which are generated by the mutation and crossover.In this paper, a self-adjusted parameter control strategy is presented to balance exploration and exploitation.For each solution   , its two control parameters which are used to generate the offspring are determined as follows: where  max is the maximal generation.Based on the above methods, a new multiobjective differential evolution based on the new ranking method is proposed (MODER) for many-objective optimization problems and the steps of the algorithm MODER are described as follows.
Step 2 (fitness).Solutions of POP() are firstly divided into two sets (the set of nondominated solutions and the set of dominated solutions) by the proposed ranking method (Algorithm 1).For each set, the fitness value of each solution in the set is calculated by the crowding distance.Then  better solutions are selected from the population POP() and put these solutions into the population POP.In this work, binary tournament selection is used.
Step 3 (new solutions).Apply ( 6)- (8) or simulated binary crossover [26] to the parent population POP to generate offspring.The set of all these offspring is denoted by  and its size is .
Step 4 (generate POP( + 1)).Solutions of POP() ∪  are firstly divided into two sets by the proposed ranking method.For each set, the fitness value of each solution in the set is calculated by the crowding distance.Then  better solutions are selected from the population POP() ∪  and put these solutions into the population POP( + 1).Let  =  + 1.

Experimental Studies
In order to validate our algorithm, MODER is fully compared with MOEA/D [17], NSGAII [3], and NSGAII based on contracting or expanding the solutions' dominance area [22] (denoted by NSGAII-CE) on DTLZ1 and DTLZ3 of the DTLZ family [27], each with 5-50 objectives.

Experimental Setting.
The experiments are carried out on a personal computer (Intel Xeon CPU 2.53 GHz, 3.98 G RAM).The solutions are all coded as real vectors.Polynomial mutation [28] operators and differential evolution (DE) [24] are applied directly to real vectors in three algorithms, that is, MOEDR, MOEA/D, and NSGAII-CDAS.The distribution index and crossover probability in polynomial mutation [28] operators are set to 20 and 1/, respectively.The crossover rate and scaling factor in DE operator are set to 1.0 and 0.5, respectively.The aggregate function of MOEA/D is the Tchebycheff approach [15], and weight vectors are generated by using the uniform design method for MOEA/D and UREA/D.The number of the weight vectors in the neighborhood in MOEA/D is set to 20 for all test problems.The parameter of the ranking method  is set to 2, and in each generation, the value of  is self-adaptive determined by the following expression: where () = min{‖()‖  |  ∈ POP} and  is the number of objectives.In this algorithm, we set  = 2.The parameter of CE is set to 0.25 for NSGAII-CE.20 independent runs are performed with population size of 100 for all these instances.The maximal number of function evaluations is set to 100000 for all test problems.The values of default parameters are the same as in the corresponding papers.

Experimental Measures.
In order to compare the performance of the four compared algorithms quantitatively, the following three widely used performance metrics are adopted: generational distance (GD), inverted generational distance (IGD) [29], and Wilcoxon rank-sum test [30], where GD measures how far the obtained Pareto front is away from the true Pareto front, which allows us to observe whether the algorithm can converge to the true PF.If GD is equal to 0, all points of the obtained PF belong to the true PF.IGD measures how far the true PF is away from the obtained PF, which shows whether points of the obtained PF are evenly "+" means that MODER outperforms its competitor algorithm, "−" means that MODER is outperformed by its competitor algorithm, and "=" means that the competitor algorithm has the same performance as MODER.
distributed throughout the true PF.If IGD is equal to 0, the obtained PF contains every point of the true PF.In particular, GD and IGD indicators are used simultaneously to observe whether the solutions are distributed over the entire PF or the solutions concentrate in some regions of the true PF.In our experiments, 100000 points which are generated by using the uniform design [31] for all these test instances which are uniformly sampled on the true PF are used to calculate the metrics GD and IGD of solutions obtained by an algorithm.Wilcoxon rank-sum test [30] is used in the sense of statistics to compare the mean IGD and GD of the compared algorithms.It tests whether the performance of MODER on each test problem is better than ("+"), same as ("="), or worse than ("−") that of the compared algorithms at a significance level of 0.05 by a two-tailed test.These results are given in Tables 2 and 3.

Comparisons of MODER with MOEA/D, NSGAII, and NSGAII-CE.
In this section, some simulation results and comparisons that demonstrate the potential of MODER are presented, and the comparisons mainly focus on two aspects: (1) the ability of the ranking method to improve the selection pressure and (2) the ability of the ranking method to maintain the diversity of solutions.
Tables 2 and 3 show the mean and standard deviation of the GD metric and IGD metric obtained by these four algorithms for test problems with 5-50 objectives, respectively.DTLZ1-5 represents that the number of objectives of DTLZ1 is 5.It can be seen from Tables 2 and 3 that, for the IGD metric, MODER outperforms NSGAII and NSGAII-CE on all twelve test problems and outperforms MOEA/D on nine test problems.For the GD metric, MODER outperforms NSGAII on all twelve test problems, outperforms NSGAII-CE on eight test problems, outperforms MOEA/D on seven test problems, and performs worse than NSGAII-CE and MOEA/D on four test problems.These results indicate that MODER outperforms all three compared algorithms.
These results of Tables 2 and 3 also show that the mean values of GD and IGD obtained by NSGAII are the largest among the four algorithms, which indicate that NSGAII has the worst performance in the convergence to the true PF among these four algorithms and Pareto dominance has little selection pressure toward the true PF.For problem DTLZ1 with 20-50 objectives, the mean values of IGD obtained by MODER are slightly larger than those obtained by MOEA/D, which indicates that MOEA/D can better maintain the diversity of solutions than MODER for these problems.However, for problem DTLZ1 with 5-15 objectives, the mean values of IGD obtained by MODER are smaller than those obtained imply that MODER can converge to the true PF and maintain the diversity of obtained solutions; even for higher dimensional objectives, its convergence and diversity performance are still good.In other words, the new ranking method not only can enhance the selection pressure toward the true PF but also can well maintain the diversity of solutions.

Conclusions
In this work, a new ranking method for many-objective optimization problems is proposed to decrease the number of nondominated solutions and reduce the number of comparisons in the process of verifying the nondominated solutions.
For each solution , the approach generates a small number of virtual vectors which are evenly distributed around the objective vector of the solution  and can determine whether  is a nondominated solution by only making the comparison between () and every virtual vector.Thus, this method not only uses a small quantity of comparisons when determining the nondominated solutions, but also is able to contract the nondominated area of the solution to increase the selection pressure toward the true PF.Moreover, a multiobjective adaptive differential evolution algorithm based on the new ranking method is designed for many-objective problems.The algorithm was tested on problems with up to 50 objectives.Compared with the existing algorithms, simulation results show that the proposed algorithm is able to maintain the diversity of the obtained solutions and has good convergence ability on problems with large number of many objectives.Future works include expanding the experiments to other problems and search another crossover operator which is more suitable for many-objective problems.

Table 3 :
IGD, GD, and HV obtained by MODER, MOEA/D, NSGAII, and NSGAII-CE on DTLZ3.These show that the convergence of solutions obtained by MODER to the true PF is good and the new ranking method can enhance the selection pressure toward the true PF.For problem DTLZ3 with 5-50 objectives, the mean values of IGD obtained by MODER are smaller than those obtained by MOEA/D and NSGAII-CE, which show that the diversity of solutions obtained by MODER is better than those obtained by MOEA/D and NSGAII-CE; the mean values of GD obtained by MODER are smaller than those obtained by MOEA/D and NSGAII-CE, which indicate that the convergence of solutions obtained by MODER is better than those obtained by MOEA/D and NSGAII-CE.These