Using Objective Clustering for Solving Many-Objective Optimization Problems

Many-objective optimization problems involving a large number (more than four) of objectives have attracted considerable attention from the evolutionary multiobjective optimization field recently. With the increasing number of objectives, manyobjective optimization problems may lead to stagnation in search process, high computational cost, increased dimensionality of Pareto-optimal front, and difficult visualization of the objective space. In this paper, a special kind of many-objective problems which has redundant objectives and which can be degenerated to a lower dimensional Pareto-optimal front has been investigated. Different from the works in the previous literatures, a novel metric, interdependence coefficient, which represents the nonlinear relationship between pairs of objectives, is introduced in this paper. In order to remove redundant objectives, PAM clustering algorithm is employed to identify redundant objectives by merging the less conflict objectives into the same cluster, and one of the least conflict objectives is removed. Furthermore, the potential of the proposed algorithm is demonstrated by a set of benchmark test problems scaled up to 20 objectives and a practical engineering design problem.


Introduction
Real-world engineering application problems often need to simultaneously optimize more than four objectives, called many-objective optimization problems [1].Existing multiobjective evolutionary algorithms (MOEAs) have been successfully applied to solve problems with only two or three objectives, but they are not appropriate for problems with even more objectives.Since conventional multiobjective evolutionary algorithms rely primarily on Pareto ranking to guide the search, this enforces only little selection pressure in a many-objective setting.The more the objectives are, the larger is the proportion of nondominated solutions in a population, which results in the loss of selection pressure to drive the population toward the Pareto front [2].Furthermore, the number of points required to approximate the Pareto front increases exponentially with the number of objectives, which makes it more difficult to capture the whole Pareto front for many-objective optimization.In addition, it is not possible to visualize the Pareto front with more than three objectives in a figure .The classical MOEAs such as NSGA-II and SPEA2 do not perform well in many-objective optimization cases.Over the last few years, a large proportion of hot issues in MOEAs have been related to many-objective optimization problems, and efforts have been made to deal with the aforementioned difficulties.The approaches for solving many-objective problems can be classified as follows.(1) The approaches based on the modifications of Pareto-dominance relations over the nondominated solutions (e.g., average and maximum ranking [3], favor preference relation [4], preference order ranking [5], and L-optimality [6]) and assigning different ranks to nondominated solutions: if the objectives are many, all the individuals in population are often Pareto-optimal solutions.There will be no difference of the selection pressure for the individuals in these algorithms.Thus, these algorithms cannot make a diverse search in the full Pareto front and can usually obtain a part of the Pareto front.(2) The approaches using some techniques to improve the scalability of MOEAs, for example, methods like MSOPS [7] and MSOPSII [8] use an aggregation method and perform many parallel searches using multiple conventional target vectors in different directions.Recently, surface evolutionary algorithm (SEA) [9] and a hybrid NSGA-II [10] were proposed, and they seem to be more efficient than the existing algorithms of this kind for many objectives.(3) The approaches based on finding the redundant objectives and reducing the number of objectives via removing some redundant objectives [11], in fact, there exists a kind of many-objective optimization problems with  objectives, where a subset of the original objectives can represent the optimization problem adequately, and the Pareto-optimal front is less than -dimensions [12].The objectives included in this subset are referred to as the essential objectives or nonredundant objectives, while the rest of the original objectives, which are unnecessary for the Pareto-optimal front and can be removed from the original set of objectives, are named redundant objectives.The process of removing the redundant objectives from the original objective set is called objective reduction or dimensionality reduction [13].A lot of research works are carried out around the objective reduction.Brockhoff and Zitzler [14,15] explored an objective reduction method for many-objective optimization problems.In their study, the effects on problems' characteristics by adding or omitting objectives are investigated and a general definition of conflicts between objectives is proposed as a theoretical foundation for objective reduction.Moreover, two greedy algorithms are proposed to reduce the number of objectives, one of which finds a minimum number of objectives and the other finds a -sized objective subset with the minimum possible error.Another method for objective reduction is based on the information of the correlation between pairs of objectives.Deb and Saxena [16,17] proposed a technique for reducing redundant objectives based on principal component analysis (PCA), which takes the correlation between objectives as an indicator of the conflict.A large set of nondominated solutions are generated by NSGA-II, and the correlation matrix is computed for analyzing the relation of the objectives.Meanwhile, the conflict between a pair of objectives is judged by the correlation coefficient calculated by the set of nondominated solutions in this pair of objectives.If two objectives are negatively correlated, they are in conflict with each other.This method aims at computing a set of most important conflicting objectives, which can be obtained by an analysis of the eigenvectors and eigenvalues of the correlation matrix.Furthermore, Saxena and Deb [18] developed two new nonlinear dimensionality reduction algorithms employing the correntropy and maximum variance unfolding, namely, C-PCA-NSGA-II and MVU-PCA-NSGA-II, respectively.They are suitable for the data points that live on a nonlinear manifold or the data structure that is non-Gaussian.At the same time, Jaimes et al. [13] developed another dimensionality reduction scheme based on an unsupervised feature selection technique.In their scheme, the original objective set is divided into homogeneous neighborhoods based on a correlation matrix generated from a set of nondominated solutions.The conflict degree between redundant objectives is proportional to their distance, that is, the more conflict between two objectives, the farther the distance between them in the objective space.Therefore, the most compact neighborhood is chosen as the most relevant objective set, and all the objectives in it except the center one are removed as redundant objectives.
The motivation of this paper is as follows.First, note that there are some limitations to use correlation coefficient to represent the relation between objectives [12,13,15,16].As well known, correlation coefficient can indicate the linear correlation between random variables.Similarly, it can make an analogy to the relationship between objectives.Thus, many scholars use correlation coefficient to represent the relation between objectives.However, nonlinear relation between objectives cannot be expressed by correlation coefficient.In order to overcome this shortcoming, by using the union of mutual information and correlation coefficient, a new metric called interdependence coefficient is proposed in this paper.Secondly, clustering algorithm is adopted to divide the original objective set into a few subsets with an aim at taking less conflict objectives into one cluster and assigning more conflict objectives into different clusters.Afterwards, the cluster which has the least conflict is chosen, and some of the objectives in it are removed based on some rules for the purpose of objective reduction.Here, partitioning around Medoid (PAM) clustering algorithm is borrowed to accomplish clustering.In this way, the procedure of objective reduction can be integrated with an MOEA to find a high quality Pareto-optimal front.This paper is organized as follows.The theoretical foundations are introduced in Section 2. Section 3 describes proposed objective reduction algorithm using objective clustering (OC-ORA).The simulation results are given and discussed in Section 4. Finally, conclusions are made in Section 5.

Many-Objective Problem and the Concept of Objective Reduction
Definition 1 (many-objective problem).Without loss of generality, the multiobjective optimization problems are mathematically defined as follows: where  = ( 1 , . . .,   ) is a solution vector of decision variables in the solution space  and   () is the th objective function in the objective space.If the number of objectives is more than four, the problem is named many-objective optimization problem.
Definition 2 (Pareto domination).A vector  is said to dominate another vector  if and only if Definition 3 (Pareto-optimal solution).A solution  ∈  is said to be Pareto optimal with respect to solution space  if and only if there is no  ∈  for which  dominates .
Definition 4 (Pareto-optimal set).Pareto-optimal set is the set that consists of all Pareto-optimal solutions in solution space , and the image of Pareto-optimal set in objective space is Pareto front.Accordingly, the analyst solving this type of problem has to decide whether all objectives are essential or not and employ an objective reduction algorithm to obtain an essential objective set   .

The Traditional Representation of the Correlation between a Pair of Objectives.
The correlation coefficient matrix is used to measure the conflict between each pair of objectives [12,13].This matrix is computed by using an approximation set of the Pareto-optimal solutions generated by MOEA, for example, NSGA-II.A negative correlation between a pair of objectives means that when one objective increases, the other will decrease, while a positive correlation represents the opposite.Thus, the more positive the correlation between two objectives is, the less conflict between them will exist, and one of the objectives can be regarded as the redundant one, which can be eliminated from the original objective set.
However, correlation coefficient can only indicate linear correlation between objectives, while the nonlinear relation cannot be expressed.In order to overcome this limitation, a new metric is proposed by using the union of mutual information and correlation coefficient in this paper to measure the correlation between objectives.The introduction of mutual information [19,20] is described as follows.

Mutual Information
Definition 8 (self-information of random event).Suppose  is a discrete random event and  is a discrete random variable, then the self-information of the random event  is defined by  () = −log 2  { = } . ( The function () can be interpreted as the amount of information provided by the event { = } or our uncertainty about the event { = } [19].According to this interpretation, the less probable an event is, the more information we receive when it occurs.A certain event (one that occurs with probability 1) provides no information, whereas an unlikely event provides a very large amount of information.
Definition 9 (self-information or entropy of random variable).Suppose that  is a discrete random variable; that is, its range  = { 1 ,  2 , . ..} is finite or countable.Let   =  { =   }.The self-information or entropy of random variable  is defined by It turns out that () is the expectation of (  ) over all possible events, and it can be thought of as a measure of the amount of information provided by an observation of  or our uncertainty about .
Definition 10 (conditional entropy).For a pair of random variables  and , a quantity ( | ) is called the conditional entropy of  with a given .More precisely, if ( |  = ) is the entropy of the variable  on condition of the variable  taking a certain value, then ( | ) is the result of averaging ( |  = ) over all possible values that may take as follows: Given the value of the other random variable , the conditional entropy quantifies the remaining amount of information needed to describe the outcome of a random variable .Here, ( | ) = 0 if and only if the value of  is completely determined by the value of .Conversely, ( | ) = () if and only if  and  are independent random variables.
Definition 11 (mutual information [19]).Consider two random variables  and  with a joint probability mass function (, ) and marginal probability mass functions () and ().The mutual information (; ) is the relative entropy between the joint distribution and the product distribution ()(), which can be defined by (6).Thus, the mutual information (; ) is the reduction in the uncertainty of  due to the knowledge of : One can see from the above expression that the mutual information is symmetric in  and .This symmetry means that this notion of uncertainty has the property that the information we gain about  when knowing  is the same as the information we gain about  when knowing .[21,22] proposed a clustering algorithm partitioning around medoids (PAM), which partitions a set of objects into  clusters, where the objects in one cluster show a high degree of similarity, while objects belonging to different clusters are as dissimilar as possible.PAM clustering algorithm only needs a distance matrix between objects and does not need the location of the objects or other information.Motivated by the idea of PAM, we use this clustering algorithm to divide the set of many objectives into different clusters.

PAM Clustering Algorithm. Kaufman and Rousseeuw
In PAM,  partitions for  objects are formed.Initially,  medoids (central points) are selected from the set of objects randomly.A medoid representing a cluster is located in the center of the cluster, and each remaining object is assigned to a cluster whose medoid is the nearest to this object.Then one of the medoids is replaced by a nonmedoid such that the quality of resulting cluster can be improved.The quality is estimated by a cost function that measures the average dissimilarity between every object in this cluster and its corresponding medoid.We replace the distance or similarity measure in PAM by a new metric named interdependence coefficient to measure the degree of correlation between pairs of objectives.may be the redundant ones.In OC-ORA, PAM is adopted to divide the current objective set into a number of clusters according to the correlation with objectives.Subsequently, we identify the most correlated pair of objectives in the most correlated cluster in order to remove the redundant objective for many-objective problems.In the proposed algorithm, the MOEA and the objective reduction are executed alternately; this process will end when no further objective reduction can be achieved.Figure 1 shows the procedure in the proposed OC-ORA.

Interdependence Coefficient between Pairs of Objectives.
Note that correlation coefficients can only reflect the linear relations between objectives [13], but they cannot represent nonlinear relations.In order to overcome this limitation, a new metric using the mutual information and correlation coefficient, named interdependence coefficient, is developed to describe the correlation between objectives.This new measure between pairs of objectives is also calculated based on a set of nondominated solutions generated by multiobjective evolutionary procedure via NSGA-II.

The Definition of Interdependence Coefficient.
For a pair of objectives   and   , its mutual information is defined as (  ;   ) by Definition 11.Note that mutual information is nonnegative, and thus it cannot distinguish the negative correlation.To overcome the shortcoming, the union of mutual information and correlation coefficient is used to measure the correlation relation between objectives.The new measure named interdependence coefficient, denoted by    ,  , is defined as follows: where    ,  and (  ;   ) represent the correlation coefficient and mutual information between a pair of objectives   and   , respectively.sign(   ,  ) is a symbolic function used to distinguish the positive and negative correlations between a pair of objectives.In addition, the mutual information is normalized in (7), and its value is limited in the range of [0, 1].Thus, interdependence coefficient    ,  ∈ [0, 2] is used to measure the degree of correlation.In this way, we could guarantee that the greater the value of the interdependence coefficient between two objectives is, the more conflict or the less interdependent between them will exist, and vice versa.Value 2 indicates that objectives   and   are completely negatively correlated or totally conflict with each other, and Value 0 indicates that the objectives are completely positively correlated or without any conflict with each other:

The Approximate Calculation of Entropy and Mutual
Information between Two Objectives.In order to facilitate understanding, we will analyze the process of calculating the mutual information between any pair of objectives   and   .
A set of nondominated solutions generated by multiobjective evolutionary procedure NSGA-II are taken as original data for calculating the entropy.In (6), the entropy of   ,   , and     , that is, (  ), (  ), and (    ), must be known before calculating the mutual information (  ;   ).In (4), for a given objective   , we take   as a random variable, denoted by  and the values of   on the nondominated solutions as the values of random variable .Meanwhile, we use the maximum and minimum values of   on these nondominated solutions to construct an interval [minimum, maximum] which can be seen as the region of random variable  and then divide it into many smaller subintervals.Here, we assume that all values of  corresponding to all nondominated solutions fall on arbitrary position of the interval with the same possibility.Then, we count the number of nondominated solutions in each subinterval and calculate the probability of the random variable  falling into each subinterval.This probability can be calculated by ( ∈ subinterval  ) =   /, where   denotes the number of nondominated solutions in the th subinterval ( = 1, 2, . ..) and  denotes the number of nondominated solutions.In order to simplify the problem and calculate the entropy of objectives   in (4), we assume that if the number of subintervals is sufficiently large, each subinterval can be approximately seen as a point and the variable  can be seen as a discrete random variable.Thus, the probability of the random variable  falling into a subinterval can be approximately regarded as the probability of  taking the middle point   of this subinterval, where   = (upper bound  − lower bound  )/2.In this way, the entropy (  ) can be calculated based on (4).Similar to the calculation of the entropy of one objective, we need to divide the region in which the nondominated solutions locate into many smaller subregions in twodimensional space and count the number of nondominated solutions in each subregion and calculate the probability of the two-dimensional random variable falling into each subregion in order to calculate (    ).

The Process of Objective Clustering and
Objective Reduction 3.2.1.The Procedure of Objective Clustering.After calculating the interdependence coefficient between every pair of objectives, we get an interdependence coefficient matrix with order , named  = (  ) × , in which each element   represents the interdependence coefficient between the th and the th objectives.This matrix is used to measure the degree of correlation between each pair of objectives.Then, we use PAM clustering algorithm to group all objectives into some small clusters.The reason of using PAM clustering algorithm is that it only needs a distance matrix between objects as the input, and it does not need the location of the objects or other information.Here, the interdependence coefficient matrix is taken as the distance matrix.The larger the interdependence coefficient is, the farther (less similar) the corresponding pair of objectives will be.
According to the procedure of PAM,  objectives are chosen arbitrarily from the original objective set as the initial centers of  clusters, and each of the other ( − ) objectives is classified into a cluster whose center is nearest to this objective.Next, a central objective is replaced by a noncentral objective repeatedly until the quality of the resulting cluster cannot be improved.In this way, the objectives in one cluster show a high degree of correlation, while objectives belonging to different clusters reflect more conflict.

The Process of Objective Reduction.
In the process of objective reduction, we calculate the interdependence coefficients matrix in each cluster and take the cluster containing the minimum interdependence coefficient as the most highly correlated cluster, and the pair of objectives with the minimum interdependence coefficient can be regarded as the most relevant objectives in the current objective set.Note that the more the minimum interdependence coefficient in the most highly correlated cluster close to zero, the less is the conflict of the corresponding pair of objectives.Here, we use a redundant threshold  to remove the redundant objective.If the minimum interdependence coefficient in the most highly correlated cluster is less than a predetermined threshold , one of the objectives in this pair will be removed from the current objective set; otherwise, all the objectives will be retained.In the proposed algorithm, the multiobjective evolutionary algorithm and the strategy of objective reduction are executed alternately where at most, one objective is removed in an iteration.Divide the objective set into 2 clusters: since 13 < 45, the cluster on the left is selected as the most highly correlated cluster The process of objective reduction consists of two steps.Figure 2 shows the main skeleton.
(1) Recalculate the interdependence coefficient matrix in each cluster obtained by objective clustering process.Take the th cluster containing the minimum interdependence coefficient, that is, the cluster with    = min ∈{1,2,⋅⋅⋅} min ,∈  (   ), as the most highly correlated cluster, where   and   are the objectives with the minimum interdependence coefficient    and are the candidates of redundant objective.Figure 2(a) shows two clusters determined by PAM with total six objectives marked from number 1 to 6, where red triangle represents the central point of the cluster and green circle represents noncentral point.As it can be seen from the figure,  13 is the minimum interdependence coefficient in the left cluster and  45 in the right cluster.Because of  13 <  45 , the cluster on the left is the most highly correlated cluster, and  1 or  3 is the candidate redundant objective.
(2) Remove one of the candidate redundant objectives.
Firstly, identify the value of the minimum interdependence coefficient    .If    > , the correlation between   and   is weak and all of objectives should be retained; else, check either   or   is the central objective in its cluster, if either of the two is central point, the other one can be removed as the redundant objective.If neither of them is the central objective, we could calculate the sum of interdependence coefficients between each of   and   and the other objectives in the current objectives set and take the one   or   with the smaller sum as the redundant objective, denoted by  redn = arg min{∑   ̸ =      ,  |   =   ,   }.As can be seen in Figure 2(b), objective 1 and 3 are the most highly correlated objectives in the cluster, since the 3th objective is the center in the left cluster, it will be retained and the objective 1 is regarded as the redundant objectiveto be removed.
Step 2. Initialize a random population   , run NSGA-II corresponding to   for   generations, and obtain a nondominated set   .
Step 3. Calculate the interdependence coefficient matrix based on the nondominated set   , and use the PAM clustering algorithm to divide the objective set   into  clusters.
Step 4. According to the clusters of objective set   obtained in Step 3, remove one of the redundant or the most interdependent objective from   according to the above objective reduction rules, and the remaining objective set is denoted as  +1 .

Simulation Results
To verify the performance of the proposed algorithm for objective reduction, we employ test functions DTLZ2 () and DTLZ5 (, ) [23][24][25] in the experiments.These test functions are described below.Furthermore, a real practical engineering design problem, storm drainage systems, is also used in the experiments to test the performance of the proposed algorithm.

Test Functions
DTLZ2 ().DTLZ2 is one of the test functions from a scalable test problems suite DTLZ formulated by K. Deb et al. [23], and none of the objectives is redundant in the problem.The motivation of choosing this test problem is to test whether the algorithm will remove any objective.If yes, it will indicate the algorithm is ineffective.We will show in the following experiment that the proposed algorithm does not remove any objective.An -objective formulation of DTLZ2 is shown as follows. Minimize where The total number of decision variables is  =  +  − 1, where  = 10 is used in the experiments.The Pareto-optimal solutions correspond to  *  = 0.5.

DTLZ5 (𝐼, 𝑀).
In the DTLZ test suite, DTLZ5 is modified to construct a set of test problems where the dimensionality of the Pareto front is less than the original number of objectives [23][24][25].In DTLZ5 (, ) problems,  represents the actual dimensionality of the Pareto-optimal front and  represents the original number of objectives.The motivation of designing these test problems is to evaluate objective reduction techniques for many-objective optimization problems.The formulation of DTLZ5 (, ) is given as follows. Minimize where where The total number of decision variables is  =  +  − 1, where  = 10 is used here.With regards to redundant objectives, all objectives with { 1 , . . .,  −+1 } are positively correlated, while each objective in { −+2 , . . .,   } is conflicting with every other objective in the problem;   = {  ,  −+2 , . . .,   } defines the true POF [18], where  ∈ {1, 2, . . .,  −  + 1}.

Parameter
Setting Used in OC-ORA.The crossover and mutation parameters for OC-ORA used in the experiments are listed in Table 1, and the experiments are done on different numbers of objectives for each test problem.The population size and the number of generations in different objective test problems are shown in Table 2.In calculating the selfinformation or entropy of an objective, we divide the interval on one objective into many subintervals.The number of subintervals is set as 20; that is, we will calculate the selfinformation or entropy of a discrete variable on 20 possible points, and the number of predefined clusters  is set from 2 to ⌊ √ ⌋.In the process of objective reduction, the threshold  [26] is set as 0.6.
For performance assessment, some evaluation criterions, such as computational complexity and the success rate in identifying the true PF, are used here to compare the performance of the different algorithms.

Complexity Comparison of OC-ORA and Other Objective Reduction
Algorithms.The computational complexity of the OC-ORA consists of three parts: executing the evolutionary multiobjective algorithm, calculating the interdependence coefficient matrix between pairs of objectives, and implementing the PAM clustering algorithm to reduce redundant objectives.The complexity of the proposed algorithm is ( 2 ) + (V 2  2 +  2 ), where  is the number of generations for each run of NSGA-II,  is the size of the nondominated set,  is the number of objectives in the current nonredundant objective set,  is the number of clusters used in PAM clustering algorithm, and V is the number of subintervals in the calculation of mutual information.Generally,  is much smaller than V 2 , so the complexity of OC-ORA is simplified as ( 2 ) + (V 2  2 ).In contrast, the computational complexities of the compared objective reduction approaches are summarized in Table 3.Note that each of the objective reduction algorithm operates on the nondominated set and share the same complexity on obtaining the nondominated set; hence, it is unnecessary to consider the computational complexity ( 2 ) of obtaining the nondominated set in each objective reduction algorithm.Thus, the complexity of OC-ORA is simplified as (V 2  2 ).It can be seen from Table 3 that (1) the computational complexity of the exact algorithm for −MOSS is almost impractical since it is quadratic in  and exponential in  and the computational complexity of the greedy algorithm for  − MOSS is likely to be more expensive.In summary, the complexity of  − MOSS is the worst compared other algorithms.(2) The complexity of the procedure of objective reduction in one iteration is listed in Table 3, which shows that the efficiency of OC-ORA is better than NL-MVU-PCA based reduction method.Besides, the population size and the number of generations of the proposed algorithm in one iteration of objective reduction are much less than those of the compared algorithms.For example, the population size and the number of generations are set to be 800 and 1000 in NL-MVU-PCA, which are much larger than those of the proposed algorithm.Although the iterations used by the proposed algorithm OC-ORA may be more than that used by NL-MVU-PCA because at each iteration, OC-ORA removes one redundant objective, while NL-MVU-PCA removes more than one the total number of individuals used by the proposed algorithm, which is a relatively fair metric to measure the computational complexity of an algorithm, is smaller than that used by NL-MVU-PCA.Thus, the computational complexity of the proposed algorithm is lower than that of the compared algorithms.

The Example Analysis on the Process of Objective Reduction.
In order to verify the efficiency of interdependence coefficient matrix [26] in measuring the relation between objectives, we use the interdependence coefficient matrix to implement objective clustering and reduction on both redundant test functions DTLZ5 (3,5) and nonredundant test functions DTLZ2 (5) problems.The processes of objective clustering and redundant objective removing are presented in Table 4.In the original objective set in DTLZ5 (3,5) is   = {1, 2, 3, 4, 5}.In order to estimate the correlation between each pair of objectives, the interdependence coefficient matrix is computed on the nondominated set generated by NSGAII.An interdependence coefficient matrix with order five is presented in the left part of Table 4(a), named  = (  ) 5×5 .According to the interdependence coefficient matrix, the objective clustering algorithm is carried out to divide the objective set   into different  clusters, where  is predetermined and set to 2. Thus, the original objective set   is divided into two subsets  1 = {1, 2, 3} and  2 = {4, 5} by using the PAM clustering algorithm.Then, the objective reduction algorithm is performed to remove the redundant objective, where the first objective  1 is the redundant objective which should be removed from the current objective set, and thus the resulting nonredundant objective set is    = {2, 3, 4, 5}.Afterwards, the next round of calculating the interdependence coefficient matrix in the new objective set    = {2, 3, 4, 5} is started, and the matrix with order four is shown on the right part of Table 4(a).Through the new round of PAM clustering and objective reduction strategy, the second objective  2 satisfies the condition of redundant objective.Finally, after two iterations, the nonredundant objective set is    = {3, 4, 5}, which is the true nonredundant objective set in DTLZ5 (3,5).With nonredundant test functions DTLZ2 (5) problems, we calculate the interdependence coefficient matrix on original objective set   = {1, 2, 3, 4, 5}, shown in Table 4(b).In the process of objective clustering, the number  of clusters is also set to 2, and the original objective set   is divided into two subsets  1 = {1, 4} and  2 = {2, 3, 5}.The minimum interdependence coefficient in two clusters is  14 = 0.6513, which represents that  1 and  4 are the most highly correlated objectives in   .According to the rule of identifying redundant objectives, the value 0.6513 is larger than threshold , so neither of them will be removed.

Comparison of Success Rate in Identifying the True
Nonredundant Objective Set   .To test the performance of the proposed algorithm for objective reduction, two different kinds of the test problems with varying number of objectives are studied, including 10 test examples.For each test example, experiments are performed for 20 independent runs.Table 5 summarizes the results of the success rate in identifying the true nonredundant objective set   with objective number increasing from 5 and 10 to 20.Meanwhile, we also compare the success rates of OC-ORA and linear objective reduction approach L-PCA [18].The experiment results are shown as follows.
For DTLZ2 (), it can be seen from Table 5 that OC-ORA can identify the true Pareto front accurately with success rate of 100% for 5 and 10 objectives, respectively, and 95% for 20 objectives.The success rates are much higher than those obtained by L-PCA.In nonredundant problems, the threshold  avoids removing any nonredundant objective and tries to keep all of objectives.
For all instances of DTLZ5 (, ), it also can be seen from Table 5 that the success rates obtained by OC-ORA are obviously much higher than those obtained by L-PCA.The superiority of OC-ORA is that it could express more comprehensive correlation between objectives, especially for nonlinear relationships of objectives.The experimental results indicate that OC-ORA could find the true nonredundant objective set efficiently.
The limitation of the proposed algorithm is that the number of clusters  must be smaller than the number of nonredundant objective in test problem.When the number of clusters is more than the number of nonredundant objectives, the objective reduction strategy is not applicable.

An Engineering Problem: Storm Drainage
Systems.This is an optimal planning problem for storm drainage systems in urban areas, which is proposed by Musselman and Talavage [27].The problem consists of 5 objectives and 7 constraints.The analytical model of the problem is given in Table 6.In order to identify the redundant objectives of the problem, the proposed algorithm is carried out.The population size is set to 200, and the generation is set to 200.In the original objective set   = {1, 2, 3, 4, 5}, the interdependence coefficient matrix based on the original objective set is shown in Table 7.After calculating the interdependence coefficient matrix, we will execute objective clustering process to divide the original objective set   into two subsets  1 = {1, 2, 3} and  2 = {4, 5}.Comparing the minimum interdependence coefficient in each cluster, the cluster  1 is identified as the most highly correlated cluster, and the interdependence coefficient between  1 and  3 is the minimum.According to the objective reduction rule,  1 is considered as the redundant objective, and thus the corresponding nonredundant objective set is    = {2, 3, 4, 5}.Thus,    can be used to reconstruct the Pareto front.
To validate this result, parallel coordinate plot is borrowed here to visualize the nondominated solution set with more objectives in a figure.It involves plotting the normalized objective values of the nondominated solutions onto parallel axes, one per normalized objective.The function values in every objective for each nondominated solution are connected by a line segment.The parallel coordinate plots corresponding to the original set of objectives   = {1, 2, 3, 4, 5} and the reduced set    = {2, 3, 4, 5} are shown in Figures 3(a) and 3(b), respectively.Figure 3(a) shows the parallel coordinate plot corresponding to the original set of objectives, and Figure 3(b) refers to the reduced set.It can be seen from Figure 3 that parallel coordinate plot corresponding to the reduced set of objectives closely matches with that obtained using the original set of objectives.This illustrates that the omitting objective  1 does not affect non-nominated set, and the reduced set of objectives    = {2, 3, 4, 5} is enough to obtain the Pareto front for this problem.

Conclusion and Future Work
In this paper, a novel method has been proposed to identify the true nonredundant objective set in many-objective problems.In order to overcome the defects of tradition-al methods in quantitative representation of the relation between a pair of objectives, we adopt a new metric, interdependence coefficient, by using the union of mutual information and correlation coefficient to measure the correlation between objectives.In addition, a new objective reduction strategy is investigated in accordance with the results of PAM clustering algorithm.The efficiency of the proposed approach is demonstrated by experiments on two kinds of benchmark test problems including 10 test instances and a real engineering practice problem, where the number of objectives tested is from 5 to 20.Moreover, a comparative analysis of computational complexity and success rate between the proposed algorithm and the correlation matrix-based algorithms has been made.All the results show that the proposed algorithm performs well in finding the true nonredundant objective set and outperforms the compared algorithm.
A number of future works can be further conducted from current work.First, the strategy of removing redundant objective can be further enhanced by designing a specific method, which should avoid the limitation of the provision of the cluster number .Also, it is important to realize that for many-objective problems, different parts of the Pareto front may give different non-redundant objective set.In such a case, the objective space needs to divide many subregions to discuss.

Figure 1 :
Figure 1: Flow chart of the proposed OC-ORA.

Figure 2 :
Figure 2: The strategy of the objective reduction method employed.

Table 1 :
Parameters used for OC-ORA algorithm.

Table 2 :
The pop size and generations in different test problems.

Table 5 :
The success rate in identifying the true nonredundant objective set   with two algorithms out of 20 runs.