A Genetic Algorithm with Fuzzy Crossover Operator and Probability

The performance of a genetic algorithm is dependent on the genetic operators, in general, and on the type of crossover operator, in particular. The population diversity is usually used as the performance measure for the premature convergence. In this paper, a fuzzy genetic algorithm is proposed for solving binary encoded combinatorial optimization problems. A new crossover operator and probability selection technique is proposed based on the population diversity using a fuzzy logic controller. The measurement of the population diversity is based on the genotype and phenotype properties. In this fuzzy inference system, the selection of the crossover operator and its probability are controlled by a set of fuzzy rules derived from the fuzzy logic controller. Extensive computational experiments are conducted on the proposed algorithm, and the results are compared with some crossover operators commonly used for solving multidimensional 0/1 knapsack problems published in the literature. The results indicate that the proposed algorithm is effective in finding better quality solutions.


Introduction
Premature convergence is a common problem in finding the optimal solution in Genetic Algorithm GA and it is strongly related to the loss of the population diversity.When population diversity is low, a GA will converge very quickly.On the other hand, if the diversity of the population is too high, it is very time consuming for a GA to converge and this may cause wastage in computational resources.
The performance of a GA is dependent on the genetic operators in general and on the type of crossover operator, in particular.During the evolution process by a GA, if the selected chromosomes are identical, some of the crossover operators have failed to create offspring that are different from their parents.Effective crossover in a GA is achieved through establishing the optimum relationship between the crossover and the search problem itself.
In this paper, a fuzzy genetic algorithm FGA is proposed for solving binary encoded combinatorial optimization problems like the Multidimensional 0/1 Knapsack Problem MKP .The aim is to design a crossover operator and probability selection technique based on the population diversity using Fuzzy Logic Controller FLC .The diversity of the population is measured on the basis of the genotype and phenotype characteristics of the chromosomes.In addition, a new technique based on Hamming Distance HD , Fitness Value FV , and Active Genes AG of the mate chromosomes is proposed during the sexual selection.
In recent years, the MKP has turned out to be the favourite playground for experimenting with metaheuristics.It serves as a suitably "difficult" test problem for developing advanced techniques in this field.The MKP is a general statement of any binary encoded problem with nonnegative coefficient.It is formulated as follows Djannaty and Doostdar 1 : max f x 1 , x 2 , . . ., x n n j 1 p j x j s.t.w ij x ij ≤ c i i 1, 2, . . ., m, x j ∈ {0, 1} j 1, 2, . . ., n, with p j > 0, w ij ≥ 0, c i ≥ 0, 1.1 where n number of objects; m number of knapsacks; w ij consumption of resource i for object j; c i capacity of the ith knapsack; p j profit associated with object j; x j decision variable with object j.
The remainder of this paper is organized as follows.Literature review is given next in Section 2. Section 3 elaborates on the population diversity.In Section 4, the proposed FGA is described in detail.The computational results of the FGA for MKP based on the benchmark problem instances deflated from the literature are presented in Section 5. Finally, a brief conclusion is given in Section 6.

Literature
Evolutionary algorithms usually use the population diversity as the performance indicator for a number of reasons like avoiding premature convergence, controlling algorithm restarting or stopping when the population diversity drops below certain thresholds, the necessity for assessing a population of distinct Pareto-optimal solutions in an optimization problem, and the need for rapid adaptation of a population to dynamic problems see Some of the literature on the population diversity focuses on the genetic operators and their probabilities.Sywerda 5 and De Jong and Spears 6 compared the various crossover operators, particularly the numbers of crossover points.Their results demonstrated that sometimes the final output would be better if the number of crossover points was increased.Besides empirical analysis, substantial efforts have been invested in comparing, from theoretical perspectives, between mutation and crossover as well as between the various crossover operators 6 .However, these theories are not general enough to allow for predicting when to apply or what type of crossover operator to employ.For instance, those theories did not address the size of the study population even though it may affect the relative usefulness of the crossover operators 7 .Likewise, there is enough evidence suggesting that the relative usefulness of mutation may be influenced by the size of the population.Mutation seems to be more advantageous than crossover with small population sizes whereas crossover may be more advantageous than mutation with large population sizes 8 .Moreover, the relative usefulness of mutation and crossover are influenced by several other factors like, fitness function, representation, and selection scheme.
The selection of crossover probability p c critically affects the behaviour and the performance of a GA.It controls the rate at which solutions are subordinate to crossover where for each problem the probability is chosen by the user.Goldberg and Sastry 9 generalized the schema theorem for p c .They argued that the selection pressure corresponds to disruption of schema, and they showed that when the building blocks are compact its genes are located close to each other in the chromosome string , the GA works well for a wide range of combinations of p c and selection pressures.
Adaptive techniques that alter the probability of applying an operator in proportion to the observed performance of the chromosomes in the course of a run were proposed by Davis 10 . Fernandes et al. 11 considered a GA that adapts the reproduction rate to the size of the population under investigation.Annunziato and Pizzuti 12 presented a technique for setting the genetic parameters during the implementation by adapting the population size and the operators' rate to the environmental constraint of maximum population size.On the other hand, Srinivas and Patnaik 13 adopted a messy approach towards the determination of p c based on the various fitness values of the population.They presented an approach that eliminates the need for parameters in a crossover-based GA.Cao and Wu 14 proposed a systematic stochastic algorithm for optimization of the control parameters for GAs.In their research, a GA is modelled as a controlled Markov chain whose transition depended on the control probability of crossover and mutation.Hong et al. 15 introduced a dynamic GA that simultaneously used more than one crossover and mutation operators to generate the next generation.The operators' rate is varied along with the evaluation results of the respective offspring in the next generation.
Numerous researchers used different parameters as the input variables in FLC for dynamically control the crossover and mutation probabilities during the GA implementation.Song et al. 16 considered the alteration of the average fitness value between two consecutive generations as the input variables.Yun and Gen 17 later introduced a scaling factor for normalization of the input variables used in 16 .Li et al. 18 utilized the information of both the whole generation and the particular chromosomes as the input variables.Wang et al. 19 considered the differences in the fitness values between two consecutive generations, and Liu et al. 20 used the average fitness value and the best fitness value of chromosomes in each generation as the input variables.
Lee and Takagi 21 considered three inputs for the fuzzy system: x 0 f avg /f best , x 1 f worst /f avg , and x 2 is the change in best fitness since the last control action input variables f avg , f best , and f worst are the average, best, and worst fitness values, respectively.Corresponding to these inputs, three outputs are introduced for the current population size, crossover probability, and mutation probability.They tested the proposed technique on the inverted pendulum control task in 22 .However, this technique had two major defects: i only the phenotype characteristic is used as the input variable, ii using the metalevel GA to find the optimized fuzzy system for the dynamic parameterized GA is computationally expensive.
Im and Lee 23 presented adaptive crossover, mutation, and selection using fuzzy system for GAs.However, it is not clear how fuzzy system is based and which parameter of GA is used for the selection rules and the membership functions.
The main problem of FLC is how to gain the fuzzy rules the inference system is based on.Several approaches have been proposed by automatic rule base identification.A family of bacterial type evolutionary algorithms has been successfully applied for solving this task.The goal is to create more accurate fuzzy rule bases from input-output data sets as quickly as possible.A comprehensive review on these bacterial-type evolutionary algorithms is given in 24 .

Population Diversity
The approach adopted in this study attempts to define a measure for the population diversity on the basis of the genotype and phenotype characteristics.The question may be raised as to why the phenotype and genotype characteristics are used together in this study.A simple example will be used to answer the question.Let f x be a linear function for which a maximum value needs to be defined as follows: 3.1 To solve 3.1 using GA, let us say five chromosomes with length 15 are considered as the initial population.In Table 1, the goal is to evaluate the population diversity based on the genotype property.To activate this, the HDs between the chromosomes with the highest fitness value and other chromosomes are calculated.Note that the HD can be defined as follows.
Definition 3.1 Hamming Distance .Let C i {c i1 , c i2 , . . ., c im } and C j {c j1 , c j2 , . . ., c jm } be two chromosomes within a population with length m, the Hamming Distance HD between C i and C j is As seen in Table 1, the HD is at most 2 while the length of chromosomes is 15.This means that the genetic diversity of the population is low.However, by considering the FVs of

Chromosome
Hamming Distance HD Fitness value FV the chromosomes, the range is somehow relatively high.Therefore, the genotype property of the chromosomes alone is not enough for explaining the diversity of the population.
In Table 2, a different initial population is used to evaluate the population diversity based on phenotype property.To achieve this, the FVs of the chromosomes are considered.As seen in Table 2, the range of differences between the chromosomes in terms of FVs is low, and hence it seems that the diversity of the population is low although the HD between the chromosomes is high.Therefore, the phenotype property of the chromosomes alone is not enough for justifying the diversity of the population.
In light of this, the genotype and phenotype properties are both considered in this paper for the measurement of the population diversity.In the case of the phenotypic measures, the methods presented by Srinivas and Patnaik 13 and Zhu and Liu 25 are used with slight modifications: where N population size, n f i number of unique fitness values in the population, f max,t maximum fitness value in generation t and f avg,t average fitness value in generation t.
In the case when the genotypic measure is used, an equation similar to part of the technique proposed by Jassadapakorn and Chongstitvatana 26 is used: where HD C f max,t , C f min,t HD between the chromosomes with the highest and the lowest FV, and L length of the chromosome.
In this technique, the HD is calculated in each generation for only two chromosomes: the chromosome with the maximum fitness value C f max and the one with the minimum fitness value C f min .It should be highlighted, however, that in the method of 26 , the HD is calculated between the first selected chromosome and the candidate chromosome c i , where i 1, 2, . . ., population size.Therefore, our proposed method requires less time than that of 26 .
T 1,t , T 2,t , and T 3,t belong to the interval 0, 1 .As the T 1,t and T 2,t approach zero, the FVs of the chromosomes are almost identical and this shows that the convergence has taken place whereas if these are close to one, the population shows a high level of diversity.Similarly,
When we speak of population diversity in terms of low, medium, or high diversity, we can see that the language becomes a fuzzy variable whose spatial denotation is imprecise.In this sense, fuzzy theory becomes easily understood because it can be made to resemble a high level language instead of a mathematical language.To describe this, fuzzy sets with names such as Low, Medium, or High are used to create a membership function.By determining the degree of membership of an input in the fuzzy sets of this membership function, one can see the role of membership functions play in decoding the linguistic terminology to the values a computer can use.In most cases, membership functions are designed by experts with knowledge of the system being analyzed.In this paper, with regards to T 1,t , T 2,t , and T 3,t , three membership functions are defined.The set of linguistic labels associated with T 1,t and T 3,t are Low, Medium, and High and those related to T 2,t are Low and High.The semantic meaning of these labels is illustrated in Figure 1.

Fuzzy Genetic Algorithm
In this section, we concentrate on the discussion of the proposed FGA for solving binary encoded combinatorial optimisation problem specifically on the MKP defined in Section 1.In the remainder of this section, we explained the framework of the proposed FGA illustrated in Figure 2 in more detail.

Gene Representation and Population
For the MKP, a solution can be simply encoded by a string of 0 and 1s, where "1" encodes that an item is being selected and "0" means otherwise.The length of the chromosome corresponds to the number of items n, in the MKP.
A good initial population makes it easy for a GA to converge to good solutions while a poor initial population can prolong a GA convergence.There are different approaches in generating initial population for a GA.The most common method is by random generation.In this paper, we use the random generation method proposed by Chu and Beasley 27 to achieve better diversity in the population.The pseudo code for the initial population is given in Algorithm 1.Note that the population size is kept constant throughout the implementation.
After new offspring are created via the crossover and mutation operators, there may be some offspring that have infeasible solutions.There are several ways to deal with the infeasible solutions in a GA as stated below: i utilize an encoding method that produced only feasible solutions, ii apply a repair mechanism to transform any infeasible solution into a feasible solution, iii separate the evaluation of fitness between the feasible and infeasible solutions, or iv apply a penalty function to penalise the fitness of any infeasible solution without distorting the fitness landscape.
In this paper, we use the repair operator introduced by Chu and Beasley 27 that converts an infeasible solution into a feasible solution as presented in Algorithm 2.

Sexual Selection
In a standard GA, chromosomes reproduce asexually, that is, any two chromosomes may be the parents in crossover.Gender division and sexual selection inspired a model of gendered GA in which crossover takes place only between chromosomes of opposite sex.Since GA mimics natural evolution, then gender is one of the crucial elements in a GA.Inspired by the nongenetic sex determination system prevalent in some reptile species where sex is determined by the temperature at which the egg is incubated, the population are divided such that the male and female would be selected in an alternate way.The layout of the male and female chromosomes in each generation is different.In other words, when the number of generation is even, the chromosomes C i and C i 1 are the male and female, respectively, and vice versa when the number of the generation is odd.For instance, suppose that a population size is five, the layout of males and females from this population is shown in Table 3.
During the proposed sexual selection, a female chromosome is selected by tournament selection size t from the female group.The selected female chromosome is denoted as female chro.On the other hand, the selection of male chromosome male chro for crossover is based on the HD from female chro, FV, or AG of the male chromosome.Note that the AG can be defined as follows.
Definition 4.1 Active Gene .The Active Gene denoted by AG C i is the number of nonzero gene in chromosome C i .For instance, let C i 1 0 0 1 0 1 0 1 1 0 0 0 1 0 0 , then AG C i 6.At first, t male chromosomes are selected randomly from the male group.Then, the male chro is selected, in order of preference based on i the maximum HD between the male chromosome and the female chro, ii the highest FV of male chromosome if more than one male chromosome is having the maximum HD , iii the highest AG of the male chromosome if more than one male chromosome having the highest FV is found , or iv random selection.
The computational experiments of the proposed sexual selection compared to other selection mechanisms from the literature can be found in Varnamkhasti and Lee 28 .Computational results on MKPs showed that the proposed sexual selection mechanism is comparable in providing high-quality solutions in reasonable execution time.

Fuzzy Crossover Operator and Probability
In this subsection, we propose a crossover operator and probability selection technique based on the population diversity using FLC.For selecting the most appropriate crossover operator, the structure of the crossover and the population diversity must be considered.In this paper, some commonly used crossover operators for binary encoding are considered as the default crossover operators.These operators and their abbreviations are given in Table 4.It is worth mentioning that, if crossover is not applied based on the given probability, then the two offspring are simply duplicates of their parents.
When the genetic diversity of the population is high T 3,t ≥ 0.50 , the 2PC operator performs relatively well compared to other crossover operators especially in terms of the computational time.The genetic diversity of the population is medium 0.25 ≤ T 3,t ≤ 0.50 when some loci in the chromosomes are the same.In such cases, the 2PC operator is not an appropriate choice, but the KPC, and UC operators would be more suitable.Both operators are capable of keeping the genetic diversity of the population while improving the fitness values of the chromosomes.On the other hand, the genetic diversity of the population is low T 3,t ≤ 0.25 when some chromosomes are identical.The 2PC, KPC, and UC operators failed to produce offspring that are different from their parents.In this case, the SC and IC operators will be more appropriate.Based on the above explanation of the relative crossover operators and genetic diversity, three levels for Crossover Ability CA called Low, Medium, and High are introduced and presented in Table 5.
As discussed earlier, genetic diversity alone is not enough to justify the population diversity as a whole.Hence, we propose a fuzzy system which takes three input variables and produces two output variables: T 1,t , T 2,t , and T 3,t are the input variables while CA and p c are the output variables.The set of linguistic labels associated with the output variables comprises the descriptions of Low, Medium, and High.For each linguistic term, there is a triangular fuzzy set that defines its semantic meaning as shown in Figure 3.
The linguistic rules describing the control system consist of two parts: an antecedent block between the IF and THEN and a consequent block following THEN .By making this type of evaluation, fewer rules can be evaluated, thus simplifying the processing logic and perhaps even improving the fuzzy logic system performance.Each rule has the possibility of generating a single rule for each output variable.In this paper, the inputs are combined logically using the AND/OR operator to produce output x i , μ i CA and y i , μ i p c response values for all the expected inputs: Therefore, the proposed fuzzy system with three input variables will have 18 3 × 2 × 3 rules for each output variable CA and p c .This fuzzy rule base is collectively presented in Table 6.The fuzzy outputs for all rules are finally aggregated to one fuzzy set.To obtain a crisp decision from this fuzzy set, we use the Centre of Gravity approach for defuzzification:

Elitism Replacement with Filtration
After the fuzzy crossover and the standard binary mutation operators are applied, elitism replacement technique is used as the replacement strategy.The offspring have to compete with their parents in order to allow transition into the new population.In other words, problems contains 27 different problem sets, each having 10 randomly generated instances, thus a total of 270 problems.
In these problem sets, w ij are drawn from discrete uniform generator U 0, 1000 and the right hand side coefficients c i , i ∈ {1, . . ., m}, are set using c i α n j 1 w ij where α is the tightness ratio.The objective function coefficients p j , j ∈ {1, . . ., m} are correlated to w ij and are generated as p j m i 1 w ij /m 500e j , where e j is a real number drawn from the continuous uniform generator U 0, 1 .
The performance of these approaches is measured by the Percentage Deviation PD between the benchmark problems taken from the ORLibrary 29 and our results, that is: where B i is the benchmark result and R i is the computational result obtained by the proposed algorithms.
In order to scrutinize the performance and the ability of the proposed FGA compared to standard genetic algorithms SGAs , the crossover operators mentioned in Table 4 are used in the computational experiments.The difference between the proposed FGA and a SGA is with regards to the use of the crossover operator and its probability.In other words, a SGA will employ one crossover operator with fixed probability, p c 0.70 during implementation.Thus, there will be five SGAs as compared to one FGA in the experiment.Each algorithm is implemented with a population size of 100.Sexual selection described in Section 4.2, the replacement strategy explained in Section 4.4, and standard binary mutation with probability p m 1/L L is the length of chromosome are used as the default genetic operators.All algorithms were coded in C and ran on a Pentium IV with 2.0 GHz CPU and 2.0 GB of RAM.For each problem instance, 30 runs were performed.In order to have a fair comparison between the algorithms in this experiment, we employed a duration of 20 CPU seconds per run.Results are listed in Table 7.The benchmark column represents the average computational results of 10 problem instances from each problem set of the MKPs that are compiled in 29 .For each algorithm, the entries report the average PD computed over 10 problem instances with 30 runs each, that is, 300 runs.For each category of m, the final line gives the average PD over all values of n.The final line of Table 7 gives the overall average PD over all categories of m.
We first observed that the FGA performs significantly better than the SGAs.This shows that the FGA is able to produce better quality solutions compared to SGAs in a fixed CPU time.There is clear evidence from Table 7 that, on average, the FGA is the best algorithm followed by the SGAs with SC, IC, UX, KPC, and finally the SGA with 2PC.The algorithms find problems of m 30 to be the most challenging.With other things i.e., n and α being equal, when we increase the value of m, then PD will increase.Note that for all the algorithms, fewer generations are executed within the time limit as the number of knapsacks or constraints become larger.

Conclusion
Genetic algorithms usually use the population diversity as the performance measure for a number of reasons such as avoiding premature convergence.This paper proposed techniques for the measurement of the population diversity based on the phenotype and genotype properties.The main contribution of this study is the crossover operator and probability selection technique based on the population diversity using fuzzy logic controllers.The performance of the proposed fuzzy genetic algorithm compared to other standard genetic algorithms for solving the multidimensional 0/1 knapsack problems showed that the proposed algorithm is effective in finding better and comparable solutions.Future research efforts will investigate extending the proposed fuzzy genetic algorithm to include the fuzzy mutation operator and probability selection technique for other binary encoded combinatorial optimisation problems.In addition, we may consider to extend the research for integer encoded fuzzy genetic algorithm.

Figure 1 :
Figure 1:The set of linguistic labels associated with T 1,t , T 2,t , and T 3,t .

Figure 3 :
Figure 3: The set of linguistic labels associated with CA and p c .
Tomassini et al. 2 , Morrison and De Jong 3 , and Wineberg and Oppacher 4 for more details .

Table 1 :
A population with low genetic diversity based on genotype property HD but with high population diversity based on phenotype property FV .

Table 2 :
A population with low population diversity based on phenotype property FV but with high genetic diversity based on genotype property HD .
Repair operator source: Chu and Beasley 27 .

Table 3 :
The layout of male and female chromosomes.

Table 4 :
Abbreviations of crossover operators.

Table 5 :
Levels of crossover operator based on CA.

Table 7 :
Comparative computational results 20 CPU seconds per run .