Hybrid Biogeography-Based Optimization for Integer Programming

Biogeography-based optimization (BBO) is a relatively new bioinspired heuristic for global optimization based on the mathematical models of biogeography. By investigating the applicability and performance of BBO for integer programming, we find that the original BBO algorithm does not perform well on a set of benchmark integer programming problems. Thus we modify the mutation operator and/or the neighborhood structure of the algorithm, resulting in three new BBO-based methods, named BlendBBO, BBO_DE, and LBBO_LDE, respectively. Computational experiments show that these methods are competitive approaches to solve integer programming problems, and the LBBO_LDE shows the best performance on the benchmark problems.


Introduction
An integer programming problem is a discrete optimization problem where the decision variables are restricted to integer values. In computer science and operations research, a remarkably wide range of problems, such as project scheduling, capital budgeting, goods distribution, and machine scheduling, can be expressed as integer programming problems [1][2][3][4][5][6]. Integer programming also has applications in bioinspired computational models such as artificial neural networks [7,8].
The general form of an integer programming model can be stated as where ⃗ is a -dimensional integer vector, Z is adimensional discrete space of integers, and ⃗ is a feasible region that is not necessarily bounded. Any maximization version of integer programming problems can be easily transformed to a minimization problem.
One of the most well-known deterministic approaches for solving integer programming problems is the branch-andbound algorithm [9]. It uses a "divide-and-conquer" strategy to split the feasible region into subregions, obtaining for each subregion a lower bound by ignoring the integrality constraints and checking whether the corresponding solution is a feasible one; if so, the current solution is optimum to the original problem; otherwise recursively split and tackle the subregions until all the variables are fixed to integers.
However, integer programming is known to be NPhard [10], and thus the computational cost of deterministic algorithms increases very rapidly with problem size. In recent years, evolutionary algorithms (EA), which are stochastic search methods inspired by the principles of natural biological evolution, have attracted great attention and have been successfully applied to a wide range of computationally difficult problems. These heuristic algorithms do not guarantee finding the exact optimal solution in a single simulation run, but in most cases they are capable of finding acceptable solutions in a reasonable computational time.
Genetic algorithms (GA) are one of the most popular EA, but the encoding of the integer search space with fixed length binary strings as used in standard GA is not feasible for integer problems [11]. Many other heuristics, such as evolutionary 2 The Scientific World Journal strategy (ES) [12], particle swarm optimization (PSO) [13], and differential evolution (DE) [14], are initially proposed for continuous optimization problems. However, they can be adapted to integer programming by embedding the integer space into the real space and truncating or rounding real values to integers, and the applicability and performance of such approach are demonstrated by experimental studies.
Kelahan and Gaddy [15] conducted an early study that performs random search in integer spaces in the spirit of a (1 + 1)-ES; that is, at each iteration a child solution vector is generated by adding a random vector to the parent vector, and the better one between the parent and the child is kept for the next generation. Rudolph [11] developed a ( + )-ES based algorithm, which uses the principle of maximum entropy to guide the construction of a mutation distribution for arbitrary search spaces.
Laskari et al. [16] studied the ability of PSO for solving integer programming problems. On their test problems, PSO outperforms the branch-and-bound method in terms of number of function evaluations (NFE), and PSO exhibits high success rates even in cases where the branch-and-bound algorithm fails. Improved versions of PSO, including the quantum-behaved PSO which is based on the principle of state superposition and uncertainty [17] and barebones PSO which is based on samples from a normal distribution and requires no parameter tuning [18], have also been applied and shown to be efficient alternatives to integer programming problems.
Omran and Engelbrecht [19] investigated the performance of DE in integer programming. They tested three versions of DE and found that the self-adaptive DE (SDE) requiring no parameter tuning is the most efficient and performs better than PSO.
In this paper, we propose three algorithms for integer programming based on a relatively new bioinspired method, namely, biogeography-based optimization (BBO). We modify the mutation operator of the original BBO to enhance its exploration or global search ability and adopt a local neighborhood structure to avoid premature convergence. Experimental results show that our methods are competitive approaches to solving integer programming problems.

Biogeography-Based Optimization
Biogeography is the science of the geographical distribution of biological organisms over space and time. MacArthur and Wilson [20] established the mathematical models of island biogeography, which show that the species richness of an island can be predicted in terms of such factors as habitat area, immigration rate, and extinction rate. Inspired by this, Simon [21] developed the BBO algorithm, where a solution vector is analogous to a habitat, the solution components are analogous to a set of suitability index variables (SIVs), and the solution fitness is analogous to the species richness or habitat suitability index (HSI) of the habitat. Central to the algorithm is the equilibrium theory of island biogeography, which indicates that high HSI habitats have a high species emigration rate and low HSI habitats have a high species immigration rate. For example, in a linear model of species richness (as illustrated in Figure 1), a habitat 's immigration rate and emigration rate are calculated based on its fitness as follows: where max and min are, respectively, the maximum and minimum fitness values among the population and and are, respectively, the maximum possible immigration rate and emigration rate. However, there are other nonlinear mathematical models of biogeography that can be used for calculating the migration rates [22,23].
Migration is used to modify habitats by mixing features within the population. BBO also has a mutation operator for changing SIV within a habitat itself and thus probably increasing diversity of the population. For each habitat , a species count probability computed from and indicates the likelihood that the habitat was expected a priori to exist as a solution for the problem. In this context, very high HSI habitats and very low HSI habitats are both equally improbable, and medium HSI habitats are relatively probable. The mutation rate of habitat is inversely proportional to its probability: where max is a control parameter and max is the maximum habitat probability in the population. Algorithm 1 describes the general framework of BBO for a -dimensional global numerical optimization problem (where and are the lower and upper bounds of the th dimension, respectively, and rand is a function that generates a random value uniformly distributed in [0, 1]).
Algorithm 1: The original BBO algorithm.
Typically, in Line 8 we can use a roulette wheel method for selection, the time complexity of which is ( ). It is not difficult to see that the complexity of each iteration of the algorithm is ( 2 + ( )), where ( ) is the time complexity for computing the fitness function .

Biogeography-Based Heuristics for Integer Programming
In BBO, the migration operator provides good exploitation ability, while the broader exploration of the search space is mainly based on the mutation operator. Simon [21] suggested that max should be set to a small value (about 0.01), which results in low mutation rates. However, when being applied to integer programming, we need to use higher mutation rates to improve the exploration of search space. According to our experimental studies, when max is set to about 0.25∼0.3, the BBO algorithm exhibits the best performance on integer programming problems. Note that the migration operator does not violate the integer constraints, and the rounding of real values to integers is required only after mutations (Line 13 of Algorithm 1). Nevertheless, even using a higher mutation rate, the performance of BBO is far from satisfactory for integer programming. This is mainly because random mutation operator does not utilize any information of the population to guide the exploration of search space. In this work, we introduce two other mutation operators to BBO, which results in three variants of BBO for integer programming.

A Blended Mutation
Operator. In the first variant, namely, BlendBBO, we use a blended mutation operator, which is motivated by the blended crossover operator used by Mühlenbein and Schlierkamp-Voosen [24] in GA and by Ma and Simon [25] in constrained optimization. In our approach, if a component of vector is subject to mutate, we first select another vector with probability ∝ and then use the following equation to work out the new value of the component: where is a random value uniformly distributed in [0, 1]. Note that if the th dimension of the search space has a bound, (4) will never result in a value outside the bound. Moreover, we employ an elitism mechanism in solution update (as used in ES [12,26]): the migration operator always generates a new vector for each existing vector (rather than directly changing ); if is better than , no mutation will be applied and directly enters to the next generation; otherwise the mutation operator is applied to . This not only decreases the required NFE but also increases the convergence speed of the algorithm. The algorithm flow of BBO with the blended mutation is presented in Algorithm 2.

DE Mutation
Operator. The second variant, namely, BBO DE, replaces the random mutation operator with the mutation operator of DE, which mutates a vector component by adding the weighted difference between the corresponding components of two randomly selected vectors to a third one: where 1 , 2 , and 3 are three unique randomly selected habitat indices that are different to , and is a constant scaling coefficient. DE is well known for its good exploration ability, and the combination of BBO migration and DE mutation achieves a good balance between exploitation and exploration. BBO DE also uses our new solution update mechanism described above. Therefore, the algorithm flow of BBO DE simply replaces Lines 15 and 16 of Algorithm 2 with the DE mutation operation described by (5). 4 The Scientific World Journal (1) Randomly initialize a population of solutions (habitats); (2) while stop criterion is not satisfied do (3) for = 1 to do (4) Calculate , , and according to ; (5) for = 1 to do (6) Let = ; (7) for = 1 to do (8) if rand() < then //migration (9) Select a habitat with probability ∝ ; (10) , ← , ; (11) if ( ) < ( ) then ← ; (12) else (13) for = 1 to do (14) if rand() < then //mutation (15) Let = rand() and select a habitat with probability ∝ ; (16) , ← round( , + (1 − ) , ); (17) Evaluate the fitness values of the habitats; (18) Update max , max , and the best known solution; (19) return the best known solution.
Algorithm 2: The BBO with the blended mutation for integer programming.

Local Topology of the Population.
The original BBO uses a fully connected topology; that is, all the individual solutions are directly connected in the population and can migrate with each other. But such a global topology is computationally intensive and is prone to premature convergence. To overcome this problem, our third variant replaces the global topology with a local one. One of the simplest local topologies is the ring topology, where each individual is directly connected to two other individuals [27,28]. But here we employ a more generalized local topology, the random topology, where each individual has immediate neighbors that are randomly selected from the population and is a control parameter [28].
In consequence, whenever an individual vector is to be immigrated, the emigrating vector is chosen from its neighbors rather than the whole population, based on the migration rates. The neighborhood structure can be saved in an × matrix : if two habitats and are directly connected then ( , ) = 1; otherwise ( , ) = 0. It is easy to see that the complexity of each iteration of the algorithm is ( + ( )). Storn and Price [14] have proposed several different strategies on DE mutation. The scheme of (5) is denoted as DE/rand/1. Another scheme is named DE/best/1, which always chooses the best individual of the population as 1 in (5). Omran et al. [29] extended it to the DE/lbest/1 scheme, which uses a ring topology and always chooses the better neighbor of the vector to be mutated.
In our approach, BBO migration and DE mutation share the same local random topology. That is, each individual has neighbors, and at each time an is chosen from the neighbors with probability ∝ to participate in the mutation such that Moreover, if the current best solution has not been improved after every generation (where is a predefined constant), we reset the neighborhood structure randomly.
The third variant is named LBBO LDE, and it also uses the same solution update mechanism as the previous two variants.

Computational Experiments
We test the three variants of BBO on a set of integer programming benchmark problems, which are taken from [16,30,31] and frequently encountered in the relevant literature. The details of the benchmark problems are described in the Appendix. For comparison, we also implement the basic BBO, DE, and SDE [19] for integer programming. The branch-and-bound method is not included for comparison, because it has shown that DE outperforms branch-andbound on most test problems [16,19].
For all the six algorithms, we use the same population size = 50 and run them on each problem for 40 times with different random seeds. The migration control parameters are set as = = 1 for BBO, BlendBBO, BBO DE, and LBBO LDE, and the mutation control parameter max is set to 0.01 for BBO and 0.25 for BlendBBO (BBO DE and LBBO LDE do not use this parameter). Empirically, the neighborhood size and the threshold of nonimprovement generations are both set to 3 for LBBO LDE. The other parameters with regard to DE and SDE are set as suggested in [14,32].
The first two problems 1 and 2 are high-dimensional problems. For 1 , we, respectively, consider it in 10 and 30 dimensions. Table 1 presents the success rates (SR) and required NFE of the algorithms to achieve the optimum in 10 dimensions, and Figure 2 presents the corresponding The Scientific World Journal 5  Figure 3, respectively, present the results and the convergence curves of the algorithms on 1 in 30 dimensions. On this high-dimensional problem, BBO, BlendBBO, and DE all fail to obtain the optimum, SDE and BBO DE, respectively, have SR of 85% and 90% for obtaining the optimum, and only our LBBO LDE can always guarantee the optimum.
From the convergence curves we can also find that the BBO algorithm converges very fast at the early stage, but thereafter its performance deteriorates because it is ineffective to explore other potentially promising areas of the search space. By combining with the DE mutation operator, our hybrid BBO methods inherit the fast convergence speed of BBO, at the same time taking advantage of the exploration ability of DE.
For 2 , we, respectively, consider it in 5 and 15 dimensions, the experimental results of which are, respectively, presented in Tables 3 and 4 and the convergence curves of which are presented in Figures 4 and 5. The results are similar to those of 1 : for the low dimensional problem, SDE, BBO DE, and LBBO LDE are efficient; for the high-dimensional problem, only LBBO LDE can guarantee the optimum; the performance LBBO LDE is the best while that of BBO is the worst; SDE performs better than DE and BBO DE performs slightly better than SDE, and BlendBBO outperforms BBO but is worse than the algorithms with the DE mutation operator.    NFE of DE is slightly better than SDE and LBBO LDE, but LBBO LDE converges faster than DE, as shown in Figure 6.
4 is a relatively easy problem, on which even the worst BBO has an SR of 75%, and all the other algorithms can guarantee the optimum. LBBO LDE is the best one in terms of both NFE and convergence speed, as shown in Table 6 and Figure 7. The remaining three test problems are also relatively easy. The experimental results are presented in Tables 7, 8, and 9, and the convergence curves are shown in Figures 8, 9, and 10, respectively. As we can clearly see, the four algorithms with the DE mutation operator can always obtain the optima on these problems, and LBBO LDE shows the best performance.
In summary, our LBBO LDE outperforms the other algorithms on all of the test problems. Generally speaking, the original BBO converges fast at first, but it is easy to be trapped by the local optima. BlendBBO alleviates the dilemma to a certain degree, but the DE mutation operator is more effective than the blended mutation operator, as       on high-dimensional problems where the performance of DE and SDE deteriorates quickly. Therefore, LBBO LDE is a very competitive heuristic method for solving integer programming problem.

8
The Scientific World Journal

Conclusion
In this paper we develop three algorithms for integer programming based on the BBO heuristic. The BlendBBO uses a blended mutation operator, BBO DE integrates the DE mutation operator, and LBBO LDE further uses a local neighborhood structure for selecting individuals for migration and mutation. Experimental results show that LBBO LDE has the best performance on a set of benchmark integer programming problem. In general, the LBBO LDE algorithm with local neighborhood size of 3∼5 is efficient on the test problem, but none of the values can provide the best performance on all the problems. Currently we are studying a mechanism that dynamically adjusts the neighborhood size as well as other control parameters according to the search state [33]. Moreover, the test problems considered in the paper only have bounds for decision variables but do not include other constraints, and we are extending the proposed approach to solve more complex constrained optimization problems, including multiobjective ones [34][35][36]. We also believe that our approach can be adapted to effectively handle other kinds of combinatorial optimization problems, such as 0-1 integer programming and permutation-based optimization.