An Improved Animal Migration Optimization Algorithm for Clustering Analysis

Animal migration optimization (AMO) is one of the most recently introduced algorithms based on the behavior of animal swarm migration. This paper presents an improved AMO algorithm (IAMO), which significantly improves the original AMO in solving complex optimization problems. Clustering is a popular data analysis and data mining technique and it is used in many fields. The well-known method in solving clustering problems is k-means clustering algorithm; however, it highly depends on the initial solution and is easy to fall into local optimum. To improve the defects of the k-means method, this paper used IAMO for the clustering problem and experiment on synthetic and real life data sets. The simulation results show that the algorithm has a better performance than that of the k-means, PSO, CPSO, ABC, CABC, and AMO algorithm for solving the clustering problem.


Introduction
Data clustering is the process of grouping data into a number of clusters.The goal of data clustering is to make the data in the same cluster share a high degree of similarity while being very dissimilar to data from other clusters.It is a main task of exploratory data mining and a common technique for statistical data analysis used in many fields, including machine learning, pattern recognition, image analysis, information retrieval, and bioinformatics.Cluster analysis was originated in anthropology by Driver and Kroeber in 1932 and introduced to psychology by Zubin in 1938 and Tryon in 1939 and famously used by Cattell beginning in 1943 [1] for trait theory classification in personality psychology.Many clustering methods have been proposed; they are divided into two main categories: hierarchical and partitional.The means clustering method [2] is one of the most commonly used partitional methods.However, the results of -means solving the clustering problem highly depend on the initial solution and it is easy to fall into local optimal solutions.Zhang et al. have proposed an improved -means clustering algorithm called -harmonic means [3].But the accuracy of the results obtained by the method is not high enough.
In recent years, many studies have been inspired by animal behavior phenomena for developing optimization techniques, such as firefly algorithm (FA) [4], cuckoo search (CS) [5], bat algorithm (BA) [6], artificial bee colony (ABC) [7], and particle swarm optimization (PSO) [8].Because of its advantages of global, parallel efficiency, robustness, and universality, these bioinspired algorithms have been widely used in constrained optimization and engineering optimization [9,10], scientific computing, automatic control, and clustering problem [11][12][13][14][15][16][17][18][19][20][21].Niknam et al. have proposed an efficient hybrid evolutionary algorithm based on combining ACO and SA for clustering problem [15,16] in 2008.In 1991, Colorni et al. have presented ant colony optimization (ACO) algorithm based on the behavior of ants seeking a path between their colony and a source of food [22].Then Shelokar et al. and Kao and Cheng have solved the clustering problem using the ACO algorithm [17,18] in 2004 and 2006.Eberhart and Kennedy have proposed particle swarm optimizer (PSO) algorithm which simulates the movement of organisms in a bird flock or fish school [8] in 1995 and the algorithm also has been adopted to solve this problem by Omran et al. and van der Merwe and Engelbrecht [19,23] in 2005 and 2003.Kao et al. have presented a hybrid approach according to combination of the -means algorithm, Nelder-Mead simplex search, and PSO for clustering analysis [14] in 2008.Niknam et al. have presented a hybrid evolutionary algorithm based on PSO and SA (simulated annealing algorithm, 1989 [24]) to solve the clustering problem [13] in 2009.Zou et al. have proposed a cooperative artificial bee colony algorithm to solve the clustering problem and experiment on synthetic and real life data sets to evaluate the performance [11] in 2010.Niknam and Amiri have proposed an efficient hybrid approach based on PSO, ACO, and -means called PSO-ACO-K approach for clustering analysis [12] in 2010.The artificial bee colony (ABC) algorithm is described by Karaboga [25] in 2005 and it has been adopted to solve clustering problem by Karaboga and Ozturk [20] in 2011.Voges and Pope have used an evolutionary-based rough clustering algorithm for the clustering problem [21] in 2012.Chen et al. have used monkey search algorithm for clustering analysis [26] in 2014.
Animal migration algorithm (AMO) is a new bioinspired intelligent optimization algorithm by simulating animal migration behavior proposed by Li et al. [27] in 2013.AMO simulates the widespread migration phenomenon in the animal kingdom, through the change of position, replacement of individual, and finding the optimal solution gradually.AMO has obtained good experimental results on many optimization problems.This paper presents an algorithm to improve the performance of AMO.We proposed a new migration method to modify the performance of AMO, the migration process based on shrinking animals living area operator; this method guarantees AMO rapid convergence to global optimum.By means of selecting the better solution space around the current solution, it improves search ability and accelerates convergence velocity, and it has more chance to find the global optima.
The structure of the paper is as follows.In Section 2, the traditional method -means for clustering is presented.In Section 3, the original AMO algorithm is introduced.Section 4 describes our proposed novel approach of migration process.Section 5 elaborates the improved AMO and some biological foundations of animal behaviors are explained.Section 6 illustrates experiments and discusses the results.Section 7 studies the extent of different size of shrinkage coefficient impact of the proposed algorithm.At the end of the paper, we conclude it with future directions and developments with the improved AMO.
The -means clustering algorithm is as follows.
The main idea is to define  centroids, one for each cluster.These centroids should be placed in a cunning way because of different location causing different result.So the better choice is to place them as much as possible far away from each other.In this study, we will use Euclidian metric as a distance metric.The expression is given as follows: Finally, this algorithm aims at minimizing an objective function, in this case a squared error function.The objective function is as follows:

Animal Migration Optimization (AMO)
Animal migration algorithm can be divided into animal migration process and animal updating process.In the migration process, the algorithm simulates how the groups of animals move from current position to a new position.During the population updating process, the algorithm simulates how animals are updated by the probabilistic method.

Animal Migration Process.
During the animal migration process, an animal should obey three rules: (1) avoid collisions with your neighbors; (2) move in the same direction as your neighbors; and (3) remain close to your neighbors.In order to define concept of the local neighborhood of an individual, we use a topological ring, as has been illustrated in Figure 1.For the sake of simplicity, we set the length of the neighborhood to be five for each dimension of the individual.Note that, in our algorithm, the neighborhood topology is static and is defined on the set of indices of vectors.
If the index of animal is , then its neighborhood consists of animal having indices  − 2,  − 1, ,  + 1,  + 2, if the index of animal is 1, the neighborhood consists of animal having indices  − 1, , 1, 2, 3, and so forth.Once the neighborhood topology has been constructed, we select one neighbor randomly and update the position of the individual according to this neighbor, as can be seen in the following formula: where  neiborhood, is the current position of the neighborhood,  is produced by using a random number generator controlled by a Gaussian distribution,  , is the current position of th individual, and  ,+1 is the new position of th individual.

Population Updating Process.
During the population updating process, the algorithm simulates how some animals leave the group and some join in the new population.Individuals will be replaced by some new animals with a probability .The probability is used according to the quality of the fitness.We sort fitness in descending order, so the probability of the individual with best fitness is 1/ and the individual with worst fitness, by contrast, is 1, and the process can be shown in Algorithm 1.
After producing the new solution  ,+1 , it will be evaluated and compared with the  , , and we choose the individual with a better objective fitness: ) is better than  ( ,+1 ) ,  ,+1 otherwise. (5)

The New Migration Process Method
In AMO, algorithm uses migration process and population updating process to find a satisfactory solution.The proposed algorithm used a new migration process by establishing a living area by the leader animal (the individuals with best fitness value) and animals migrate from current locations into this new living area to simulate animal migration process.At first, there are  animals that live in living area, as shown in Figure 2(a), moving, eating, drinking, reproducing, and so on; some individuals move randomly and their position be updated, and then we calculate the best position of animals by fitness function and record it.But the amount of food or water gradually diminished as the time wore on, as shown in Figure 2(b), and some animals migrate from the current areas which have no food and water to a new area with abundant food and water, as shown in Figure 2(c).In Figure 2, the green parts represent the living areas with abundant food and water; animals can live in these areas.And the yellow parts represent the areas that lack food or water; animals can no longer live in these areas, and they must migrate to a new living area (the green parts in Figure 2(c)).We shrink the living area after a period of time (as shown in Figures 2(a) and 2(c)), and then animals migrate to the new living area ceaselessly.As a rule of thumb, the globally optimal solution always nearby is the current best solution; in IAMO, the animals living area is smaller and smaller (by formula ( 6)) after each iteration, and the individuals get closer and closer to the globally optimal solution, so we can accelerate the convergence velocity and precision of the algorithm to some extent.
The boundary of the living area is established by where  best is the leader animal (the current best solution), low and up are the lower and upper bound of the living area,  is living area radius,  is shrinkage coefficient,  ∈ (0, 1), and low, up, and  are all 1 ×  row vector.In general, the original value of  depends on the size of the search space.
As iterations go on, a big value of  improves the exploration ability of the algorithm and a small value of  improves the exploitation ability of the algorithm.Animals are randomly and uniformly distributed between the prespecified lower initial parameter bound  and the upper initial parameter bound .So the th component of the th vector is as follows:

The IAMO Algorithm for Solving Clustering Problem
where rand , [0, 1] is a uniform distribution random number between 0 and 1.

Animals Migration.
During the migration process, because of animals hunting, foraging, or drinking in the living area, some parts of the living area are lacking food or water or climate condition change, and some animals migrate from the current living area to the new living area which has abundant food and water or climate condition suitable for living.We assume that there is only one living area, and animals out of the new living area would be migrating into the new living area, as depicted in Section 4. We calculate the If  ,+1 is better than   then (18)   =  ,+1 (19) End If (20) End for (21) According to the fitness function, we obtain the best individual  best , and the new living area can be established by  best and .

Individuals in Population
Updating.During the population updating process, algorithm simulates some animals that are preyed by their enemies or some animals leave the group and some join in the group from other groups or some new animals are born.In IAMO, we assume that the number of available animals is fixed, and every animal will be replaced by , as shown in Section 3.2.
Specific implementation steps of the improved animals migration optimization algorithm (IAMO) can be shown as in Algorithm 2.

Numerical Simulation Experiments
All of the algorithm was programmed in MATLAB R2008a, numerical experiment was set up on AMD Athlon(tm)II * 4 640 processor and 2 GB memory.
Iris Data ( = 150,  = 4, and  = 3).This data set with 150 random samples of flowers from the Iris species setosa, versicolor, and virginica collected by Anderson [34].From each species, there are 50 observations for sepal length, sepal width, petal length, and petal width in cm.This data set was used by Fisher [35] in his initiation of the linear-discriminantfunction technique [11,12,33].
Teaching Assistant Evaluation ( = 151,  = 5, and  = 3).The data consist of evaluations of teaching performance over three regular semesters and two summer semesters of 151 teaching assistant (TA) assignments at the Statistics Department of the University of Wisconsin-Madison.The scores were divided into 3 roughly equal-sized categories ("low, " "medium, " and "high") to form the class variable [33].
Wine Data ( = 178,  = 13, and  = 3).This is the wine data set, which is also taken from MCI laboratory.These data are the results of a chemical analysis of wines grown in the same region in Italy but derived from three different cultivars.The analysis determined the quantities of 13 constituents found in each of the three types of wines.There are 178 instances with 13 numeric attributes in wine data set.All attributes are continuous.There is no missing attribute value [11,12,33].
StatLog (Heart) Data ( = 270,  = 13, and  = 2).This data set is a heart disease database similar to a database already present in the repository (heart disease databases) but in a slightly different form [33].
Hagerman's Survival ( = 306,  = 3, and  = 2).The data set contains cases from a study that was conducted between 1958 and 1970 at the University of Chicago's Billings Hospital on the survival of patients who had undergone surgery for breast cancer.It records two survival status patients with the age of patient at time of operation, patient's year of operation, and number of positive axillary nodes detected [33].
Balance Scale Data ( = 625,  = 4, and  = 3).This data set was generated to model psychological experimental results.Each example is classified as having the balance scale tip to the right, to the left, or balanced.The attributes are the left weight, the left distance, the right weight, and the right distance.The correct way to find the class is the greater of (left-distance * left-weight) and (right-distance * right-weight).If they are equal, it is balanced [33].
Here we set the parameters of AMO and IAMO as follows.The population size of the AMO and IAMO is 100.In IAMO, the original living area radius  = 0.3( − ) and shrinkage coefficient  = 0.92.For the PSO, inertia weight  = 0.729, acceleration coefficients 1 = 2, 2 = 2, and population size  = 100.The population size of the CPSO is 20.The population size of the ABC and CABC are 50 and 10, respectively.In order to compare with other algorithms, the maximum generations of all algorithms are 100.
For every data set, each algorithm is applied 20 times individually with random initial solution.For the Art1 and Art2 data set, once the randomly generated parameters are determined, the same parameters are used to test the performance of three algorithms.We ranked each algorithm according to the mean result.The results are kept four digits after the decimal point.The mean value, the best value, the worst value, the standard deviation, and the rank value are recorded in Tables 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10.Tables 1-10 show that IAMO is very precise than other algorithms in solving the ten data sets.As seen from the results, the IAMO algorithm provides the best value and small standard deviation in comparison with other methods.For the Art1 and Art2 data set in Tables 1 and 2 which were randomly generated, IAMO obtained the best mean and smallest standard deviation compared to other algorithms.The mean    As seen from Table 1 to Table 10, we can conclude that although the convergence rate is not quick enough at the beginning of the iteration compared to ABC and CABC, the final results are the best compared to other algorithms in all test data sets.The most results of ABC and CABC are better than PSO and CPSO, and the -means algorithm is the worst for most of test data sets.

Living Area Radius Evaluation
The performance and results of the proposed algorithms are greatly affected by the size of living area.At the beginning of the iteration, a big value of  improves the exploration ability of the algorithm and; at the end of iteration, a small value of  improves the exploitation ability of the algorithm.We adopted a fixed shrinking coefficient  = 0.92 to change the living area radius after each iteration, as shown in formula (6).
To study the extent of  impacts on the proposed algorithm, we selected Art1 data set and Iris data set, using different  to evaluate the performance of the proposed algorithm.Figure 17 shows the results of an experiment on Art1; we can conclude that if we choose  between 0.6 and 0.9, it has a better convergence precision than that of  = 0.99 or  = 0.40.If we choose  = 0.40, IAMO algorithm plunges into local optima, and if we choose  = 0.99, the IAMO algorithm has a very low convergence rate.And likewise in Figure 18, for Iris test data set, IAMO algorithm quickly converged at global optimum before 30 iterations if we choose  = 0.80, while IAMO could not escape from poor local optima and to global optimum if we choose  = 0.70,  = 0.60, or  = 0.40.So the best  for solving Iris data set must exist between 0.7 and 0.99.
The results suggest that a proper  can greatly improve the algorithm convergence velocity and convergence precision, and an improper  may lead the IAMO fall into local optimum.

Conclusions
In this paper, to improve the deficiencies of the AMO algorithm, we improved the algorithm by using a new migration method based on shrinking animals living area.By 10 typical standard test data sets simulation, the results show that IAMO algorithm generally has strong global searching ability and local optimization ability and can effectively avoid the deficiencies that conventional algorithms easily fall into local optimum.IAMO has improved the convergence precision of AMO and rank 1st in all test data sets; therefore, it is very practical and effective to solve clustering problems.At last, how to define a proper and unified radius of living area needs to be considered in subsequent work.

Figure 1 :
Figure 1: The concept of the neighborhood of an animal.

Figure 5 :Figure 6 :
Figure 5: The convergence curve of the Art1 data.

Figures 5 ,
6,7,8,9,10,11,12,13, and 14 show the convergence curves of different data sets for various algorithms.Figures15 and 16show the original data distribution of Iris data set and the clustering result by IAMO algorithm.

Figure 7 :Figure 8 :
Figure 7: The convergence curve of the Iris data.

Figure 9 :Figure 10 :
Figure 9: The convergence curve of the wine data.

Figure 11 :Figure 12 :
Figure 11: The convergence curve of the heart data.

Figure 13 :Figure 14 :
Figure 13: The convergence curve of the balance scale data.

DifferentFigure 18 :
Figure 18: The convergence curve of the Iris with different .
) Assign each object   to the group that has the closest centroid.The principle of division is as follows: if (  −   ) < (  −   ),  = 1, 2, . . .,  and  ̸ = .The data   will be divided into classified collection   .

Table 1 :
Results obtained by the algorithms for 20 different runs on Art1 data.

Table 2 :
Results obtained by the algorithms for 20 different runs on Art2 data.

Table 3 :
Results obtained by the algorithms for 20 different runs onIris data.

Table 4 :
Results obtained by the algorithms for 20 different runs on TAE data.

Table 5 :
Results obtained by the algorithms for 20 different runs on wine data.
CABC also sought the best solution 96.6555, but the standard deviation is bigger than IAMO when the best solutions of AMO, PSO, CPSO, ABC, and -means are 97.0751,96.6567,96.6580,96.6566, and 99.4582, respectively.Table4shows the results of algorithms on the TAE data set.The mean value

Table 6 :
Results obtained by the algorithms for 20 different runs on seeds data.

Table 7 :
Results obtained by the algorithms for 20 different runs on StatLog (heart) data.

Table 8 :
Results obtained by the algorithms for 20 different runs on Hagerman's survival data.ABC, CABC, and -means within 20 runs.For wine data set, IAMO reached the mean value 16292.1855while CABC reached the mean value 16292.1982.The best value and worst value of IAMO are 16292.1849and 16292.1862,which are also better than 16292.1858and 16292.2094obtained by CABC, and the standard deviation value of IAMO is also the smallest one.Table 6 provides the results of algorithms on the

Table 9 :
Results obtained by the algorithms for 20 different runs on balance scale data.

Table 10 :
Results obtained by the algorithms for 20 different runs on cancer data.

Table 7 ,
IAMO gets the best value is 10622.9824and the same as CABC, while the mean values of the two algorithms are 10622.9824and10622.9904,so the IAMO is better than CABC algorithm.For Hagerman's survival data set, the optimum value 2566.9888canbeobtained by IAMO, ABC, and CABC, but the standard deviations of ABC and CABC are 2.3919−04 and 2.8709−03 which is worse than that of 2.3022 − 06 obtained by IAMO.The standard deviation of PSO is a little bigger than that of CPSO.For balance scale data set in Table9, as seen from the results, the mean, best, and worst ones are all 1423.8204,whichreflect the stable characteristics of IAMO.The three best algorithms in this test data are IAMO, CABC, and ABC, and the best results of them are 1423.8204,1423.8206, and 1423.8308.For Wisconsin breast cancer data set in Table10, the mean value, the best value, and the worst value are all 2964.3870which are obviously superior to -means, PSO, CPSO, ABC, and AMO.