Annealing Ant Colony Optimization with Mutation Operator for Solving TSP

Ant Colony Optimization (ACO) has been successfully applied to solve a wide range of combinatorial optimization problems such as minimum spanning tree, traveling salesman problem, and quadratic assignment problem. Basic ACO has drawbacks of trapping into local minimum and low convergence rate. Simulated annealing (SA) and mutation operator have the jumping ability and global convergence; and local search has the ability to speed up the convergence. Therefore, this paper proposed a hybrid ACO algorithm integrating the advantages of ACO, SA, mutation operator, and local search procedure to solve the traveling salesman problem. The core of algorithm is based on the ACO. SA and mutation operator were used to increase the ants population diversity from time to time and the local search was used to exploit the current search area efficiently. The comparative experiments, using 24 TSP instances from TSPLIB, show that the proposed algorithm outperformed some well-known algorithms in the literature in terms of solution quality.


Introduction
One of the most popular combinatorial optimization problems is the traveling salesman problem (TSP) [1]. Given a set of cities, a salesman attempts to find the shortest or at least near to the shortest tour by visiting each city only once and turning back to the starting city. TSP is a representative of variety of combinatorial problems. It has been studied for the last 40 years. It has many real world applications such as the movement of people, postal delivery, school bus routes, garbage collection, design of hardware devices and radio electronic systems, machine scheduling, integrated circuits, and computer networks [2][3][4].
Metaheuristic algorithms are formally defined as algorithms that inspired by nature and biological behaviors. They produce high-quality solutions by applying a robust iterative generation process for exploring and exploiting the search space efficiently and effectively. Recently, metaheuristic algorithms seem to be a hot and promising research areas [5]. They can be applied to find near-optimal solutions in a reasonable time for different combinatorial optimization problems [6].
In applying standalone metaheuristic algorithms, there is possibility of losing the diversity of the population through premature convergence and thus the algorithm gets stuck in local optima. Therefore, maintaining the diversity and making tradeoff between diversification and intensification by combining two or more algorithms to produce high-quality solutions and speed up the execution time is indispensable [14].
For hybrid ACO, the earliest study was conducted by McKendall and Shang [15]. They presented a hybrid ant system algorithm to solve dynamic facility layout problem.

Computational Intelligence and Neuroscience
Another research was a hybrid ant system algorithm for solving TSP in which ant colony, genetic algorithm, and simulated annealing are hybridized [16]. For the hybrid ant colony system (ACS), many researches were conducted including the work by Huang and Liao [17], Yoshikawa and Otani [18], Xing et al. [19], Liao et al. [20], Lin et al. [21], Hajipour et al. [22], and Min et al. [16]. The research by Katagiri et al. [23] is an example for hybrid MAX-MIN Ant System. To solve TSP problems, several hybrid ACO variants with other metaheuristic algorithms such as SA, PSO, ACO, ABC, and ANN were proposed. Bontoux and Feillet [24] proposed a hybrid ACO algorithm with local search procedures to solve TSP. Tsai et al. [25] presented a hybrid ACO called ACOMAC algorithm for solving TSP. Beam-ACO algorithm is a hybrid ACO with beam search for solving TSP [26]. Chen and Chien presented a hybrid algorithm, called the genetic simulated annealing ant colony system with particle swarm optimization techniques, for solving TSP [27]. Junqiang and Aijia proposed a hybrid ant colony algorithm (HACO), which combined ACO with delete-cross to overcome the shortcoming of slow convergence speed of ACO [28]. Dong et al. [29] proposed an algorithm, called cooperative genetic ant system (CGAS) for solving TSP, which hybridized both GA and ACO to improve the performance of ACO. Recently, Gündüz et al. [30] presented a hybrid ACO with ABC for solving TSP. In addition, Mahi et al. [31] proposed a new algorithm in which ACO was hybridized with PSO and 3-Opt for solving small TSP instances. The PSO was used to determine the optimum values of the two main parameters of ACO which affected algorithm performance and the 3-Opt was used to escape from the local optima found by ACO algorithm. Furthermore, Yousefikhoshbakht et al. [32] proposed REACSGA for solving small TSP instances which employed the modified ACS for generating initial diversified solutions and GA for intensification mechanisms.
As noted above, previous studies show that ACO still has drawbacks. The performance of these studies was dramatically decreased when dealing with large-scale instances. To the best of my knowledge, no research has been done to hybridize elitist ant system with SA, mutation, and local search. Therefore, in this research a new hybrid elitist ant system with SA, mutation operator, and local search procedure is introduced for solving TSP. Introducing SA can help ACO to escape from the local optima. On the other hand, determining initial solution of SA is almost difficult. Therefore, the use of the ACO is useful in the generation of SA initial solution. While introducing the mutation operation to ACO algorithm will enhance the algorithm performance, expand the diversity of population, and inhibit the premature convergence. Applying either SA or mutation is based on the diversity level of the population. After applying SA or mutation, elitist ant system goes through a local search procedure to speed up the convergence.
The rest of the paper is structured as follows. Section 2 presents the TSP formulation. Section 3 describes the hybrid algorithm. The experimental results are presented in Section 4. Conclusions and future work are given in Section 5.

Traveling Salesman Problem
TSP is an active field of research in computer science. It demonstrates all the aspects of combinatorial optimization and comes under the set of NP-hard problems which cannot be solved optimally in a polynomial time [33]. Solving TSPs is an important part of applications in many practical problems within daily live [2][3][4].
TSP is represented as a connected graph , consisting of a set of vertices , an edges set , and a set of distances associated with each edge in and stored in a distance matrix . The value is the cost of traversing from vertex ∈ to vertex ∈ and the diagonal elements are zeros. Given this information, a tour in TSP is formulated as a cyclic permutation, called Hamiltonian cycle of visiting each vertex in the graph exactly once, of {1, 2, . . . , }, where ( ) is the city, on the tour, following city ( ). The aim in solving TSP is to find a permutation that minimizes the length of the tour as shown in It is worth mentioning that the total number of possible distinct feasible routes covering all cities is ( − 1)!/2. This produced a problem which is very hard to solve (NP-hard problem).

Algorithm Design
An overview of the ACO, SA, mutation operator, and the proposed algorithm is presented in the following subsections.

Ant Colony Optimization.
ACO is a population-based metaheuristic algorithm which was inspired by the foraging behavior of the real ants when searching for the shortest path from the food source to their nest. Analogically, the artificial ants search for good solutions iteratively in several generation. In each generation, every ant constructs its feasible solution path step by step guided by a transition rule that is a function of artificial pheromone and distance between two cities (heuristic information) [34] as shown in (2). After that, the ant lays down an amount of pheromone trail on the edges of its completed tour. In the next generation, ants are attracted by the pheromone trail. Therefore, this will guide the search in the search space towards good quality solutions. TSP is identical to the foraging behaviors of real ants in nature. Therefore, applying ant colony optimization to solve TSP will be very simple. Equation (2) is used to calculate the probability of selecting city by ant to be visited after city .
where denotes the amount of pheromone between city and city , indicates the distance between city and city (priori available heuristic information), is the parameter that represents the relative importance of the pheromone (dynamic evaluation) versus the heuristic value (static Computational Intelligence and Neuroscience 3 evaluation), and is a set of cities which ant has not yet visited. Therefore, the selection probability is proportional to the product of the static and dynamic evaluation.
In the dynamic evaluation, two pheromone update rules are used to calculate the amount of pheromone on each edge between cities. The first rule is called the local update rule as shown in where 0 < ≤ 1 is the pheromone trail evaporation rate in local update rule and is the number of ants. Thus, the local update rule is decreasing the pheromone trails by a constant factor (pheromone evaporation). The second rule is the global update rule which adds extra amount pheromone trail to the best route in the population. It is worth mentioning that the best route is the shortest route as in elitist strategy [10], the extended version of original ant system algorithm. Equation (4) shows the definition of the global update rule in elitist ant system: where gb is the best route, gb ( ) is the distance of the best route, and is a positive integer. This means that the edges belonging to the global-best tour get an additional amount of pheromone each time the pheromone is updated. The pseudocode of the basic ACO is illustrated in Algorithm 1.

Simulated
Annealing. SA is a trajectory-based optimization technique. It is basically an iterative improvement strategy with a criterion that accepts higher cost configurations sometimes. The first attempt to apply SA for solving the combinatorial optimization problems was in the 80s of the last century [35,36]. An overview of simulated annealing, its theoretical development, and application domains is shown in [9]. Simulated annealing was inspired by physical annealing process of solids in which a solid is first heated and then cooled down slowly to reach a lower state of energy. Metropolis acceptance criterion [37], which models how thermodynamic systems moves from one state to another state, is used to determine whether the current solution is accepted or rejected.
The original Metropolis acceptance criterion was that the initial state of a thermodynamic system was chosen at energy and temperature . Holding constant, the initial configuration of the system is perturbed to produce new configuration and the change in energy Δ is calculated. The new configuration is accepted unconditionally if Δ is negative whereas it is accepted if Δ is positive with a probability given by the Boltzmann factor shown in (5) to avoid trapping in the local optima: This processes is then repeated until reaching a good sampling statistics for the current temperature, and then the temperature is decreased and the process is repeated until a frozen state (free energy state) is reached at = 0. The analogy between the states of a physical system and optimization problems is given as follows: (i) the current configuration of the thermodynamic system is similar to the current solution of the optimization problem; (ii) the thermodynamic system energy is similar to the objective function of optimization problem; and (iii) ground status of the thermodynamic system is similar to the global minimum of the optimization problem. Algorithm 2 shows the general structure of SA.

Mutation Operator.
Mutation operator is inspired from the evolutionary algorithms in which each ant in the population will be given the chance to be altered based on a predefined probability. This operator may help the ant colony algorithm to explore new areas in the search space. It can be applied by randomly exchanging the position of two cities in the tour which leads to generate a new solution that is not very far from the original one. Algorithm 3 shows the main steps of the mutation operation.

The Proposed Algorithm.
In this section, a new algorithm called annealing elitist ant system with mutation operator and local search for solving the traveling salesman problem is introduced. Given cities in TSP instance, first, the elitist ant system will generate the initial population with ants and each ant will randomly choose a city as its starting city. The pheromone intensity level between any two cities is initialized with small positive constant 0 . The iteration counter, which will record the number of iterations the proposed algorithm executes so far, is initially set to zero. For every interval during the execution of the algorithm, where is a predefined number, the elitist ant system will call either simulated annealing or mutation operation, based on the diversity of the elitist ant system, to enhance the algorithm performance. If the diversity is greater than 0.5, the algorithm needs intensification which can be achieved by applying simulated annealing for ratio of the solutions pool. On the contrary, if the diversity is less than 0.5, it means that the algorithm will lose the diversity and may get stuck in the local minima. Therefore, the ant algorithm needs to increase the diversity by applying the mutation operator with a predefined probability. The diversity between the fitness of the ants in the population was measured by calculating the Euclidean Distance (ED) as shown in where is the average distance between the fitness of the best ant and the fitness of the remaining ants in the population.
Step 1. After the initial stage mentioned above, each ant in the population builds its tour by applying the transition rule followed by the local pheromone update rule: (1) Transition Rule. The th ant decides the next city to be visited according to (2).
(2) Local Pheromone Update Rule. After all ants complete their tours, the local update rule of the pheromone trails is applied for each route according to (3).
Step 2. Calculate the route cost of each ant. After that, apply the global pheromone update rule in which the amount of pheromone is added to the best route which has the lowest cost. This rule is defined in (4).
Step 3. Calculate the diversity of the population. If the diversity is high, the algorithm needs intensification by applying SA. Otherwise, the algorithm needs to regain the diversity by applying the mutation operator.
Simulated Annealing. If the diversity value is greater than 0.5, then the elitist ant system will use SA technique to enhance Computational Intelligence and Neuroscience

5
(1) begin (2) Step 1: For ant , generating mutation position random as 1 ; (3) Step 2:; Step 3: Exchanging 1 and 2 ; (10) Step 4: Getting new ant ; (11) end Algorithm 3: The pseudocode for applying the mutation operator for TSP problem. the selected ant by encoding its tour reached by the ant system into SA. Here, the tour of the selected ant is encoded as the initial state of SA. Initialize the system temperature to 0 . In SA, two bits in the ant are randomly selected for exchange to generate a new mutate ant and its energy (cost) is evaluated accordingly. If the energy of ant is better than that of ant or a random number generated between 0 and 1 is less than the Boltzmann factor as defined by (5), then the new ant is accepted. Otherwise, the ant remains unchanged. Reduce the temperature by annealing schedule or by factor 0.0 < < 1.0. Iteratively, this process will be repeated until 0 reaches to a predefined low temperature. If tour path of ant is better than that of , then the enhanced ant is included in the ants population. Otherwise, the ant remains unchanged and is placed back into the ants population.
Mutation Operator. To maintain the diversity of the proposed algorithm, the mutation operator is introduced for further exploration of new areas of the search space. If the diversity value is less than 0.5, then the elitist ant system will use the mutation operation to enhance the selected ant. In this process, an ant is selected randomly with a predefined probability. If a random number between 0 and 1 is less than the predefined mutation rate, then the algorithm selects an ant to mutate. Two bits in the ant are selected randomly for exchange to generate a new mutate ant and calculate its tour.
Step 4. Apply the local search procedure for further enhancement.
Step 5. Again, apply the global pheromone update rule according to (4).
Step 6. If the termination condition is satisfied, then return the best route with its length. Otherwise, go to Step 1. Figure 1 shows the flowchart of the proposed algorithm.

Experimental Results and Discussion
The experimental results to examine the validity and the performance of the proposed algorithm were introduced in this section. The experiments were conducted using 24 TSP standard benchmark problems, with different length, from TSPLIB [39,40]. The proposed algorithm has been implemented in Java on an Intel Core-i7 PC. The Object Oriented Paradigm (OOP) and different data structures have been used to optimize its code. All experiments were conducted on the symmetric TSP. As in other metaheuristic algorithms, the quality of the solutions created by the proposed algorithm was affected largely by the different values of the parameters. Thus, a number of different alternative values were examined to tune the parameters of the proposed algorithm. Table 1 provides the parameters which show variations in the range of values while default values of other parameters were taken. Finally, the selected parameter values are those that achieved the best computational results with respect to the quality of the solution. All of the parameter values have been determined by the experiments on Eil51, lin318, and fl1400 TSP instances which represent small, medium, and large instances, respectively. In these experiments, the proposed   Table 1. Afterward, the algorithm was initialized with a population of 25 ants using = 1 and = 5 which control the influence of pheromone trail and heuristic information (edge cost) in selection of the next city by transition rule. The parameter 0 was set to 0.05 which specifies the intensification/diversification rate. The initial value of the pheromone trail was set to be 0 = 0.5 and the maximum number of iteration was 1000 iterations.
All of the instances included in TSPLIB have already been examined in the literature and their optimality results can be used to compare algorithms according to the best and the average values. Different instances with different size were selected. These instances can be classified into three groups based on their lengths. The first group is the smallest group which includes 8 instances varying in length between 51 and 100 cities. The second is the medium group which includes 10 instances varying in length between 101 and 318 cities. The third is the large-scale group which includes 5 instances with length between 575 and 1655 cities. Therefore, the results were collected after conducting the experiments 10 times for each instance and took the best results, the average results, and the standard deviation for comparison with previous work. Two different experiments were conducted for the evaluation. Both of them were configured according to the parameters setting as shown in Table 1.
In the first experiment, the proposed algorithm was compared with the basic elitist ant system algorithm (EAS). Six evaluation measures were used to evaluate both algorithms. These measures are best solution, worst solution, average solution, standard deviation, number of iterations, and running time of algorithms. Table 2 shows the superiority of the proposed algorithm over EAS in computational results for all evaluation measures. As can be seen from the tabulated values, the quality of the solutions obtained by the proposed algorithm was significantly better than the solutions obtained by EAS. This superiority of the proposed algorithm may be attributed to the introduction of SA which exploited the detected promising solutions to speed up the learning capability of the algorithm. Meanwhile, the addition of the mutation operator enhanced global search capability of the algorithm, prevented it from being trapped in local optima, and thus improved its performance. Moreover, the introduction of the local search strategy increased the speed of algorithm convergence. Figure 2 shows the behavior of the proposed algorithm with three TSP instances. They were chosen to represent short, medium, and large instances, and as such, the findings can be generalized to the other instances. Figure 2(a) shows the results from a single run for korB200 instance. In this instance, the proposed algorithm reached the optimal solution that is 29437 in all ten runs. This figure depicts best, average, and worst solution obtained during the run. Strong optimization capability of proposed algorithm could be inferred. Diversified solutions, high convergence speed, and stagnation avoidance can be observed from the convergence behavior of the algorithm. Specifically, the figure shows how quickly the optimal solution, that is 29437, was found after 26 iterations. The trend goes down fast towards the optimum solution in the early iterations until it approaches the optimum solution. It is clear that no stagnation happened during the search process as observed from the figure. Figure 2(b) illustrates the results from a single run of the proposed algorithm for the lin318 instance. In nine out of ten runs, the proposed algorithm reached the optimal solution that is 42029. This figure presents the graph which shows the convergence behavior of the algorithm. It is clearly noticed that the proposed algorithm has high convergence speed with diversified solutions. The algorithm succeeded in avoiding the potential stagnation and premature convergence as can be observed from the convergence behavior of the algorithm. According to the figure, there is no stagnation happened during the search process. Additionally, the algorithm converged to the best solution after a maximum of 106 iterations. The search space for this instance is medium although the algorithm has no problem in quickly finding the optimum solution. Figure 2(c) demonstrates a single run for rl1323 instance containing 1323 cities. This graph plots the convergence behavior of the proposed algorithm over 1000 iterations. The algorithm reached a solution with cost near to optimal solution at iteration number 293 and never once changed afterwards. As it can be seen from the figure, in the initial stage, the diversity was high due to the variation of the population. However, as fitness function decreased, the diversity also decreased until the suboptimal solution was attained that is 270388. This smooth convergence was due to the good balance between diversification and intensification that proposed algorithm could provide. Although the search space for that instance was large, no stagnation happened during the search process.
In the second experiment, the proposed algorithm was compared with four state-of-the-art metaheuristic algorithms: Chen and Chien [27], Wang et al. [41], Yousefikhoshbakht et al. [32], and Mahi et al. [31]. The authors of these algorithms proved that their algorithms outperformed the other algorithms in the literature. The best found solution, the average one over all runs, the standard deviation, the percentage deviation of the best results, and the percentage     Tables 3 and 4 and Figure 3.
In Tables 3 and 4, column 1 shows the TSP instances, column 2 shows the best known solutions, column 3 shows the algorithms, column 4 shows the best solutions over all runs, column 5 shows the average solution of all runs, column 6 presents the standard deviation, column 7 reveals the percentage deviation of the best results (PD Best) compared to those of the best known solution, and column 8 reveals the percentage deviation of the average of the best solution of all runs (PD Avg) in comparison to the best known solution. PD Best was calculated by (7) and PD Avg was calculated by (8).
PD Avg = avgsolution − bestknownsolution bestknownsolution × 100. (8) In Table 3, the proposed algorithm was compared with Chen and Chien [27] and Wang et al. [41] on 24 benchmark instances with cities from 51 to 1655. As can be seen in Table 3, for the 24 TSP instances, the proposed algorithm was much better than both algorithms on all medium and large instances, such as lin318, rat575, rat783, rl1323, fl1400, and d1655, with respect to the five evaluation measures mentioned above. There was no significant difference between the proposed algorithm, Chen and Chien [27] and Wang et al. [41] on the small instances with cities less than or equal to 100, with respect to best found solution and PD Best.
In Table 4, the proposed algorithm was compared with Yousefikhoshbakht et al. [32] and Mahi et al. [31] on 15 and 8 benchmark instances, respectively, with cities from 51 to 200 as reported in their studies. As can be seen in there was no significant difference between the proposed algorithm and Yousefikhoshbakht et al. [32] on the small instances with cities less than or equal to 100. For the larger instances, the proposed algorithm gained much better results than Yousefikhoshbakht et al. [32]. Comparing with Mahi et al. [31], the proposed algorithm achieved better results in all the 8 instances with respect to best found solution, average solution, PD Best, and PD Avg. Figure 3 shows a comparison of the proposed algorithm to Chen and Chien [27] and Wang et al. [41] based on the percentage deviations of the average solution to the best known solution. It is clear that the proposed algorithm significantly gained smaller percentage deviations than Chen and Chien [27] and Wang et al. [41] in the large-scale TSP instances that is lin318, rat575, rat783, rl1323, fl1400, and d1655.
In summary, numerical results show that the proposed algorithm was effective. It was able to solve small and large size instances better than the existing algorithms. This is because of the proposed algorithm capability of searching the optimal solution until the last iterations without stagnation or premature convergence, especially for the medium and large TSP instances, compared to the other algorithms. In Proposed algorithm Chen and Chien [27] fl1400 Wang et al. [41] rat575 rl1323 d1655 general, the results indicate that the structure of the proposed algorithm, which depends on the concepts of embedding simulated annealing, mutation operation, and local search procedure, achieved the balance between diversification and Table 4: A comparison of the proposed algorithm with Yousefikhoshbakht et al. [32] and Mahi et al. [31] according to the best solution, average solution, standard deviation, the percentage deviation of the average solution (PD Avg), and the percentage deviation of the best solution (PD Best) found by the algorithms.