Rank-Based Ant System via the Relative Position in a Local Hierarchy

ASrank has been proposed as an improved version of the ant colony optimisation (ACO) model. However, ASrank includes behaviours that do not exist in the actual biological system and fall into a local solution. To address this issue, we developedASmulti, a new type of ASrank, in which each agent contributes to pheromone depositions by estimating its rank by interacting with the encountered agents. In this paper, we attempt further improvements in the performance of ASmulti by allowing agents to consider their position in a local hierarchy. Agents in the proposed model (AShierarchy) contribute to pheromone depositions by estimating the consistency between a local hierarchy and global (system) hierarchy.We show that, by using several TSP datasets, the proposed model can find a better solution than ASmulti.


Introduction
Ant colony optimisation (ACO) is a well-known strategy for determining short tours in the travelling salesman problem (TSP) [1]. Many ACO models have been developed by extending the ant system (AS) proposed by Dorigo [2][3][4][5][6]. Rank-based ant system (AS rank ) is one of the representative models that extend the original AS [7]. Using the pheromone procedure update for exploring simulations, this model finds a solution to the TSP. In AS rank , ant agents are ranked in the order of shorter tour lengths at the end of each tour. Only the top-ranking agents are allowed to deposit pheromones. is function improves the convergence of the system by adding pheromones on only the edges toured by top-ranked agents. us, the system converges faster by concentrating pheromones on specific edges. However, because pheromones tend to concentrate solely on specific edges, the system frequently falls into local solutions. Furthermore, while AS rank is inspired by the behaviour of real ants, the viewpoint of a third party is introduced outside the system to classify agents in a single hierarchy, i.e., to place agents in the order of tour lengths. Agents that can complete tours in less time are rewarded with higher positions in the hierarchy.
To eliminate the viewpoint of the third party from the system, we recently proposed a new type of AS rank named AS multi in which an individual agent constructs a subjective hierarchy based on the interaction with encounters [8]. In AS multi , agents decide whether to deposit pheromones based on a unique hierarchy for each agent, i.e., deposit pheromones when they encounter other agents that have worse solutions than theirs. Furthermore, if two agents meet one other during a tour, a subjective/unique hierarchy is used to rank them and other agents according to the tour lengths. To this end, individuals have a unique list of ranked agents based not on the viewpoint of the third party but on the firstperson view. We found that such a contrivance allows the AS multi system to find better solutions than AS rank . However, agents in AS multi sometimes fall into a local solution caused by the mismatch between their subjective hierarchies and the system hierarchy, i.e., the actual single rank of the agents' tours. erefore, there is room for further improvement in AS multi .
In this study, we propose an improved version of AS multi . We name this as AS hierarchy . AS hierarchy follows AS multi ; however, agents in AS hierarchy sometimes deposit pheromones even when they encounter other agents with better solutions than themselves.
is occurs because agents consider the mismatch between their local hierarchies and the global (system) hierarchy as individual agents cannot know the absolute rank in the system hierarchy. From the viewpoint of a specific agent, the greater the number of agents offering better solutions than the agent who encounters them, the fewer the opportunities for the agent to get away from a local solution.
erefore, the agents are allowed to deposit pheromones to some extent in cases where they encounter other agents having better solutions.
us, individual agents in living systems seem to adapt as they consider the local information [9]. rough numerical simulations using several TSP datasets, we show that the proposed model outperforms AS multi .
Eliminating the viewpoint of the third party from the system is related to the subpopulations. Several studies have employed this approach for some optimisation problems [10][11][12][13]. For example, Cai et al. used the cooperative coevolution strategy to improve the global search capability for airport gate allocation optimisation [10]. Deng et al. used a similar mechanism, i.e., the CC framework, to deal with large-scale problems [11]. Multipopulations or population diversity contribute toward finding the global optimal solution. In our method, individual agents independently modulate their pheromone depositions based on their experiences during each tour.

Multirank-Based AS (AS multi ).
We propose a multirankbased ant system (AS multi ) [8] as an improved version of AS rank [7]. In AS multi , the probability of the next destination selection determined in the same manner as in AS rank , but a new approach is used for regulating agents who contribute to pheromone deposition. e new method of pheromone update in AS multi is explained here. First, we define the parameters as follows: s: step count in one tour θ step : threshold for tour step in agent competition ratio: the value calculated in agent competition φ ratio : threshold for ratio in agent competition C k : the binary parameter indicating whether agent k survived the competition N agent : the number of agents (equal to N city : the number of cities) N city : the set of cities city x (s, t): city where the agent x is in at step s city y (s, t): city where the agent y is in at step s ℓ x (s, t): accumulated tour length of agent x at step s ℓ y (s, t): accumulated tour length of agent y at step s L k : tour length of the solution found by agent k T k : set of edges included in the solution found by agent k If the following two conditions are satisfied, then a competition occurs among agents: (i) e parameters satisfy the relationship s/N city > θ step (ii) ere are other agents in the same city When one agent x encounters another agent y, i.e., city x (s, t) � city y (s, t), the agents mutually compare the accumulated tour lengths at tour step s of this tour to determine whether there is enough difference between their accumulated tour lengths. Using their accumulated tour lengths ℓ x (s, t) and ℓ y (s, t), an evaluation value ratio is calculated. If the ratio exceeds the threshold in agent competition φ ratio (ratio > φ ratio ), one long-length solution is defeated. In this case, 0 is assigned to the parameter C k of the defeated agent. Here, the competition process is defined as follows: e following prerequisites should be noted as well: (i) All parameters of C k are initialised to 1 at the beginning of a tour (ii) If an agent x is defeated even once in the middle of a tour, C x remains 0 until the end of that tour (iii) If three or more agents exist simultaneously in the same city, comparison for agent k is performed in all pairs until C k is replaced with 0 Finally, some agents who have never been defeated after completing a tour (C k � 1) can perform pheromone deposition. In this method, each agent decides about whether to take part in pheromone deposition using the information from contacted agents. As this method does not require the viewpoint of the external third party for summarising the agent's information, such as AS rank , it is faithful to the actual habits of ant swarms. After creating a tour, the pheromone update process is executed as follows: Agents in AS multi cannot perform pheromone deposition when they encounter other agents with better information. To this end, agents determine whether they are in a good/bad position in the global (system) hierarchy by using the subjective local hierarchy to confirm whether they have better/worse information than the agents encountered. However, identifying the local hierarchy as a subhierarchy of 2 Complexity the global hierarchy does not enable convergence to specific solutions. To solve this problem, we proposed AS hierarchy .

Hierarchical Rank-Based AS (AS hierarchy ).
Here, we describe the hierarchical rank-based ant system (AS hierarchy ) proposed in this study. Agents in AS hierarchy sometimes deposit pheromones even if they have worse solutions. From the viewpoint of a specific agent, the greater the number of encountered agents offering better solutions, the fewer the opportunities to get away from a local solution. To this end, agents in the proposed AS hierarchy sometimes deposit pheromones even when they have inferior information than the agents encountered to prevent the system from falling into a local solution.
Here, we define the parameters as follows mem k (t): lost-competition counter ω mem : threshold for weak pheromone deposition weak: pheromone value for weak pheromone deposition In AS hierarchy , equations (2) and (3) are modified as follows: then mem y ←mem y + 1, C y ←0.

(5)
Equation (4) is also modified as follows: where mem k is reset to 0 at the beginning of each iteration t. Figure 1 shows the algorithm flowchart. e computational complexity of AS hierarchy can be almost similar with that of AS rank because agents in AS hierarchy need not be sorted according to tour lengths before pheromone updates; instead, the encounter between agents is considered.

Results
We conducted a tour exploration experiment on TSP benchmark datasets and compared the AS hierarchy simulation results with that of AS multi as we have already compared the performance of AS multi with AS rank [8]. We found that AS hierarchy yielded better solutions than AS rank . We used four TSP datasets: Eil51.tsp (N city � 51), Berlin52.tsp (N city � 52), Lin105.tsp (N city � 105), and Pr124.tsp (N city � 124). Some of these datasets have often been used for benchmark tests [14,15]. e TSP datasets used here are classified as symmetric travelling salesman problem wherein the edge length between two cities in the opposite directions is the same. We focused on the TSP datasets adopting 2EUD. As shown in Table 1, the same values were adopted for the parameters common to AS multi and AS hierarchy [8]. AS hierarchy also has characteristic parameters ω mem and weak; these parameter values were selected after a series of preliminary experiments comprising 10 attempts for each dataset. e parameters common to AS multi and AS hierarchy were set according to the previous study where we found that AS multi outperformed AS rank [8].
For each model, Table 2 shows the averaged best solutions obtained from 100 trials. In these benchmark datasets, AS hierarchy outperformed AS multi for averaged best solutions. e p value in Table 2 shows that there is significant difference between the averaged best solutions of both models in the U test. In addition, the term "(optimal)" denotes the tour length of the optimal solution in each benchmark dataset. From the results in Table 2, it is seen that AS hierarchy , which allows agents who became a loser several times to update small quantities of pheromone, performs better than AS multi . Figure 2 shows the step interval of usual/weak pheromone depositions. We calculated the step interval of one agent using Eil51. According to Figure 2, the agent deposits weak pheromones more often than usual pheromones, satisfying equation (6). Such behaviours might inhibit rapid pheromone evaporation and result in the system getting away from local solutions.
Finally, we examined the parameter effect using Eil51. Here, we used the parameter value of ω mem as 10 (instead of 5). Table 3 shows the results. Again, AS hierarchy performed better than AS multi even after the parameter replacement.

Discussion
In this paper, we investigated whether AS hierarchy outperforms AS multi . AS multi is the basis for the proposed model. However, AS hierarchy allows agents who become a loser several times to update small quantities of pheromone. By contrast, agents in AS multi are never allowed to deposit any pheromones if they encounter other agents with better solutions. Weak pheromone depositions frequently occur in

Complexity
AS hierarchy , which might improve the performance of AS hierarchy . e important point in the proposed model is that a group that has relatively better information than a specific agent does not always exhibit better information because agents are allowed to use only limited information via interaction with local other agents. ACO does not guarantee convergence to the optimal solution [16,17]. Usual/weak pheromone depositions can serve as a key to achieve the balance between convergence and divergence to overcome this problem [18]. In this paper, we focused only on the proposed model's performance in terms of finding solutions; we did not consider other aspects such as the time required to find a correct solution or comparing our model with metaheuristics other than ACO. We would like to emphasise the importance of separation between third-party viewpoints and the first-person point of individual agents in multiagent systems. Researchers often introduce third-party viewpoints on metaheuristic systems derived from living systems that may not represent the original meaning of living/ natural systems to find better solutions. e strength of our findings is that individual agents decide to deposit pheromones based only on their experiences, which does not require seeing the system from a higher viewpoint. To do so, a system comprising many agents solves complex tasks flexibly without falling into a certain solution [19].
In future studies, we will investigate whether a stable solution can be obtained in other TSP datasets and find methods for parameter selection and for improving the algorithm's efficiency by considering the program execution steps.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare no conflicts of interest.

Authors' Contributions
TS and IA conceived and designed the experiments; TS and KU developed algorithms; TS performed analysis and wrote the paper.