Developing Programming Tools to Handle Traveling Salesman Problem by the Three Object-Oriented Languages

The traveling salesman problem (TSP) is one of the most famous problems. Many applications and programming tools have been developed to handle TSP. However, it seems to be essential to provide easy programming tools according to state-of-theart algorithms. Therefore, we have collected and programmed new easy tools by the three object-oriented languages. In this paper, we present ADT (abstract data type) of developed tools at first; then we analyze their performance by experiments. We also design a hybrid genetic algorithm (HGA) by developed tools. Experimental results show that the proposed HGA is comparable with the recent state-of-the-art applications.


Introduction
The objective of TSP is to find the shortest tour among a set of cites.Given the distance matrix (  ) where   stands for distance between the city  and , the problem is called symmetric TSP (STSP) when   =   and, otherwise, it is named asymmetric TSP (ATSP).
Since TSP is NP-Complete, there is no exact algorithm with time complexity better than an exponential time.It means that exact algorithms are not practical for the largescale instances in reasonable running times, so we have to use approximate algorithms to find the semioptimal solutions in acceptable running times.Recently, many approximate algorithms have been developed to handle TSP instances [1][2][3][4].The types of metaheuristics like genetic algorithms (GA) [5][6][7], simulated annealing [8], swarm based algorithm [9], artificial bee colony algorithm [10], ant colony algorithms [11,12], and combination of these algorithms have been applied to the TSP [13,14].However, if we consider the experiment sections of these references, we observe that almost all of these algorithms have not been applied to the instances with size of more than 1000.Among these metaheuristics, surly, Lin-Kernighan (LK) which is a type of local search algorithms (LSAs) (in this paper, LS points to the local search) is one of the best algorithms in which its extended types have been successfully applied to the large-scale instances with size of more than 85000 nodes [3,15].In addition, in many cases, these algorithms have been used in other metaheuristics and have increased their performance [2,11,16].
LSAs include 2-and 3-opt and Lin-Kernighan (LK) algorithms have been based on edges exchange process [1,3,15,17].GAs are population-based and their efficiency depends on their operators [4].To easily use these algorithms, we have programmed objective tools by three object-oriented languages which include C++, C#, and Java.These tools allow the researchers or developers to exploit these metaheuristics easily and create their own hybrid algorithms (these tools will be available via email request to esmkha@gmail.com(subject: TSP Tools)).
Developed tools in this paper mainly focus on types of genetic operators and LSAs; however, types of ant colony optimization (ACO) have been implemented separately and will be available.The implementation of LSAs has been based on LKH implementation [1,15,18] which is one of the most famous and effective implementations of LK.In addition, some famous initial-solution constructors like Suppose tour  with   edges that is defined on graph G(V, E): (1) Suppose direction for .
(2) If there are not nodes like A, B, C and D with below conditions then go to end.(i) AB, CD ∈   (a) In supposed direction, B and D are right nodes of A and C respectively.(ii) Cost(AB) + cost(CD) > cost(AC) + cost(BD) (3) Remove AB and CD form   and add AC and BD to it (2-opt-move).(4) Go to (2).
Quick-Boruvka and nearest neighbor (NN) strategy have been included in these tools.The genetic operators have been selected from literature.These operators include the PMX [19], EPMX [20], VGX [21], IGX [5], GX (description of this operator and its versions can be found in [5,19,21,22]), GSX-0, GSX-1, GSX-2, DPX [16], and OX [23].The implementations of these operators are effective.Experimental results show that, in almost all cases, the performance (in the terms of running time and accuracy) of developed operators is even better than reported results in their references.
This paper is not limited to the developed tools only.A type of hybrid GA which is proposed in this paper and uses a two-storey strategy is fast and accurate.Experimental results show that performance of proposed hybrid algorithm outperforms one of the recent state-of-the-art algorithms.
With these descriptions, this paper is organized as follows: in the rest of this paper, we briefly describe LSAs and the ADT of their programming pack.We review GA, its operators, and the ADT of their class in the third section.In Section 4, we combine the LK into the GA and design a hybrid GA.We put forward experimental results of these algorithms in Section 5 and finally summarize the paper in Section 6.

LSAs
Majority of LSAs for TSP have been based on the edges exchange process.The 2-opt, 3-opt, and LK are three important algorithms that are categorized in LSAs.We have programmed these heuristics by C++, C#, and Java.In this section, we review their algorithms briefly, and then we state ADTs of their programming tools.

2.1.
The 2-Opt.The 2-opt is a special case of the -OPT.A tour is named -OPT, if it is impossible to decrease the cost of tour by changing  number of edges.The 2-opt converts an input tour to its possible 2-opt case.Algorithm 1 shows the general algorithm for the 2-opt.
Instruction 3 in Algorithm 1 is named 2-opt-move or 2-change that is shown in Figure 1.In the 2-opt algorithm the 2-opt-move occurs when conditions in instruction 2 are satisfied.Time complexity of running exact 2-opt is high, so, to improve speed of the 2-opt algorithm, researchers usually use two important rules which have been proposed by Bentley.(1) For each node A in line 2, we only consider its candidate nodes.Usually the five nearest neighbors are selected to make a candidate set of each node.These sets can be approximately calculated by k-d-tree [25] in  ( log 2 ).
(2) In the instruction 2, only the active nodes are considered.The nodes which have participated in tour cost reduction in previous iteration are activated for the next iteration.This heuristic is known as "do not-look bits" [26].

2.2.
3-OPt.The 3-opt operates like the 2-opt but its conditions to exchanging edges is rather complicated (see [27]).Algorithm for the 3-opt in each step probes 3 edges to exchanging, so when three edges are deleted, six considerable cases appear and probing these cases increases time complexity and algorithm becomes more complicated to implement.

LK.
The types of LK may be the best heuristics that have been successfully applied to TSP.Furthermore, other metaheuristics like GAs widely use variant versions of this heuristic to improve their solutions.For more description about LK, we recommend readers to refer to [15,17] but here we present this algorithm in brief.The LK can be introduced by the three words: "break, " "link, " and "condition test".The LK algorithm is done in some iterations.In each iteration, it exchanges some edges by another to reduce tour cost.
Appendix A shows a simple algorithm for LK.

Tour Class.
Tour ADT is shown in Algorithm 3. Tour object is created to belong to Graph object.Tour object is completed after adding  (= dimension) nodes, either adding to the right (by using function in line 6) or adding to the left (by using function in line 7).

Heuristics Class.
We have packed 2-opt, 3-opt, LK, Quick-Boruvka, and double-bridge into the Heuristics class.To manipulate the candidate sets, we have also added some functions into the Heuristics class.Quick-Boruvka is effective tour constructor algorithm.Double-bridge is usually used to mutate.Algorithm 4 shows Heuristics class ADT.The LK method in Heuristics class has been based on latest version of LKH, so-called LKH-2, and its source code is in C language and free for academic use [18].

Genetic Algorithm
Genetic algorithm is one of the search algorithms that is inspired by evolutionary process of nature.In recent years, researchers have solved many NP-Complete problems by GA, scheduling [28,29], routing [30], and assignment [31,32] and many other problems have been solved by GA effectively in recent years.GA works with population of solutions and, in each step, new solution is created by the crossover operator, or one or more solutions are changed by the mutation operator.The crossover operators usually get two solutions from the population.These two solutions are so-called parents (or the father and mother).The crossover creates new solution(s) based on the parents.The new solution is called child or offspring.There is question in which solutions are suitable to submit to mutation or crossover operators.There are some papers answering this question [33,34].Crossover and mutation are two operators of GA which play an important role in evolution of solutions of GA.Generally, LSAs include LK extensions such as iterated LK (ILK) [3] and LKH versions [1,15] are very powerful in dealing with TSP.However, there are some effective GA or extensions like Nagata's one [4] that uses very efficient crossover operator, so-called edge assembly crossover (EAX).In this section, we review some of these crossover operators which have been included in the developed tools.3.1.Genetic Operators Review.Many GA crossover operators have been invented by researchers because the performance of GA depends on an ability of these operators.PMX [19] is one of the first crossovers which have been proposed by Goldberg and Lingle in 1985.Reference [20] states some shortcomings for PMX and to overcome them, proposing extended PMX (EPMX).DPX [16] is another crossover that produces child with greedy reconnect of common edges in two parents.Greedy subtour crossovers (GSXs) [24,35,36] family is another group of crossovers that operate fast.GSX-2 [36] is improved version of GSX-0 [35] and GSX-1 [24].Order crossover (OX) proposed by Davis is another one in which its extensions not only have been applied on TSP [23] but also solved many other NP-Completes [32,37].
In this subsection, we represent some of the recent GA crossovers and introduce them by examples.In these examples, we use the graph with eight nodes as this set: {1, 2, 3, 4, 5, 6, 7, 8} that its edges weight is as shown in Table 1.according to two parents by exchanging nodes between two arbitrary points.PMX is unable to detect the same nodes from mapped areas.In Figure 2, it can be easily seen that PMX cannot determine that node 7 is common in both mapped areas.PMX is double point crossover and these crossovers are not suitable to solve TSP.These defects can cause repetitive children production by this crossover [20].
(1) Node X ← select random node; (2) Copy X to child;   If remaining nodes are copied to child according to father, then child produced is the same as father. Child: Mother: Father:

Greedy Crossovers (GXs)
. Some versions of GX like very greedy crossover (VGX) [21] and improved greedy crossover [5] have been proposed by researchers in recent years.To review these crossovers, readers can refer to [5].

Improved Greedy Subtour Crossover (GSX-2)
. GSX-2 [36] is improved version of GSX-0 [35] and GSX-1 [24].GSX-0 is first version of GSX family.Algorithm 5 shows GSX-0 algorithm.Figure 3(a) shows GSX-0 example.In this example, after node 5 that has been included in child is met again, GSX-0 fills remaining places with random nodes but, same as Figure 3    that shows GSX-1 example, it fills remaining nodes in order of one of parents.
In general, GSX-1 operates better than GSX-1 because it can preserve order of remaining nodes but, in some cases, it produces repetitive tours; same as Figure 3(b), child tour is the same as father tour.Reference [36] states some shortcomings of GSX-0 and GSX-1 and, to overcome these shortages, proposes GSX-2.

Distance Preserving Operator (DPX)
. DPX [16] operates as follows: it detects common subpaths of two parents at first and then reconnects them greedily and produces child.Figure 4 shows DPX example that uses presented edges weight of graph in Table 1.

ADT of Class for Crossovers.
We have packed crossover operators into the Crossover class.Algorithm 6 shows ADT of Crossover class.Lines 5 to 14 show definitions of crossovers functions.Functions in lines 5 to 12 show crossovers which take two tours as father and mother and produce only one child, so their functions take three tour-pointers as the input arguments.The first two arguments are to point to the parent tours and third argument points to the child tour.Lines 12, 13, and 14 show OX, PMX, and EPMX which produces two children so their functions take four arguments.The first two arguments point to the parent tours and the second two arguments point to the two children tours.The performance of these crossovers, which are based on speed and accuracy, has been analyzed in [5].Results in [5] show that heuristic crossovers like IGX and DPX have more accuracy than others.These results also show that the crossovers like GSX family have more diversity than others which mean that these crossovers can produce wide range of different solutions.

Types of Genetic Algorithm.
There are two major models for GA: generational and steady-state GA.The main deference between generational and steady-state GA is that, in all evolutionary algorithms that incorporate local searches to improve their solutions.

Developing HGA by the Developed Objective Tools
To show applicability of proposed objective tools, we develop new model of HGA which differs with another versions in two main cases (See Appendices).
(1) The proposed HGA is two-storey GA.It means that the proposed HGA has been formed from two storeys of GA.First storey of GA uses heuristic genetic operator such as GX versions.This storey increases quality of population, so LSA can operate quickly in second storey.It should be tended that LSAs can operate quickly on high quality solutions.Therefore, this storey affects the second storey where LK is utilized.The LK operates quickly when it is applied to high quality tour.
(2) The second storey of the HGA is also HGA itself.
Like other HGA algorithms that incorporate LSA to increase tours quality, proposed HGA also does and exploits LK as its LSA but (I) it is updating LK candidates' sets periodically while these instructions of storey are executing; (II) in order to produce wide variety of solutions it should use quick crossover operator with high diversity same as classical PMX, GSX-1, or EPMX instead of heuristic crossovers that are usually slow.Notice that LK guarantees solutions' quality so it is not reasonable to use time consumer heuristic crossovers.

Experiments
In this section, we show objective tools performance.We divide this section to three subsections.In first subsection, we focus on LSAs tools, in second subsection, we put forward experimental results for the crossovers, and, finally, we exhibit experimental results of HGA designated by developed objective tools.

Performance of the Developed Tools.
To test developed LS tools including 2-opt, 3-opt, and LK, we apply them on fifteen TSPLIB instances twenty times.Users may need to be informed about accuracy and speed of LS tools, so here we report best, worst, average, and standard deviation of recorded costs and runtimes for LS tools per each instance in each of the twenty runs.Table 2 shows average, best, worst, and standard deviation of twenty solution costs for each instance achieved by each of the stated heuristics.Moreover, this table shows error percent of best, average, and worst solution costs which is calculated by (cost − optimum cost) × (100/optimum cost).Please consider that optimum cost for usa115475 is unknown so we have used best solution cost (6283142) that is obtained by LK tool.Table 3 presents runtime information of each heuristic applying to each instance in twenty runs.The minimum, average, maximum, and standard deviation of required runtimes have been listed in this table.
To make comparison among heuristic tools easy, we have introduced average error percent column of Table 2 and average time column of Table 3 by diagrams in Figure 5.

Performance of the Developed Crossovers' Tools.
To present crossover performance, we should show effect of crossover in GA accuracy, convergence speed, and ability of crossover in generating wide range of various solutions.To achieve these goals, we have to use generational GA because, in steady-state GA, generation size is constant but, in generational GA, the total generated solutions depend on ability of crossover in generating various solutions; if crossover can generate different solutions, so it delays generational GA convergence; then total generations increase.On the other hand, when generated solutions count is high, it shows that crossover diversity is high and it can produce wide range of various solutions.We used each of stated crossovers in generational GA to solve some instances from TSPLIB twenty times and Table 4 shows results of this experiment.
Table 4 shows information about best, worst, average, and standard deviation of solution costs for each of the twenty runs by each crossover when solving each instance.Figure 6 summarizes average error percent and STDV columns of Table 4.
In Figure 6 it can be easily seen that IGX has better performance in both average error percent and standard deviation.In average error percent and for kroA100, a280, lin318, rat783, and pr1002, IGX has first best rank and, only in att532, it has second minimum error percent.For standard deviation, also IGX has minimum in dealing with kroA100, a280, lin318, rat783, and pr1002.In solving att532, IGX has second minimum STDEV.Tables 5 and 6 show experimental results information about required runtime and total generations count in each of the twenty runs.These tables list best, average, worst, and standard deviation of required runtime and minimum, average, maximum, and standard deviation of total generations count.Average and STDEV columns of both tables have been introduced in Figure 7.

HGA Performance Analysis.
Comparing developed HGA with other state-of-the-art methods is not our purpose here but we want to show that it is possible to design and develop new memetic algorithms by our objective tools.Therefore, to achieve this goal we compare HGA with latest windows based version of LKH in period of 100000 seconds in solving pla85900 that is the largest problem in TSPLIB.Diagram in Figure 8 shows result of this competition.This diagram shows that HGA produces better tours than LKH during 100000(s) and its prominence is noticeable.

Conclusion
In this paper, we present highlight of our TSP programming tools that have been based on LKH implementation.In fact, these tools are source codes in three object-oriented languages: C++, C#, and JAVA.These tools can help engineers, researchers, and those who are dealing with TSP to write and develop their TSP applications more easily by one of the stated programming languages arbitrarily.Here, we tried to show our tools' performance by experiments.In order to show their applicability, we designed a hybrid algorithm that
3.1.1.PMX Crossover.Partially mapped crossover (PMX) is one of the first genetic operators.It produces two children nodes are copied to random places.
already been copied to child.

Figure 7 :
Figure 7: (a) Average required runtime and (b) STDEV of required time.

Table 1 :
Distance matrix for examples.

Table 2 :
(a) 2-Opt performance based on solution costs.(b) 3-Opt performance based on solution costs.(c) LK performance based on solution costs.

Table 4 :
Crossovers performance analysis on solution costs.

Table 5 :
Required runtime for generational GA with each crossover.