Heuristics for Routing Heterogeneous Unmanned Vehicles with Fuel Constraints

This paper addresses a multiple depot, multiple unmanned vehicle routing problem with fuel constraints. The objective of the problem is to find a tour for each vehicle such that all the specified targets are visited at least once by some vehicle, the tours satisfy the fuel constraints, and the total travel cost of the vehicles is a minimum. We consider a scenario where the vehicles are allowed to refuel by visiting any of the depots or fuel stations. This is a difficult optimization problem that involves partitioning the targets among the vehicles and finding a feasible tour for each vehicle. The focus of this paper is on developing fast variable neighborhood descent (VND) and variable neighborhood search (VNS) heuristics for finding good feasible solutions for large instances of the vehicle routing problem. Simulation results are presented to corroborate the performance of the proposed heuristics on a set of 23 large instances obtained from a standard library. These results show that the proposed VND heuristic, on an average, performed better than the proposed VNS heuristic for the tested instances.


Introduction
Small unmanned vehicles (UVs) are being used in several environmental monitoring applications [1][2][3][4][5][6] to collect information such as temperature, moisture, and humidity.These applications require UVs to visit specific target locations and monitor large swathes of land to collect data.Even though there are both economical and operational benefits [7] in using small UVs, they also come with other resource constraints due to their size and limited payload.Typically, small UVs due to their limited fuel capacity may have to revisit the depots (or fuel stations) multiple times for refueling while executing a surveillance mission.Path planning for small vehicles becomes critical in this scenario if the available resources such as fuel must be used as efficiently as possible.
This paper considers a fundamental routing problem that arises in these monitoring applications and is stated as follows: given a set of target locations, fuel stations (or depots) and UVs, find a path for each vehicle such that each target is visited at least once by a vehicle, each vehicle satisfies the fuel constraint as it traverses along its respective path and the travel cost of all the vehicles is a minimum.The travel cost we consider is the total fuel consumed by all the vehicles.To simplify the problem, we assume that the fuel consumed by a vehicle is directly proportional to the distance traveled by the vehicle.The vehicles are expected to refuel at the fuel stations as they run out of fuel.Vehicles are heterogeneous as they are allowed to carry fuel tanks with different capacities.This problem is referred to as the multiple, heterogeneous unmanned vehicle routing problem (MHUVRP).
In the absence of fuel constraints, MHUVRP is a generalization of the traveling salesman problem (TSP) and is NP-hard [8].The difficulty in solving the TSP is further compounded when multiple vehicles are considered and even more when fuel constraints are imposed on these vehicles.Therefore, the focus of this paper is on developing heuristics that can find good solutions to the MHUVRP as quickly as possible.We accomplish this through the framework of the variable neighborhood search (VNS) and variable neighborhood descent (VND).VNS and VND are metaheuristics [9] used to solve difficult combinatorial and global optimization problems.These are iterative algorithms where in each iteration, the algorithms search through multiple neighborhoods of the current feasible solution to find a feasible solution with lower cost.The use of multiple neighborhoods allows the solution in the VNS and VND heuristics to move away from local optima as quickly as possible.
Routing for UVs has been addressed by several authors in [10][11][12].The single vehicle version of the MHUVRP has been addressed by Khuller at al. [13] and Sundar and Rathinam [14].Authors in [13] present an approximation algorithm for the symmetric version of the problem.An -approximation algorithm is a polynomial time algorithm that produces a solution whose cost is at most  times the optimal cost for any instance of the problem.Authors in [14] present an approximation algorithm for the asymmetric version of the problem.In addition to an approximation algorithm, 2-opt and 3-opt heuristics were presented in [14] to find good feasible solutions to the single vehicle problem.Computational results [14] showed that the -opt heuristics in combination with the approximation algorithm can produce near-optimal solutions for single vehicle instances with 25 targets within a couple of seconds of CPU time.
The MHUVRP is also closely related to routing problems with intermediate facilities [15,16].Other variants of the MHUVRP have also been studied in the literature.Authors in [17] consider a multiple vehicle TSP incorporating time windows and equity constraints.Approximation algorithms for a heterogeneous multiple vehicle TSP and a Hamiltonian path problem were studied in [18,19].Oberlin et al. [20] present a transformation of a heterogeneous multiple vehicle, multiple depot TSP into an asymmetric TSP so that algorithms for the standard TSP can be put to good use.
The contributions of this paper are as follows.
(i) We develop algorithms based on the VNS and VND heuristics proposed by Hansen and Mladenovic [9] for the MHUVRP using multiple neighborhoods.
(ii) To understand the effect of choosing different neighborhoods and initial solutions on the quality of the final solutions, we implement the proposed algorithms and test them on large problem instances from the multiple depot vehicle routing library by Cordeau [21].These results give insights as to which of the neighborhoods provide substantial improvements in the search process and the order in which the neighborhoods must be selected for the VNS/VND algorithms to obtain good solutions for the MHURVP.

Problem Statement
Let there be  vehicles denoted by V A tour for a vehicle is denoted by a sequence of vertices (,  1 , . . .,   , ) where   ∈  for  = 1, . . .,  and  ∈  is the depot corresponding to the initial location of the vehicle.The cost of traveling a tour is defined as the sum of the cost of traveling all the edges in the tour.A tour satisfies the fuel constraint for its corresponding vehicle if the vehicle does not run out of fuel while traversing its tour.The objective of MHUVRP is to find  tours, one for each vehicle, such that each target is visited at least once by some vehicle, the tours satisfy the respective fuel constraints of the vehicles and the total cost of traveling all the tours is a minimum.

Algorithms
Heuristics based on the variable neighborhood search (VNS) are developed in this section for solving the MHUVRP.VNS was proposed by Hansen and Mladenovic in [9,22].The main idea of the VNS is to perform a local search systematically using multiple neighborhoods.It explores increasingly distant neighborhoods of the current solution iteratively and jumps from its current solution in the solution space to a new one if and only if an improvement has been made.
The steps of the basic VNS are shown as in Algorithm 1.In Algorithm 1,   ( = 1, . . .,  max ) denotes a finite set of neighborhood structures.The stopping conditions may include criteria based on the allowable CPU time, maximum number of iterations, or maximum number of iterations between any two improvements.One can observe that the basic VNS heuristic also contains a probabilistic component in the shaking phase.The shaking step is a characteristic feature of the VNS heuristic and it allows the algorithm to get out of a local optimum.The solution obtained from the local search phase is compared to the incumbent solution and is accepted as a new starting point if an improvement can be made; otherwise, it is rejected.Therefore, the VNS procedure is a descent, first-improvement method with randomization.This method can also be used without the randomization phase.Such a method, named as the variable neighborhood descent (VND) heuristic [9], is the same as the VNS heuristic save for the absence of the shaking phase.In the following sections, we describe our algorithms for generating the initial solutions and the improvement procedures in detail.

Initial Solution: Construction Phase
Construction heuristics are developed by generalizing the approach of the single vehicle algorithm in [13] to multiple vehicles.In particular, we develop two construction heuristics that provide two initial feasible solutions for the MHUVRP.In the following discussion, we summarize the main steps of these construction heuristics and give the details in the appendix.
In the first construction heuristic, we first compute a new, modified traveling cost for each vehicle that also includes the fuel constraints of the vehicle.This traveling cost will also include the extra fuel a vehicle may require Initialization: Select a set of neighborhood structures denoted by   ( = 1, . . .,  max ) that will be used in the search; find an initial solution ; choose a stopping condition.Repeat the following until the stopping condition is met: (1) Set  ← 1.
( to visit all the refueling stops when it traverses between any two targets.We compute this new traveling cost based on the approach presented in [13].For completeness, we have presented the details of this approach in the appendix of this article.Suppose   (, ) denote the modified cost of traveling from target  to target  for the th vehicle including the refueling stops.It is easy to verify that these traveling costs monotonically increase with the index of the vehicles since the vehicles are ordered such that the fuel capacity of the vehicles decreases, that is, Once these traveling costs are computed, the primal-dual heuristic presented in [23] is used to assign the targets to the vehicles.After partitioning the targets to the vehicles, the best single vehicle algorithm available in [14,24] is used to find a feasible tour for each vehicle.This construction heuristic is referred to as the Approximate Primal Dual (APD) algorithm.
In the second construction heuristic, each target is assigned to the nearest initial depot of the vehicle based on the Euclidean distances between the targets and depots.Once the targets are partitioned among the vehicles, the best single vehicle algorithm available in [14,24] is used to find a feasible tour for each vehicle.This construction heuristic is referred to as the Voronoi algorithm.Note that the two construction heuristics primarily differ only in the partitioning of the targets.

Variable Neighborhood Search: Improvement Phase
This section explains the choice of neighborhoods and the main steps in the improvement phase of the VNS algorithm (Algorithm 1) that includes the shaking step, local search, and the move or not step.

Neighborhood Selection.
We use four neighborhoods; three of these are intra-route neighborhoods (2-exchange [25], 3-exchange [25], and depot exchange [24]) and the last one is an inter-route neighborhood (relocate [26]).Any two solutions can be present in an intra-route neighborhood only if the assignment of targets to the vehicles are the same in both the solutions.On the other hand, inter-route neighborhood considers solutions where the assignment of targets to the vehicles are different.We first formally define all the neighborhoods in the ensuing discussion.A solution  2 is said to be in the exchange neighborhood of solution  1 if there is exactly one vehicle V  such that (i)  2 and  1 may differ only in the tours of vehicle V  , and (ii) the tour for vehicle V  in  2 differs from the tour for vehicle V  in  1 by at most  edges.
A solution  2 is said to be in the depot exchange neighborhood (Figure 1) of solution  1 if there is exactly one vehicle V  such that (i)  2 and  1 differ only in the tours of vehicle V  , and (ii) the tour for vehicle V  in  2 differs from the tour for vehicle V  in  1 by exactly two edges and one depot in  \   where   denotes the initial depot of vehicle V  .
A solution  2 is said to be in the relocate neighborhood (Figure 2) of solution  1 if there are exactly two distinct vehicles V  and V  satisfying the following conditions: (i)  2 and  1 differ only in the tours of vehicles V  , V  , (ii) there is exactly one target  such that  is visited by V  in  1 whereas  is visited by V  in  2 . of these neighborhoods is to sufficiently perturb the initial solution (computed in Section 4), while ensuring that this new solution keeps certain aspects of the initial incumbent.In this step, a random feasible solution of the currently selected neighborhood of  is found and denoted as   .This random selection enables the algorithm to avoid local optima in the local search procedure.For certain neighborhoods, it may be possible that there are no feasible neighbors; that is, the neighborhood of  is empty.In this special case, the shaking step is skipped, and   is the same as . Figure 3(a) shows the solution space where each solution is shown using a black dot, the neighborhood of  is denoted by a circle, and the outcome of the shaking step is denoted as   .

Local Search.
The output solution (  ) from the shaking procedure is then improved using a local search procedure.
The local search examines all the feasible neighbors of   in the current neighborhood and finds the solution   which has the least cost.Again, it may be possible that the current neighborhood of   is empty.If this is the case,   is set to be the same as   .

Move or Not: Acceptance Decision.
In this step, the cost of   is compared to the cost of .The two possible outcomes of interest are when the cost of   is less than the cost of  and when the cost of   is greater than or equal to the cost of .In the first case,  is set to be   (see Figure 3(b)).When the first outcome is true, the first neighborhood is set as the "current" neighborhood ( ← 1 in Algorithm 1), and computations continue with the shaking step.When the second outcome is true,   is forgotten, and the next neighborhood is set as the "current" neighborhood ( ← +1 in Algorithm 1).If there is no next neighborhood, that is, the "current" neighborhood is the last neighborhood designated, the algorithm terminates.

Computational Analysis
The VNS/VND heuristic proposed in Section 5 was implemented using Visual C++ and the elements of the Standard Template Library (STL).Simulations were performed on a 2.4 GHz AMD Phenom machine.The problem instances on which the simulations were performed derives from the Cordeau's benchmark instances [21] for the multiple depot vehicle routing problem.Additional information was added to these benchmark instances to satisfy the requirements of the MHUVRP as follows.
(i) The number of customers in the benchmark instances was used as the number of targets for the MHUVRP.
(ii) The depot locations in the benchmark instances were used as the initial depot locations of all the vehicles.
(iii) The number of vehicles in the MHUVRP was set to be equal to the number of depots in the benchmark instances.
(iv) The fuel capacity of the th vehicle in an instance was computed using the formula   :=  − 15( − 1) where  is the size of the surveillance area. was set to be equal to max ,∈ |  −   | where   denotes the  coordinate of vertex  (rounded to the nearest 100 units).
The above modifications resulted in 23 distinct instances (listed in Table 1) on which the computational study was performed.The number of targets and vehicles in these instances varied from 50 to 360 and 2 to 9, respectively.We use a maximum allowable time of 1000 secs as the stopping condition for the heuristics. 1 shows the cost of the initial feasible solutions produced by the two construction heuristics.On an average, these results show that the initial feasible solution produced by the APD algorithm was superior to the initial feasible solution produced by the Voronoi partitioning algorithm for a majority of instances.

Analysis on the Best Sequence of Neighborhoods.
In this section, we first study the effect of the choice of the neighborhoods used in the VNS and VND algorithms.The % improvement in the quality of solutions using an algorithm is defined as 100 × ((Cost  − Cost alg )/Cost  ) where Cost alg denotes the travel cost of the final solution produced by the algorithm and Cost  denotes the travel cost of the initial solution produced by a construction heuristic.As there are two construction heuristics used, we evaluate the effect of the sequence of neighborhoods used on both the initial solutions obtained using the construction heuristics.Table 2 shows the average improvement in the quality of the solutions using a sequence of 4 neighborhoods.We only report the sequences of the neighborhoods that produced the best   improvements.In general, the order of the neighborhoods did not significantly affect the improvement in the quality of the solutions.Table 3 shows the average improvement in the quality of the solutions using a sequence of 3 neighborhoods.These results show that the VND heuristic provides more improvement in the quality of the solutions, on an average, compared to the VNS heuristic.We also note that majority of the best sequences has 3-opt neighborhood as the last search neighborhood.In general, we found that the addition of the 3-opt neighborhood produced high quality solutions for the problem at the expense of computation time.Table 4 shows the average improvement in the quality of the solutions using a sequence of 2 neighborhoods.This table provides a clear picture of the overall improvement capability of the different neighborhoods.These results show that the relocate and 3-opt neighborhoods appear most frequently followed by the 2-opt, indicating that the neighborhood combinations containing 3-opt or relocate are the most effective, even though they take an order or two of magnitude longer to run than the other neighborhood combinations.They also provide the best improvement in the quality of solutions on an average.Finally, Table 5 shows the effect of choosing a single neighborhood in the improvement of the quality of the solutions.Results in Table 5 shows a clear tradeoff in the quality of the solutions produced by an algorithm versus the computation time the algorithm requires.In terms of solution quality, algorithms using the 3-opt neighborhood performed the best while using the maximum amount of computation time.On the other hand, the algorithms using only the relocate neighborhood ran quickly while producing an improvement of up to 17.56%.

Effect of the Construction Heuristics on the Final Cost.
Table 6 shows the effect of the two construction heuristics on the quality of the final solutions obtained by the VNS and VND algorithms.In this table, Cost  V and Cost  apd denote the cost of the final solution obtained by the algorithms using the Voronoi and the APD heuristic, respectively.The table shows the average values of (( × 100 for some of the best combination of neighborhoods.For a majority of instances, these results show that the choice of the construction heuristic, on an average, does not significantly affect the quality of the final solutions produced by the algorithms.Figures 4 and 5 show the initial and the best solution obtained using the proposed algorithms for the instance pr05.

The Best Algorithm with the Neighborhood Combinations.
In this subsection, we specify the best combination of the neighborhoods obtained for the instances and the corresponding cost of the final solutions obtained by the algorithms.For a single neighborhood, the VND heuristic with 3-opt (Table 7) performed the best with the initial solution provided by the APD heuristic.This heuristic improved the quality of the solutions by 22.96% on an average.For two neighborhoods, the VND heuristic with relocate as the first neighborhood and 3-opt as the second neighborhood (Table 8) performed the best with the initial solution provided by the Voronoi heuristic.This heuristic improved the quality of the solutions by 39.42% on an average.We found that the relocate neighborhood in combination with either 2-opt or 3-opt were effective in improving the quality of the solutions substantially.
For three neighborhoods, the VND heuristic with relocate, depot exchange, and 3-opt neighborhoods (in the given sequence) performed the best with the initial solution provided by the Voronoi heuristic.In particular, they improved the quality of the solutions by 42.59% as shown in Table 9.For four neighborhoods, the VNS heuristic with relocate, depot exchange, 2-opt, and 3-opt neighborhoods (in the given sequence) performed the best with the initial solution provided by the Voronoi heuristic.This heuristic improved the quality of the solutions by 42.22% as shown in Table 10.In general, using four neighborhoods as compared to three neighborhoods only increased the computation time while not providing any substantial improvements in the solution quality.

Conclusion
The effects of different neighborhoods and partitioning heuristics on benchmark instances for the MHUVRP were examined.The neighborhoods included the 2-opt, 3-opt, depot exchange, and the relocate neighborhoods.Simulation results showed that, on an average, the construction An edge is present in this graph only if traveling the edge can satisfy the fuel constraint.For example, as the vehicle V  has at most   −   units of fuel to start with, the vehicle can reach a depot  from  only if   ≤   −   .Therefore,   contains an edge (, ) if the constraint   ≤   −   is satisfied.Similarly, the vehicle can travel from a depot  to target  only if there are at least   units of fuel remaining after the vehicle reaches .Therefore,   contains an edge (, ) if the constraint   ≤   −   is satisfied.In summary, the following are the edges present in   :  Any path starting at  and ending at  in this auxiliary graph will require the vehicle V  to carry at most   −   units of fuel at target , satisfy all the fuel constraints, and reach target  with at least   units of fuel left.Also, we let the cost of traveling any edge (, ) ∈   to be equal to   (as defined in Section 2).Now, we use Dijkstra's algorithm [27] to find a shortest path to travel from  to .This shortest path (also referred to as the indirect path using intermediate depots) can be represented as PATH  (, ) := (,  1 ,  2 , . . ., ).Let the length of this path be represented by   (, ).This new cost function is computed for every vehicle between every pair of targets and is denoted by   where  = 1, . . ., .

) 1 ; 1 :
Repeat the following steps until  =  max : (a) Shaking: Generate a feasible solution   at random from the th neighborhood of  (  ∈   ()); (b) Local search: Apply a local search method with   as initial solution; denote the so obtained local optimum as   ; (c) Move or not: If   is better than incumbent , set  ←   and continue the search with  1 ( ← 1); otherwise set  ←  + Algorithm Steps of the basic VNS by Hansen and Mladenovic.
Move or not step

Figure 3 :
Figure 3: Shaking step and move or not step of the VNS heuristic.

Figure 6 :
Figure 6: The first step of the algorithm: the solid edges represent the shortest path PATH  (, ) for the vehicle V  , to travel from target  to target , and the cost of traveling this path is denoted by   (, ).
1 , V 2 , . . ., V  with fuel capacities denoted by  1 , . . .,   , respectively.Without loss of generality, we assume that  1 ≤  2 ⋅ ⋅ ⋅ ≤   .Let  denote the set of targets to be visited, and let  denote the set of depots (or fuel stations) that are available.Let  := ∪.Each vehicle is initially located at one of the depots.The problem is formulated on a complete undirected graph  = (, ) where  is the set of edges joining any two vertices in .Let the amount of fuel (travel cost) required to travel between any two vertices ,  ∈  be represented by   .The travel costs are assumed to be symmetric and satisfy the triangle inequality; that is, for distinct vertices , ,  ∈ , we have   =   and   +   ≥   .Additionally, for any target  ∈ , it is assumed that there is at least one vehicle V  and a depot  ∈  such that 2 ⋅   ≤   .This is a reasonable assumption because target  will be unreachable for any vehicle if this assumption is not true.

Table 1 :
Comparison of the initial feasible solutions.

Table 2 :
Influence of 4 neighborhoods sequences on improving quality.

Table 6 :
Comparison between the final solutions produced by the algorithms.

Table 7 :
Results using the VND heuristic with 3-opt.

Table 8 :
Results using the VND heuristic with relocate and 3-opt.

Table 9 :
Results using the VND heuristic with relocate, depot exchange and 3-opt.

Table 10 :
Results using the VNS heuristic with relocate, depot exchange, 2-opt and 3-opt.Instance Initial Cost (Cost  ) Final Cost (Cost  ) (Cost  − Cost  )/Cost  (%) yThe shortest path from x to y