A P-Based Hybrid Evolutionary Algorithm for Vehicle Routing Problem with Time Windows

The ability to solve optimization problems using membrane algorithms is an important application of membrane computing. This work combines membrane systems and genetic operators to build an approximated algorithm for the vehicle routing problem with time windows. The algorithm is based on a tissue-like membrane structure combined with cell separation rules and communication rules; during such processes membranes collect and disperse information. Genetic operators are used as the system’s subalgorithms. We also design a special improvement strategy to speed up the search process in subsystems. The experimental results show that the solution quality from the proposed algorithm is competitive with other heuristic or metaheuristic algorithms in the literature.


Introduction
The membrane computing is a branch of natural computing: it is inspired by the structure and the function of living cells [1] as well as the organization of cells in tissues, organs, and other higher order structures.The computational devices in membrane computing are called P systems, which consist of three parts: the membrane structure, the objects in the membrane regions, and the evolution rules.Roughly speaking, there are three types of P systems, that is, cell-like P systems [2,3], tissue-like P systems [4,5], and neuron-like P systems [6,7].They provide distributed parallel and nondeterministic computing devices.Membranes delimit space to different regions, and objects evolve according to evolution rules in each region.
The membrane algorithm, which was introduced by Nishida [8] in 2005, is a membrane computation optimization algorithm.It combines the concepts of P systems with some local approximate algorithms.In the framework of membrane computing, the objects and the membranes evolve step by step according to specific evolution rules such as rewriting, communications, and division rules.This process can describe many of the evolutionary concepts of living cells.A membrane algorithm is also called a P-based algorithm.In a membrane algorithm, the membranes separate space into several regions, where subalgorithms and tentative solutions are placed as well as transporting mechanism and transforming mechanism of membrane structure.The subalgorithms are always selected from heuristics operators.From a computational point of view, a P system provides a good framework for an approximation algorithm because of its hierarchical structure and intrinsic parallelism.Compared with a traditional hybrid optimization algorithm, the membrane algorithm contains rich changes in algorithm structure and stochastic processes.
Proper architecture design is an important research aspect for membrane algorithms.The membrane system proposed by Nishida (see [8]) includes a nested membrane structure (sometimes the structure can be changed during the computation process), while a local search algorithm supplies the evolution scheme for each region.Tentative solutions were placed in the regions delimited by the membranes.When computing the results for the different regions, exchanges are included by applying transportation and membrane structure transformation mechanisms.The membrane algorithm designed in [8] efficiently deals with the traveling salesman problem (TSP).In [9], the minimum storage problem was successfully solved using a membrane algorithm.In [10], Zaharie and Ciobanu discussed the similarities between distributed evolutionary algorithms and P systems.They suggested novel variants of distributed evolutionary algorithms to solve numeric optimization problems.Several approaches combining (nested membrane structure) NMS with genetic algorithms have been presented to solve numeric optimization problems [11], traveling salesman problems [12], controller design problems for a marine diesel engine [13], gasoline blending and scheduling problems [14], and optimal control problems for a time-varying unstable plant [15].A quantum-inspired evolutionary algorithm based on P systems (QEPS) was proposed in [16].It incorporated a one-level membrane structure (OLMS) and a quantuminspired evolutionary algorithm.The difference between the performances of a membrane algorithm with OLMS and a membrane algorithm with a nested membrane structure was also analyzed.In [17], a modified QEPS with a local search was discussed by analyzing radar emitter signals.
In [18], OLMS was integrated with DE and ant colony optimization methods to solve numeric optimization and traveling salesman problems.These studies clearly illustrate that membrane systems can be used to improve the performance of many metaheuristic search methods and that they sometimes balance exploration and exploitation efficiently [19,20].To date, there are very few papers that consider a tissue P system to be a membrane structure of P-based algorithms [21], but our paper is a contribution to this line of research.
The vehicle routing problem with time windows (VRPTW) is an NP-hard problem that can be applied to many distribution systems [22][23][24][25], such as the rail distribution system, the school bus routing system, and the mail and newspaper delivery system [26].In the literature, many heuristic or metaheuristic approaches have been proposed for solving the VRPTW [27].In [28], tabu search was first applied to the VRPTW.Other studies on tabu searches for the VRPTW can be found in references [29][30][31].Genetic algorithms were proposed for the VRPTW by [32][33][34][35][36][37].Other metaheuristics have been applied to the VRPTW that are based on simulated annealing or the ant colony approach.In recent years, hybrid algorithms have drawn much attention [38][39][40][41]; they combine the advantages of different algorithms and always yield better solutions than a single algorithm.However, very few papers emphasize the effect on the experimental results of architecture design throughout the computation process.
In this study, a P-based hybrid evolutionary algorithm (PHEA) is proposed to address the VRPTW.The structure of PHEA is based on a tissue-like membrane structure, and the genetic operators are used as the system's subalgorithms.The subalgorithms associated with membrane systems are inspired by biological phenomena.As we know, DNA sequences are composed of a special material in the nucleus.DNA carries almost all genetic information, and it has its own specific evolutionary concepts including recombination, mutation, and selection.We focus on the influence a membrane structure has on those genetic operations.
The PHEA can be considered as a bioinspired technique.It begins with a tissue membrane structure.After cell separation, genetic operators in the subsystems are used to make better solutions step by step.Some characters of membrane systems, such as cell separation and cell communication, can greatly improve the performance of genetic operators.Moreover, this paper includes a special improvement strategy to speed up the search process.Our algorithm is tested by Solomon's benchmark problems.Compared with the bestknown solutions in the literature, our algorithm is competitive in terms of its solution quality.

Vehicle Routing Problem with Time Windows
The VRPTW can be described as a problem of designing an optimal set of routes that services all customers.Each customer is visited only once by exactly one vehicle within a given time interval; all routes start and end at the depot, and the total demands of all customers on one particular route must not exceed the capacity of the vehicle.Mathematically, it can be described as follows [33].There are  customers who are serviced from a depot with vehicles of the same capacity, .For each customer , where  = 1, . . ., , there is a demand (  ), a service time (  ), and a service time window (  = [  ,   ]); the service begins at time   .The lower bound,   , describes the earliest time to start servicing and the upper bound,   , describes the latest time to start servicing.The demand   of customer  is to be covered by exactly one service within the time window of   .In addition,  0 describes the earliest time a vehicle can depart from depot 0, and  0 describes the latest time for the vehicle's arrival at the depot.The objective is to determine a feasible route schedule which primarily minimizes the number of vehicles and secondarily the total travel distance.Consider the following: subject to where   represents the transportation cost from customer  to customer  and  is the number of vehicles.

A P-Based Hybrid Genetic Algorithm
In this paper, we propose a P-based hybrid evolutionary algorithm (PHEA).It is designed with the concept of tissue P system and genetic evolution.The flowchart of PHEA is shown in Figure 1.
The representation of a solution we use here is an integer string, first applied by Tan et al. [33].The length is the number of customers in a route.Each point in the string is the integer node number originally assigned to that customer.The sequence of the points in the string is the order for visiting these customers.
The initial structure is [ ] 1 [ ] 2 .We call the membrane with label 2 the operation membrane and call the membrane with label 1 the output membrane.The local best solution in an operation membrane is denoted as  best .The global best solution of the system is denoted as  best .The operation membrane which obtains  best is called the best operation membrane, while the operation membrane which obtains the worst  best is called the worst operation membrane.
The procedure of the PHEA is described as follows.
Step 1.After the parameters have been set, such as the iteration number, the population size, the crossover rate, and the mutation rate, the PHEA randomly generates the initial chromosomes of the problem in membrane 2 using a random number generator.
Step 2. Separation rules are used to generate  cells labeled with the number 2. The structure variant is shown in Figure 2. The initial chromosomes are separated into  membranes, also with label 2. The number of chromosomes will differ by no more than one.
Step 3. In each operation membrane, a hybrid evolutionary algorithm is used to improve the initial routes.
Step 4. To avoid premature convergence and to add diversity to the new population, communication rules are used to exchange several chromosomes between the best operation membrane and the worst operation membrane for each  max step; see Figure 3(a).Step 5.The operation membrane sends its chromosomes into the output membrane for each  max step, and then the output membrane reassigns the population to  operation membranes; see Figures 3(b) and 3(c).The number of chromosomes will differ by no more than one.
Step 6.If the iteration number reaches the maximum iteration number Iter max , the output membrane outputs  best ; see Figure 3(d); otherwise, the chromosomes are sent to the operation membranes, and return to Step 3.

Hybrid Genetic Algorithm in Subsystems.
In each operation membrane, hybrid evolutionary algorithm (HEA) combines several genetic operators with an improvement strategy.
The flowchart is shown in Figure 4.
The procedure of HEA is described briefly as follows.
Step 1. Initialize the population.
Step 2. Evaluate individuals and select parents.
Step 3. Use the order crossover operator, the single parent crossover operator, and the mutation operator to generate children from selected parents.
Step 4. Use the improvement strategy to improve the individuals and update the population.
Step 5.If the iteration number of the local search in the operation membrane reaches  max or the multiples of   max , the operation membrane sends its chromosomes into the output membrane.Otherwise, return to Step 6.
Step 6.If the iteration number of the local search in the operation membrane reaches  max or the multiples of  max , exchange several chromosomes between the best operation membrane and the worst operation membrane.Otherwise, return to Step 2.

Selection and Evaluation.
We implemented the tournament selection method to select parents and then generate offspring.A "tournament" is performed among a few individuals chosen at random from the population and the best parent is selected.The tournament size is an important parameter to adjust selection pressure.If the tournament size is larger, weak individuals have a smaller chance of being selected.However, a smaller size is a better choice to ensure diversity.The evaluation of individuals is based on the objective to determine a feasible route schedule that primarily minimizes the number of vehicles and secondarily the total travel distance.

Recombination.
Recombination operators are designed to generate children from selected parents.We used two types of reproduction operators in our algorithm.One of the reproduction operators used was a classical order crossover (OX), in which the relative order of the customers in the parent chromosomes is preserved in the child chromosome [43].The other reproduction operator used was single parent crossover operator (SPO) [44].This operator allows for more diversity.In it, each trip (a sequence of customers) in the parent solution has a 50% chance of being duplicated in the offspring solution.All other customers not being duplicated will be inserted into the sequence one by one, randomly.Among all of the possible positions to insert a customer, the customer is added to the position that results in the smallest increase in the total cost.

Mutation.
We used three-point mutation in this paper to promote population diversity.The procedure of this operation is listed as follows.
Step 1. Pick three points at random from a sequence.
Step 2. Generate all of the possible permutations of the selected points while keeping the other points in the sequence unchanged.Then, five new sequences can be obtained, and they are regarded as the offspring.
Five chromosomes can be created from a parent's sequence by exchanging three genes, as shown in Example 2. Only the best transformation is used as the offspring produced by the mutation.[19].We first set a length .All of the customers in the route whose length is less than  will be reinserted in other routes using the INSERT Algorithm  1 .Thus, if the   length on a route is no less than , the route can be saved in the next step.Otherwise, the customers of the routes whose lengths are less than  need to choose a new suitable insertion place.We try to add those customers to other routes such that their length is no less than .Equation (3) [19] is used to calculate the best feasible insertion place for customer .Equation ( 5) [19] is used to choose the customer  * who will be inserted in the next step.Consider the following: where The triple function ( −1 , ,   ) represents customer  being inserted between the points  −1 and   ,  represents the number of customers,   represents the distance from customer  to customer ,   represents the time when service at customer  begins, and   represents the start time for customer  after inserting customer .
Consider the following: where When no more customers with feasible insertions can be found, the algorithm begins a new route (unless it has already routed all customers).This process is illustrated in Figure 5. In

Results of PHEA.
The comparison between the bestknown solutions from the literature [29,42,45] and the solutions obtained by the PHEA is shown in Table 2.The solutions obtained by the PHEA are better than those in the previous literature for R107, R109, R112, RC101, RC105, RC106, and R202.The solution would be improved further if every parameter is set more reasonably.A comparison is also made between the optimal solutions obtained by the PHEA, a hybrid genetic algorithm [29], and an improved genetic algorithm [42]; see Table 2.The results show that our algorithm is competitive with other genetic algorithms.
The solutions for C103 and C104 comprise one vehicle less than the previous best solutions.Two complete solutions that include the sequence of customers are described in Table 3.

4.4.
Comparison with the Results of PHEA −,− .In this subsection, we briefly analyze the algorithm without the processes of cell separation and cell communication.We use the term PHEA −,− to represent the method, which is based on PHEA but does not include membrane structure transformations.Two instances are selected from each type of benchmark problems.They are R107, R109, C109, C109, RC101, RC105, R201, R210, C201, C202, RC201, and RC208.The performances of the PHEA −,− are evaluated by using the following criteria: the best solution and the worst solution over 10 runs, the mean best solution over 10 runs, the standard deviation, and the elapsed time.Experimental results are shown in Table 4.We can see that the solutions are much worse than those solutions generated by PHEA, even though the improvement strategy and the genetic operators continue to be used.

Conclusions and Future Work
A membrane system is a good framework for approximate algorithms because of its hierarchical structure and intrinsic parallelism.As objects in membranes evolve according to specific rules, this process is similar to many types of heuristic algorithms.The rules in a membrane system can sometimes describe those algorithms.For example, if we suppose that chromosomes are represented by string objects in a membrane system, the mutation operator and the crossover operator can be described by rewriting rules.
Structure design is a very important research topic in the field of optimization algorithm.Our study focuses on the effects of algorithmic structure variation on the performance of genetic operators.In this work, the novel membrane algorithm PHEA is proposed.It contains a tissue-like membrane structure with separation rules and communication rules.Membrane structure variation is introduced in this algorithm.Several subsystems are initially produced via the separation rules.In each subsystem, chromosomes evolve through genetic operators in a parallel way.Antiport rules are used between subsystems to exchange some number of individuals.This type of rule can increase the diversity of the population in each subsystem and improve the performance of genetic operators.To study VRPTW, we also designed a special improvement strategy to speed up the search process in the subsystems.The proposed algorithm was tested using Solomon's benchmark problems.The experimental results show that our algorithm is very competitive with the classical algorithms described in the literature.
Further research may include investigations into scalable methods for large VRPTW or other types of optimization problems.The vehicle routing problem considered in our paper has one depot and hard time windows.Other types of VRPTW, such as multi-depot vehicle routing problem with time windows [46,47], vehicle routing problem with soft time windows [48,49], and vehicle routing problem with split deliveries [50], can be tested by our algorithms with some changes.On the other hand, the influence of other characters of membrane system on the performance of approximate algorithms should be considered in the future, for example, choosing rules in a nondeterministic manner.

Figure 2 :
Figure 2: Separation process of the membrane with label 2.

Figure 4 :
Figure 4: The flowchart of HEA in the operation membranes.

Figure 5 :
Figure 5: A route example made using the improvement strategy.

Example 1 .
Consider the following order crossover:

Figure 5 (
a), the round dots indicate customers, and the square represents the storage facility.In Figure5(b), a tour is constructed by three routes.Supposing that  = 4, only the longest route in Figure5(c) remains unchanged.The customers in the other routes, whose lengths are less than 4, are used to reconstruct new routes.Figure5(d)shows the new routes that were constructed by the improvement strategy.

Table 1 :
Parameters used for the PHEA.

Table 2 :
Comparison of PHEA with the best published results.
The proposed algorithmic framework was coded in C and executed on an AMD 3500+2.21GHzwith 1 GB of RAM under Windows XP.The parameters used in the proposed algorithm PHEA are shown in Table1.For each benchmark problem, 10 independent runs were carried out.
The number of runs is 10.BS, MBS, WS, STD, and ET represent the best solution, worst solution, standard deviation, and elapsed time (in seconds), respectively.