Optimizing Electric Truck Routing and Charging with Soft Time Windows Using Vehicle-to-Grid Technology

. Due to the rapid increase in the use of electric vehicles and instability in energy supply, the application of vehicle-to-grid (V2G) technology has gained attention in the freight transportation sector. V2G has the potential to increase the efciency of power grid and make additional profts by utilizing surplus power from electric vehicle batteries. Tis paper proposes an optimization model for electric trucks (ETs) to provide operational decision-making support for the freight transportation sector. Te objective of the model is to minimize the total net cost, which includes charging cost, discharging reward, and time penalties, while considering changes in ET charging cost and the system marginal price. Furthermore, we conduct sensitivity analysis in the vehicle routing problem with soft time windows using ETs in the V2G system.


Introduction
In recent years, global sales of electric vehicles (EV) have rapidly increased, and several new start-up companies have emerged to produce them.EVs are viewed as a way to reduce greenhouse gas emissions that contribute to climate change, and with fuctuating gas prices due to various factors, it is reasonable to expect that fossil fuel-powered cars will eventually be replaced by EVs in the near future.Te widespread adoption of EVs can have a substantial efect on the power system.Previous research has demonstrated that, in the absence of efective battery management systems, EVs could account for a signifcant portion of the overall demand [1], exacerbate the diferences in demand between of-peak and peak hours, and ultimately increase the need for ramping, which may impact the stability and reliability of power networks [2].However, the implementation of timeof-use (TOU) pricing policies or government incentive policies can encourage EV owners to avoid charging their vehicles during peak hours and instead utilize the batteries of their EVs as energy storage to supply power to the grid during those times.In this study, we consider that future plug-in connectors for EVs will be equipped with communication capabilities that allow for seamless communication among the battery management system, the power grid, and charging stations located at diferent places like homes and workplaces.Tis will enable the full potential of vehicle-to-grid (V2G) technology to be realized.V2G technology shows great potential as a means of utilizing EVs to store surplus energy from renewable energy sources (RESs) in their battery packs and then feed it back into the grid during peak energy demand hours.Te utilization of V2G could lead to a substantial improvement in the efciency of integrating renewable energy at a large scale.Furthermore, V2G is currently in the process of developing and approaching commercialization of ancillary services, such as grid frequency and voltage regulation management systems [3].
Compared to electric passenger cars, electric trucks (ETs) can have a more signifcant impact on reducing greenhouse gas emissions because trucks are generally larger and heavier and travel longer distances, consuming more fuel and emitting more greenhouse gases.Amazon, which is the top logistics company in the United States, is launching a new delivery service that uses electric vans [4].Additionally, the United States Post Ofce (USPS) has declared that they intend to replace 40% of newly acquired postal delivery trucks with EVs [5], and they anticipate a further surge in the use of EVs [6].One of the challenges of using ETs is that their batteries may not provide the same range as diesel trucks [7].However, advancements in battery technology with respect to charging, capacity, and cost are expected, which will make it economically viable to not rely on gas [8].Additionally, in contrast to passenger cars which are often randomly parked and without a set schedule, logistics company ETs are typically instructed to park in predetermined zones for specifed durations.Tis established operational pattern allows for accurate predictions of the amount and timing of power needed to charge electric trucks, enabling the scheduling of discharges in advance.By factoring peak and of-peak periods, the charging and discharging schedule can be optimized, leading to economic advantages and enhanced system stability.Additionally, it is worth noting that ETs generally possess larger battery capacities compared to electric passenger vehicles.As a result, the implementation of V2G technology is expected to be more efective with ETs than with electric passenger vehicles.
Employing V2G technology to sell electricity stored in the battery during peak hours and charging the battery during of-peak hours can yield signifcant economic benefts.However, logistics companies cannot rely solely on batteries to generate power for sales; they must ensure that sufcient power is supplied for transportation purposes as well.Moreover, due to the slow charging and discharging times, V2G application is limited by the requirement of sufcient time for charging and supplying power to the grid.In order to optimize the fnancial gains through V2G using ETs, it is essential to identify the appropriate truck routes that take into account both the locations of charging stations and the amount of electricity that can be supplied to the grid.In this study, we address the electric vehicle routing problem (EVRP) to maximize profts obtained from V2G operations with given delivery schedules.
Te vehicle routing problem (VRP) has been the subject of extensive research since Dantzig and Ramser [9] frst proposed the VRP as an extension of the traveling salesman problem (TSP).Te VRP involves the assignment of geographically dispersed customers with associated demands to vehicles, such that the demands are met and the total distance travelled by the feet is minimized.Each vehicle's route should begin and end at the depot.Tere are diferent types of VRPs, depending on the specifc requirements and constraints of the problem.Te common VRPs include the capacitated VRP (CVRP), VRP with time windows (VRPTW), multidepot VRP (MDVRP), and VRP with pickup and delivery (VRPPD).In the CVRP, a group of vehicles with restricted capacity is tasked with delivering goods to a predetermined set of customers while minimizing the overall distance travelled [9].Building upon the CVRP, the VRPTW incorporates a particular time window during which each customer can be attended to [10].Similarly, the MDVRP considers the locations of the customers in relation to their assigned depot from which the vehicles must deliver goods [11].Additionally, the vehicle routing problem with pickup and delivery (VRPPD) takes into account the vehicles that must collect goods from certain customers and transport them to other customers [12].Te main focus of our study is on VRP with soft time window (VRPSTW) [13], a variation of VRPTW that allows for the violation of time window restrictions at a certain cost in penalties.
In contrast to using fossil-fueled vehicles in the conventional VRP, EVRP faces certain limitations related to the batteries [14].In recent works, Erdem [15] presents a heuristic strategy to create an efective decision-making framework for routing electric trucks, handling the transportation of milk with diverse qualities from producers.Meanwhile, Amiri et al. [16] explore a biobjective programming model focus on simultaneously minimizing transportation expenses and greenhouse gas (GHG) emissions.Tey consider a combination of ET and diesel trucks.Teir computational results reveal that a slight increase in transportation costs leads to a signifcant reduction in GHG emissions.While the majority of related studies concentrate on identifying the best alternative for a particular setting, the primary objective of our research is to explore the economic advantages and disadvantages of V2G technology when applied to ETs.We believe that the fndings could ofer valuable insights for the formulation of economic policies aimed at fostering the integration of V2G technology in ETs.
In a recent comprehensive review paper, Sabet and Farooq [17] conducted a thorough evaluation of multiple adaptations and specifc instances of the green vehicle routing problem (GVRP) aimed at resolving challenges concerning charging, pickup, delivery, and energy usage.Teir analysis highlighted the prevalence of metaheuristic techniques in most studies, with limited attention given to emerging machine learning approaches.Tey suggest a future direction wherein not only machine learning but also reinforcement learning [18], distributed systems, the Internet of Vehicles (IoV), and innovative fuel technologies play signifcant roles in advancing GVRP research.Furthermore, to enhance the economic viability of electric vehicles, a strategy involving the utilization of V2G technology can be examined as a means of generating additional revenue [19].Te key contributions of this research include the following: (1) developing an optimization model for EVRPs considering V2G operations; (2) providing operational decision-making support for the logistics feld in order to increase revenue in a system of V2G; and (3) analyzing changes in various environmental factors related to the V2G technology.Te remaining part of the paper proceeds as follows.In Section 2, we frst defne our problem with the general assumptions.Next, a new mixed integer programming (MIP) model is formulated with the objective of minimizing total costs.Section 3 introduces charging and discharging rate system in South Korea.Also, we perform two simulations with the proposed model to explore the possibilities of applying V2G technology to the logistics feld.Lastly, Section 4 provides conclusions and future research topics.

Model Presentation
In this section, we present the electric vehicle routing problem with soft time windows (EVRPSTW) in a system of V2G.Te objective of the problem is to fnd a delivery route such that a set of customers are served for a given time frame and the total net cost including charging cost, discharging reward, and time penalty is minimized.Te following assumptions are considered: (a) A single depot dispatches ETs to handle a set of customers (b) ETs can be charged and discharged only at depot in time intervals (c) ETs can wait for charging or discharging at the depot when the charging cost is high or the discharging reward is low (d) ETs can be discharged as much as the remaining battery level after the completion of scheduled delivery, and they are fully charged at the lowest electricity rate for next-day delivery (e) Hourly charging cost, hourly discharging reward, and delivery information including customer locations and time windows are given in advance (f ) Customers can be served early or late with some penalty cost (g) Te energy consumption rate is constant (h) Te charging and discharging rate are the same (i) Each ET has a limited battery capacity and the time required to fully charge the battery is the same Considering a time frame T, we examine EVRPSTW within the context of V2G operations during a discretized time interval t on a complete directed graph G � (V 0,N+1 , E), is the set of edges.Te nodes include three types of nodes, i.e., v 0 � v N+1 is the depot node, D is the set of dummy nodes for charging and discharging at depot, and V � v 1 , . . ., v N   is the set of customers.For convenience, we introduce two additional node sets, V 0 and V N+1 , in our formulation.V 0 is the set of nodes including customers, dummy nodes, and depot 0. Similarly, V N+1 is the set of nodes including customers, dummy nodes, and depot N + 1.Each customer has a service time s i , a time window [e i , l i ], and a soft time window [e i ′ , l i ′ ].For any given (v i , v j ) ∈ E, Assumption (f ) implies that travel time between nodes v i and v j is t ij and the charging time required for travel is f ij .Lastly, from Assumptions (g) and (h), if the K homogenous ETs are charged/discharged δ period per charging/discharging unit, C is denoted as time intervals to fully charge the battery from 0, then the battery capacity is C/δ.Note that ETs wait until the start of next time interval before charging and discharging begin at depot because the time interval is discretized as |T|.
For a given time frame T, we propose a MIP model that determines the optimal charging, discharging, and delivery times for a feet of K ETs to minimize the total cost including charging cost, discharging reward, and time penalty.Te mathematical model considers the following parameters and decision variables: Parameters: Journal of Advanced Transportation x ijk , y ik , α ikt , Te objective of the proposed model, as stated in (1), is to minimize the total cost which consists of four parts; the frst and second terms are the charging cost and discharging reward, respectively, the third and fourth terms are the time window penalty, and the last term is the fully charging cost at the remaining battery level after delivery is completed.Te set of constraints (2)-( 5) guarantee that each ET starts for delivery at depot v 0 , serves one customer once, stops by dummy nodes if necessary, and then comes back to depot v N+1 .Delivery and service time are calculated in the set of 4 Journal of Advanced Transportation constraint (6) when ETs are neither charged nor discharged.Te set of constraints ( 7) and ( 8) ensure that ETs can be only either charged or discharged at depot during delivery.If they are either charged or discharged, the corresponding time is calculated by the set of constraints ( 9) and (10).Te set of constraints ( 11)-( 13) consider soft time windows for customers.Te set of constraints ( 14)- (19) consider ETs' battery level.First, after ETs serve customer v i , they can reach other nodes v j located within the charging time required to travel between v i and v j , i.e., f ij , in the set of constraints (14).Next, if ETs arrive at depot, the set of constraints (15) deal with ETs' charging and discharging options.Also, the set of constraints ( 16)-( 18) ensure that ETs can be charged or discharged within the battery capacity, the remaining battery level, or the battery level above 0. Lastly, all ETs are dispatched at fully charged battery level at depot in the set of constraints (19).Te set of constraints ( 20) and ( 21) defne the domains of the decision variables.

Simulation Results
In this section, two simulations are performed using the proposed model to explore the benefts of implementing V2G technology in the logistics industry and to assess the efectiveness of V2G in a changing environment in the future.Te frst simulation involves comparing a V2G system that enables charging and discharging ETs with a general ET delivery system without V2G technology under dynamic electricity prices.In the second simulation, three sensitivity analyses are performed to examine the impact of charging cost, discharging reward, and battery capacity on the objective function of the proposed model in the V2G system.
To investigate the practicality and advantages of utilizing ETs for ancillary services in V2G systems, we perform simulations using the Porter II Electric, a commercial ET ofered by Hyundai Motors [20].Tis ET features a battery capacity of around 60 kWh and has a range of up to 211 km on a single charge.With a 100-kW fast charger, it takes approximately one hour to reach full charge, though this may vary depending on the battery's condition and the type of charging used.For simplicity in the simulation, we assume that the discharge rate is equal to the charge rate, as no ofcial information is provided on this matter.Additionally, we suppose that a logistics company operates two ETs, using them for less than 200 hours per month.As per Table 1, we employ the high-voltage electric vehicle charging cost for service providers, which was announced by KEPCO in July 2022 [21].Finally, if we assume that the logistics company sells electricity to KEPCO, we can estimate the electric discharging reward based on the system marginal price data provided by the Electric Power Statistics Information System [22].As the monthly price fuctuation is signifcant, as illustrated in Figure 1, we consider fve scenarios in our simulations that capture the hourly price trends: (1) January and February, (2) March, April, and May, (3) June, July, and August, (4) September and October, and (5) November and December.
Due to the NP-hard nature of our problem, the computational time required increases exponentially with the number of nodes, including the depot, customers, and dummy nodes.We execute all scenarios on a PC with an Intel Xeon Gold 3.1 GHz processor and 128 GB RAM.Our simulations are implemented in Python 3.8 using IBM ILOG CPLEX 20.1 with the default options for MIP problems to solve the problems.Note that the CPLEX solver is an optimization software package provided by IBM ILOG [23].Given the complexity of the problem, the CPLEX solver is unable to fnd optimal solutions within the provided 3,600second time window, so we ofer near-optimal feasible solutions instead.Te locations of a single depot and ffteen customers are depicted in Figure 2, where the expected delivery time within a 24-hour time window is denoted by the square brackets next to the customers.2 presents the result of comparing the total net costs with and without the V2G system.Note that the term "discharging reward" refers to the proft earned by selling electricity, while "night cost" denotes the full charging cost at the lowest rate after all assigned deliveries are completed.Five scenarios were executed using two ETs to process the same customer delivery requests.Te highest total net cost was KRW 9,882.3(about USD 7.41) in January and February, while the lowest was KRW 5,018.5 (about USD 3.76) in September and October.Note that the current exchange rate is about 1 KRW � 0.00075 USD.Except for January and February, the V2G system typically results in lower overall operating costs.Our study does not consider the reduced battery lifespan that can result from frequent charging and discharging.Terefore, in the case of March to August, where the relative diference is small, it is challenging to assert that the V2G system's impact is signifcant when factoring in the ET's battery replacement cost.Moreover, the diference between the charging cost and the discharging reward has a considerable efect on the benefts of the V2G system.From November to February, the charging cost remains unchanged.However, due to the relatively higher system marginal price in November and December than in January and February, ETs are frequently discharged during delivery, demonstrating the signifcant beneft of the V2G system.Lastly, since the night cost is comparable in all scenarios, this implies that the battery is charged and discharged as frequently as possible during delivery, and full charging is completed after delivery.If the battery capacity is increased, this result is expected to further enhance the V2G system's advantage.We will verify this assumption in the subsequent simulation.

Comparison of V2G and Non-V2G Systems. Table
In Figure 3, two delivery routes are displayed for November and December that show the largest cost diference between the V2G and non-V2G systems.Notably, the delivery routes are completely distinct from one another.Although the ETs violate the customer's time windows, they are repeatedly charged and discharged at the depot to sell electricity under the V2G system.If the battery capacity was increased, it is anticipated that the V2G system's advantage Journal of Advanced Transportation would grow since the ETs would save travel time between the depot and customers.We will verify this expectation in the next simulations.Additionally, the delivery route of the V2G system is much more complex than that of the non-V2G system.Terefore, since our EVRPSTW model in V2G presents a highly complex problem, we ofer near-optimal feasible solutions within a 3,600-second time window.It is believed that more efcient delivery routes and charging/ discharging schedules exist.

Sensitivity Analysis.
Te V2G system's advantage varies depending on the diference between charging cost and discharging reward, with some cases providing signifcant advantages while others ofer none at all.Additionally, battery capacity plays a crucial role in determining the total net cost when there is a signifcant diference between charging cost and discharging reward.Tus, the subsequent simulations aim to investigate the impact of battery capacity, charging cost, and discharging reward on the total net cost.
Figure 4 illustrates how increasing battery capacity can have a signifcant impact on the total net cost, except for January and February when the V2G system provides no advantage.Te doubling of battery capacity signifcantly reduces the total net cost for logistics companies, allowing them to earn profts through the V2G system.For example, the V2G system can earn about KRW 9,500 in September and October, as the large battery capacity is efciently utilized to charge and discharge in the substantial diference between charging cost and discharging reward.
Furthermore, the efect of varying charging cost on the total net cost is presented in Figure 5.As the charging cost increases, the total net cost also increases overall.A signifcant change in the total net cost is observed in September to December, where frequent charging and discharging occur in the V2G system.Tus, an electricity usage-based discount contract could be considered to reduce charging costs and overcome this issue.
Next, Figure 6 demonstrates the impact of varying discharging reward on the total net cost.When the discharging reward increases by 10% in January and February, the total net cost decreases slightly since one of the ETs sells electricity at the beginning of delivery.On the other hand, as shown in Table 1, the charging cost is highest from July to August.Consequently, the more discharging occurs, the   Jan, Feb more charging takes place, leading to a signifcant increase in the charging cost.As a result, the total net cost slightly rises when the discharging reward increases by 10% in July to August.Te months of September and October, which exhibit a more signifcant diference between charging cost and discharging reward, are the ones that beneft the most from the variation in discharging reward.Terefore, it can be concluded that the diference between charging cost and discharging reward is one of the most important factors for logistics companies to consider when utilizing the V2G system.Lastly, Figure 7 describes how the total net cost changes as the charging cost changes given a discharging reward.We conduct each simulation with one charging cost, increased by 20 from 60 to 120, and one discharging reward, increased by 20 from 80 to 140, in a 24-hour time window.First, as the charging cost increases, the total net cost increases gradually because ETs should be fully charged with the high charging cost at night when all deliveries are completed.Note that the total net cost does not increase consistently due to the consideration of delivery routes and time penalties.Furthermore, when the charging cost is low, the V2G system provides great advantages, especially given a high discharging reward.However, the total costs converge around KRW 21,000 as the charging cost increases.In other words, as the charging cost rises, the total net cost increases gradually.When the charging cost is expensive, there is no signifcant diference in the total net cost, regardless of whether the discharging reward is high or low.Note that when the charging cost is 80, the reason why the total net cost of the discharging reward 100 is slightly higher than that of the reward 80 is because of the discretized time problem and near-optimal feasible solutions.Consequently, to obtain the benefts ofered by the V2G system, the crucial factor is to achieve economical charging during nighttime while also considering the potential for optimizing the discharge schedule to maximize rewards.

Conclusion
With the rise in sales of electric vehicles and growing interest in V2G systems, companies involved in transportation and logistics that operate a feet of ETs now have the potential to boost their profts by utilizing the batteries of these vehicles    for V2G systems.In this study, we explore the possible uses of V2G systems within logistics companies that operate ETs.Managing batteries efciently is crucial in a logistics company's V2G system to ensure that it does not interfere with the primary goal of transportation.Terefore, companies using ETs can reap additional fnancial benefts by utilizing their batteries for V2G systems to store excess energy during periods of low demand, which can be sold for a proft during periods of high demand.Tis study involves developing an optimization model for EVRPs that integrates V2G operations, which is intended to ofer decision-making support to logistics operations and enhance revenues in V2G systems.
As evidenced by the outcomes of our simulation, the benefts arising from employing V2G systems are subject to the infuence of various factors such as battery capacity, charging cost, and discharging cost.To stimulate the acceptance of V2G systems within the logistics sector, a range of subsidy policies may be considered, particularly based on electricity trading rates.Additionally, we conduct a sensitivity analysis to explore the efects of diferent environmental factors associated with V2G systems.
Based on the simulation results, we suggest the following approaches for efectively commercializing ETs that incorporate V2G technology in the logistics industry.Firstly, although strategic planning contributes to minimizing travel time between customers and depots, considering the advantages of V2G systems can change the routes of ETs to increase profts by trading stored electricity within the ETs.As battery technology improves and its capacity grows, there will be less need to travel for charging and discharging.However, it may take some time for battery technology to mature enough to increase battery capacity.To address this, we suggest that logistics companies set up additional charging and discharging facilities close to major customer locations to minimize wasted travel time and maximize long-term benefts for V2G technology in the logistics feld.In downtown areas where space is scarce, companies can consider sharing their facilities with customers.Secondly, operational planning can leverage hourly diferences between charging costs and discharging rewards.Based on simulation results from the frst (January and February) and ffth (November and December) scenarios, we found that pricing diferences play a signifcant role in determining whether discharging occurs.To take advantage of this, time windows for customers can be adjusted so that ETs are charged when charging costs are low, and ETs are discharged when discharging rewards are high.Tis operational approach can help optimize the use of V2G technology in the logistics feld.
Our current development has some limitations which present opportunities for interesting future research.Te primary limitation is the computational burden that restricts the efectiveness of our approach in solving large-scale problems.Due to the complexity of the EVRPSTW problem in a system of V2G, the CPLEX solver cannot fnd optimal solutions within a time of 3,600 seconds.To overcome this limitation, Markov decision processes (MDP) can be considered to model the operations of ETs' batteries instead of relying on the MIP model for fnding optimal routes.Lin et al.
provide [24] an end-to-end deep reinforcement learning (RL) framework to solve the EVRPSTW problem without the context of V2G operations and show that their model can efciently solve larger problems.Approximation methods for MDP problems such as RL techniques can be applied to address the EVRP with V2G operations.Another limitation of our approach is that it does not consider the fuctuation of electricity and delivery demands.We assume that the power supplied by ETs is incapable of signifcantly impacting demand; consequently, all power generated by ETs is accepted.However, energy storage systems (ESS) are utilized to store energy during low-demand periods and supply energy during peak times, while logistics companies often need to transport goods during peak times.Terefore, uncertain delivery demand could afect their profts.For a more realistic V2G environment, we need to incorporate additional restrictions into our model.
Lastly, it is worthwhile to note that V2G systems serve not only economic purposes but also play a critical role in stabilizing load fuctuations.With the rise in renewable energygenerated power, managing the disparity between power supply and demand becomes imperative.In this context, V2G stands as a potential solution for mitigating this imbalance.Terefore, to address the diverse objectives of V2G systems, it is possible to consider the demand leveling efect in addition to cost minimization.Furthermore, when environmental concerns take priority over economic gains, proftability might not be the primary focus.In such cases, to meet environmental objectives, strategies such as subsidy policies may need to be employed to steer the goal, and the economic optimization methodology employed in this study can serve as a benchmark to aid in formulating subsidy policies.

Figure 6 :
Figure 6: Comparison of the total cost when discharging reward is changed.

Figure 7 :
Figure 7: Relationship between the total net cost and charging costs when a discharging reward is given (unit: KRW).

Figure 5 :
Figure 5: Comparison of the total cost when charging cost is changed.
Earliness penalty per one unit of time p l Tardiness penalty per one unit of time Decision variables: x ijk Binary decision variable indicating if edge (v i , v j ) is travelled by ET k y ik Binary decision variable indicating if ET k is charged (�1) or discharged (�0) at node v i α ikt Binary decision variable indicating if ET k is charged at node v i in time interval t β ikt Binary decision variable indicating if ET k is discharged at node v i in time interval t τ ik Continuous decision variable specifying the arrival time of ET k at node v i τ e ik Continuous decision variable specifying the earliest arrival time of ET k at node v i τ l ik Continuous decision variable specifying the tardiness arrival time of ET k at node v i b ik Continuous decision variable specifying the time to drive with the remaining battery level on arrival at node v i

Table 1 :
ET charging cost by KEPCO as of 1st of July 2022 (unit: KRW/kWh).

Table 2 :
Comparison of charging cost, discharging reward, time penalty, night cost, and total cost in fve scenarios (unit: KRW).MonthCharging cost Discharging reward Time penalty Night cost Total net cost Relative diference (%)