Deep Learning-Based Service Scheduling Mechanism for GreenRSUs in the IoVs

Green roadside units (RSUs), also called renewable energy-powered RSUs, are utilized recently rather than the traditional electricpowered RSUs with high power consumption and the large infrastructure deployment cost in the Internet of vehicles (IoVs). However, the power of the green RSUs is limited and unstable, which is affected by the battery size and charging environment. Therefore, a big challenge to deploy green RSUs in the IoVs is to schedule their service process properly, in order to extend the service efficiency of RSUs. In this paper, a deep learning-based communication scheduling mechanism is proposed regarding the service scheduling problem. In particular, a three-part scheduling algorithm consisting of RSU clustering, deep learning-based traffic prediction, and a vehicle access scheduling algorithm is presented to maximize the service number of vehicles and minimize the energy cost. An extensive simulation is done, and the simulation results indicate that our algorithm can serve more vehicles with less energy consumption compared with other scheduling mechanisms under different scenarios.


Introduction
The Internet of vehicles (IoVs) is composed of lots of vehicles and RSUs. These vehicles are equipped with on-board units (OBUs) including a global positioning system (GPS) and the RSUs are connected to the centralized network or service providers [1]. In the IoVs, each vehicle can communicate with each other at the ad hoc style with the DSRC technique or Pc5 interface of LTE-V. This kind of communication is referred as vehicle-to-vehicle (V2V) communication. Besides, these vehicles can also communicate with RSUs with the interface of LTE-V [2] and this kind of communication is commonly referred to vehicle-to-infrastructure (V2I) communication. Based on the two styles of communication, most of services in the IoVs can be carried out, such as location services [3], blockchain services [4], social services [5], and crowd sensing services [6].
In the IoVs, the RSUs play important roles. They not only extend the service range of the IoVs but also provide high-speed message forwarding services for vehicles with high bandwidth links to the service providers [7]. However, the forwarding services require high power consumption [8]. In addition, the RSUs are powered by physical infrastructures linked to the electrical grid, which costs too much. Especially in some rural areas, the cost of deploying electrical facilities for RSUs may be much higher. At last, traditional electric-powered RSUs cause carbon dioxide emissions indirectly. Therefore, the requirement of renewable power, such as being polar and windy, to be integrated into RSUs instead of the traditional powered functions is proposed by several papers [9].
However, there are some critical problems to be solved while deploying the renewable energy-powered RSUs. First, the power on the RSU is limited due to the battery size and charging environment. In addition, the power consumption of communication is influenced by the communication time, communication frequency, communication distance, and channel quality. Without a proper communication scheduling strategy, the power may be exhausted quickly, which will have a serious impact on the normal development of services in the IoVs. Thus, this paper focuses on how to maximize the performance of renewable energypowered RSUs in order to serve as many vehicles as possible with less energy consumption.
Motivated by this, a deep learning-based communication scheduling mechanism is proposed in this paper, composed of three parts. First, a clustering method named t-SNE is utilized to classify all the RSUs to several clusters, each of which has a special evolving rule on traffic flows. Second, we use a deep learning-based algorithm to predict the future traffic flow for each RSU cluster, with the history knowledge of the traffic status under each RSU. At last, once the prediction results are obtained, a service ability scheduling algorithm and a priority-based RSU access algorithm are proposed to schedule the vehicle access order. The contributions of this paper are outlined as follows: (1) A clustering method t-SNE is utilized, based on which the future traffic flow for each RSU can be a prediction with a deep learning algorithm (2) Based on the future traffic prediction, we propose a service ability scheduling algorithm and a prioritybased RSU access algorithm to maximize the RSU service efficiency (3) An extensive simulation is deployed to demonstrate that the proposed algorithm can achieve higher efficiency with less energy cost compared with other scheduling mechanisms under different scenarios The rest of the paper is organized as follows. Section Related Work summarizes the related work. Section System Model presents the system model and problem formulation. Section Deep Learning-Based Communication Scheduling Algorithm details the deep learning-based communication scheduling mechanism. And section Simulations presents the simulation results. Lastly, section Discussion and Conclusion draws the conclusion.

Related Work
It is necessary to reduce energy consumption as much as possible for RSUs in the IoVs, and several papers have done some work referring to the grid-powered RSUs. In [10], the authors focus on the hardware design of RSUs and propose a centralized control module to schedule the energy cost on RSUs. Their idea is based on the artificial neural network and is composed of three algorithms to achieve the purpose of the green scheduler. In [11], the authors incorporate a sleep mechanism on grid-powered roadside units to further reduce power consumption. However, it is not fit for the renewable energy-powered RSUs. The off-grid windpowered RSUs are considered in [12]. Varied from conven-tional reliability analysis, the reliability in this paper is redefined in a highway environment. Then, the energy models are developed and the minimum battery size can be determined when a certain reliability standard and quality of service are achieved.
Due to the high cost of grid-powered RSUs, the renewable energy-powered RSUs are proposed. As the energy on this kind of RSUs is limited, how to minimize the energy consumption is also a wide concern. An energy-efficient scheduling framework is proposed in [13]. The task scheduling and energy consumption are considered jointly in the proposed heuristic algorithm. To minimize energy consumption and satisfy task latency constrains, an imitation learning-enabled online task scheduling algorithm is proposed in [14]. In [15], it is assumed that any vehicle's position in the network is strongly deterministic. Then, three energy-efficient online traffic scheduling algorithms are introduced to minimize the long-term power consumption subject to the communication requests associated with the passing vehicles.
The scheduling problem in renewable energy-powered RSUs is also a hot research topic. A joint scheduling and power control scheme is proposed in [16]; it is formulated as a mixed-integer nonlinear programming (MINLP) problem. Paper [17] investigates the problem of scheduling the downlink communication for renewable energy-powered RSUs toward vehicles, with the objective of maximizing the number of served vehicles. In [18], the authors propose low-complexity algorithms for downlink traffic scheduling in green vehicular roadside infrastructure. However, the algorithms require some priori information, which may not be always available in the IoVs. In [19], a reinforcement learning technique for optimizing downlink scheduling is proposed in an energy-limited vehicular network. Its objective is to equip RSUs with the required artificial intelligence to realize an optimal scheduling policy that will guarantee the operation of the vehicular network during the discharge cycle while fulfilling the largest number of service requests. However, works [17,19] just simulate the scenario in the simulator without testing the algorithms with vehicle trajectories of reality.
Moreover, some works have been done from the perspective of services to save energy of green RSUs. In a green communication scenario, the safety and QoS are both a concern in [20]. A deep reinforcement learning model named deep Q-network is proposed, which learns an energyefficient scheduling policy. With the policy, the battery of an RSU is extended and the safety of environment is promoted while the quality of service (QoS) levels is met. Aiming at the problem of power deficiency in solar-powered roadside units (SRSUs), the challenge of QoS loss is addressed in [21], in which a two-phase approach is proposed. With the purpose of energy consumption and time delay guarantee, a distributed packet scheduling optimization strategy is proposed in [22] for the renewable energypowered RSUs. Based on the proposed strategy, the system energy consumption can be minimized and the delay of the system can also be reduced, in which an optimization model is established based on the Lyapunov theory. To solve 2 Wireless Communications and Mobile Computing the problems that existed in battery-enabled RSUs and electric vehicles (EVs), an intelligent energy-harvesting framework is constructed in [23], where RSUs and EVs are integrated. Based on a three-stage Stackelberg game developed by the authors, the utilities of RSUs and EVs are maximized.

Scenario Description.
We consider an Internet of vehicles scenario, shown as Figure 1. In the scenario, all the vehicles are equipped with a DSRC or LTE-V communication module, with which each vehicle can interact with the ITS service center. Besides, any vehicle accessing to the centralized network must connect to an RSU with its communication module. Due to consideration of energy saving, more and more RSUs are powered by renewable energy such as solar energy. Thus, the energy on RSUs is limited and each RSU can only serve finite number of vehicles. Thus, in this paper, we focus on the communication scheduling mechanism of RSUs to maximize the number of service vehicles.

Problem
Formulation. Based on the above scenario, we construct the system model as an undirected graph GðR, E, VÞ, shown in Figure 2. Figure 2 is a formalized definition of the described scenario. RSUs in Figure 1 are mapped to the vertexes in Figure 2, and the communication links among RSUs are described as the edges between vertexes. Graph G denotes the Internet of vehicles scenario and the details of the network are described as follows: (1) R is the set of RSUs, denoted as R = fr 1 , r 2 , ⋯, r n , ⋯, r M g. In the network, there areM RSUs and everything can be accessed by vehicles passing by. With the assumption that each RSU is powered by renewable energy, the communication bandwidth of each

Wireless Communications and Mobile Computing
RSU can be tuned with time. In this paper, S r n is the service ability of RSU r n ðr n ∈ RÞ and its value is determined by the energy charged, with max r n being the maximum value of S r n determined by the battery size. Here, we assume that S r n is an integer and it denotes the number of vehicles that r n can server. For any RSU r n , each communication with a vehicle will cost some service ability of r n , which is assumed to be 1 in this paper. If the time period is considered, then S r n can be rewritten as S Δt r n , which means the service ability of r n at the time period Δt. Besides, in order to maintain the service ability at a time period, some energy still will be costly. In other words, if there is no vehicle accessing, the RSU still spends service ability on maintaining the communication service. Here, another assumption is made that if no vehicle uses the allocated service ability in the time period, the service ability will be wasted to maintain the service, and the amount of service ability is the same as the communication cost to vehicles (2) An edge is denoted as E ij ðE ij ∈ EÞ that connects two RSUs r i and r j . It means that r i and r j are adjacent RSUs and a vehicle can move from the coverage range of r i to that of r j For any vehicle v i ∈ V, it moves from the coverage of one RSU to another RSU, and at any time, it can only communicate with no more than one RSU. The trajectory of vehicle v i is L v i , which is a location sequence, such as ðr In this scenario, we focus on the communication scheduling problem on each RSU to optimize the network. The objective is to find a proper combination S v i r n ðΔt x Þ for each RSU and vehicle for each time period that maximizes the service time. The problem is formulated as follows.
Maximize service rate = serviced vehicles subject to Formula (1) means that given a proper service scheduling set for each RSU and vehicle, with the limited energy in RSUs, maximize the successful communication rate between vehicles and RSUs. Formula (2) is used to maximize the ratio between the number of vehicles that can communicate with RSUs and the total number of vehicles. Constraint

Wireless Communications and Mobile Computing
(3) ensures that for each RSU, the energy cost for its communication with vehicles should be no more than the energy it is charged. Constraint (4) ensures if an RSU can communicate with a vehicle; the vehicle should be in the RSU's coverage range.

Main Idea.
In order to maximize the service number of vehicles and the throughput, all the vehicles in the network should be connected to an RSU at any time, and then, the service number is maximum intuitively. Thus, we should figure out the number of vehicles in the network, based on which the total energy required to satisfy the communication between all vehicles and RSUs can be calculated. As the energy cost of each RSU is relative to the number of vehicles covered by each RSU, the first step is to predict the number of vehicles passed by each RSU. Based on this, the required energy of each RSU can be obtained. However, the amounts of energy in each RSU are not the same, which are determined by surrounding environments. When the energy of an RSU cannot afford the energy cost on the vehicles passed by, a scheduling algorithm should be designed. A reasonable communication scheduling algorithm should be designed to save energy or reduce energy waste, especially when the energy of an RSU cannot afford the energy cost on communications. The main idea of the algorithm is to tune energy cost over all time periods and then to select more vehicles to access.

Deep Learning-Based Communication
Scheduling Algorithm

Deep Learning-Based Traffic Prediction
Mechanism. As Section 3 shows, the energy on each RSU is limited and each time of communication between an RSU and a vehicle costs energy. In this subsection, our concern is on how to predict the future traffic according to the current traffic status for each RSU. Here, we use the LSTM model to predict future traffic. Compared with the RNN model, LSTM can tackle a longer sequence better to avoid the affection of short memory. In order to make the traffic data fit the LSTM model, the traffic data should be preprocessed first. However, due to the sparsity of the training data of each RSU and the similarity of traffic-evolving rules on many RSUs, we cluster RSUs to several classes using the clustering method. At last, we use the LSTM model to train the traffic-evolving data in different classes to get a prediction model for each cluster of RSUs. Next, we will introduce the deep learning-based traffic prediction mechanism in the following parts.   There are three advantages for RSU clustering. First, the amount of RSUs in a city is usually linearly related to the size of the city and there may be thousands of RSUs in a city. If we train a model for each RSU, it will cost too much computing resource, which is also not necessary. In addition, it is found that traffic flows on many RSUs reveal some similar evolving patterns, which can be tackled together. At last, it can solve the data sparsity problem as there may not be enough data to be trained for some RSUs in practice. When some RSUs are clustered into one class, the data on this class of RSUs can be shared together, so that the training set for this cluster of RSUs is extended.
After data preprocessing, the evolving rules for a cluster of RSUs are denoted as a sequence ð N Δt 1 , N Δt 2 , N Δt 3 ,⋯, N Δt x Þ . Here, this sequence is used as the feature vector for model training. As mentioned before, the time slot Δt can be chosen as a quarter, an hour, or longer. In this paper, it is needed to catch the evolving rule from a microscopic time perspective but the length of the sequence cannot be large considering the computing consumption. Besides, there should be obvious up or down trending for the sequence value; rather than that, most of the values in the sequence are closed to zero. Thus, Δt is set to a quarter, so that the sequence length, namely, the number of dimensions of the feature vector, is 60/15 × 24 = 96.
So far, each RSU has a special feature vector, which will reveal individual characteristic. Next, all these feature vectors are used as the input source to cluster all the RSUs. But, it is hard to get a good result if we directly cluster such high-dimensional feature vectors with general clustering methods [24]. In order to solve the problem of high-dimensional data clustering, many approaches have been proposed. Clustering with a subset and fuzzy clustering are typical representatives of these methods [25,26]. Moreover, the deep learning-based clustering algorithm is also used to cluster high-dimension data, in which a deep learning strategy is used to learn low-dimensional representation of high-dimensional data. Then, the ordinary clustering algorithm can be used after the lowdimensional representation is obtained. Although a better clustering effect can be obtained using these clustering methods, they cluster high-dimensional data in an abstract way and there are still challenges in data visualization and clustering rationality.
T-distributed stochastic neighbour embedding (t-SNE) is a new dimension reduction and visualization technique for high-dimension data [27], which is used to reduce the dimensionality of each RSU's traffic flow so that all RSUs can be clustered well. T-SNE is developed from SNE that has a "crowding problem." Different from SNE, the Euclidean distance is converted to joint probability to express the similarity between data points in t-SNE (Euclidean distance is converted to conditional probability in SNE). As shown in equations (6) and (7), p ij and q ij are the joint probability of the original high-dimensional data and low-dimensional data after dimensionality reduction, respectively.
The difference in the joint probability distribution of the original data and the data after dimensionality reduction will be very small, even p ij = q ij . Based on this idea, Kullback-Leibler (KL) divergence is used to measure the difference between two distributions, shown in equation (8), where P is the joint probability distribution in the high-dimensional space (the original data), Q the joint probability distribution in the low-dimensional space (the data after dimensionality reduction), and C the cost or the sum of KL divergence difference value between two distributions. The smaller C is, the better the dimensionality reduction effect is.
As shown in equation (9), for the purpose of minimizing C or the sum of the KL divergence, the gradient descent method is used so that a better dimensionality reduction effect can be obtained. Finally, the desired lowdimensional data will be obtained by continuous iterative solutions.
By using t-SNE, the low-dimensional representation of the traffic flow in each RSU can be obtained. Then, the Kmeans algorithm can be used for clustering RSUs, in order to determine a proper K value. The elbow method is used to fix the best number of clusters K.
Its main idea is to choose the optimal K according to the varying trend of the sum of squared errors (SSE) in a cluster. SSE is defined as equation (10).
Here, C i is the ith cluster, p is the sample vector of C i , and m i is the center of cluster C i .When K is smaller than the actual cluster number, the value of SSE descends fast with K increasing. However, when a proper K is found, the value of SSE tends to be stable with K growing. Therefore, the inflection point is the best K value.

Deep Learning-Based Traffic Prediction.
Once all the RSUs are clustered, the future traffic flow should be predicted based on the history data of RSUs. As real-time prediction is hard to achieve, we use the LSTM model to train the dataset offline.
First in the learning stage, we use the vectors from each RSU class as the training dataset individually. However, if the 96dimension features of the vector are used as the training set, it will cost too much computing resource. Intuitively, the current traffic conditions are usually affected by the last few hours of traffic conditions. Thus, we use a method called the sliding window with size n to cut off the feature vector. So, in the learning stage, the consecutive n elements in a vector compose the feature sequence and the following element is the tag.
In the prediction stage, the consecutive n elements in a vector are used to predict the following element, shown as the left part in Figure 4. Another problem is how to predict the first n elements. As the traffic flow is changing periodically, the vector can be transformed to a clock-like vector, shown as the right part in Figure 4. For example, to predict the first element, we can use the last n elements of the vector as input. There are two parts in the RSU communication scheduling algorithm, namely, service ability distribution algorithm and the priority-based RSU access algorithm. When a new period starts, such as end of daytime, the energy of each RSU has already been charged during daytime. From then on, the energy is continually discharging until the next period. We assume that the energy charged during daytime will be used in the next period, rather than in the current period. Therefore, the energy should support the RSU to work for the entire period. Without a proper scheduling, the energy might be exhausted quickly, so that the vehicles traveling during the end of the period may not be served no matter its priority.
Therefore, we propose the service ability distribution algorithm to schedule the energy distribution for each RSU. The main idea of the Algorithm 1 is sketched below. First, when the energy harvested during daytime is determined, the total service ability in this period of the RSU will  9 Wireless Communications and Mobile Computing be determined, too. Next, we will assign the total service ability to each time slot according to the predicted number of vehicles in each time slot. In order to predict the number of vehicles, we use the above traffic prediction algorithm to get the traffic flow from the following slot to the ending slot of the period. Then, the service ability for each time slot is assigned according to the total service ability multiplied by the ratio of the current slot traffic flow to the total traffic flow 10 Wireless Communications and Mobile Computing of the period. Once the service ability on each time slot for an RSU is assigned, it is necessary to determine the service vehicles and the service order.
Here, we propose a conception of service priority, defined as formula (11) (where we assume that the destination of each vehicle is known).
It means that the longer the time that the vehicle has travelled, the higher the priority it has, because it may have more data to submit. Besides, if a vehicle has a long way to its destination, it may have more chances to communicate with RSUs, so that the priority is small. Here, the distance can be Euclidean distance or Manhattan distance. Based on this algorithm, when several vehicles are in the coverage of an RSU. The RSU calculates the priorities of all vehicles and provides service in the descending order of the priority, shown as Algorithm 2.

Simulation Scenario.
We evaluate our mechanism in a scenario with real vehicle trajectories, which come from the taxis in Shenzhen city, and a snapshot of the taxi location at a time point is shown in Figure 5. In this scenario, we assume that all the RSUs are uniformly distributed with the coverage range of 1 km × 1 km. Then, it can be considered that any area in Shenzhen is covered by an RSU. The parameters of the scenario are shown in Table 1.
The simulation is composed of three parts, namely, RSU clustering, traffic flow prediction, and RSU communication scheduling process. First, in the RSU clustering process, a day is divided to 96 time slots with each 15 minutes, so the clustering is based on the dataset of 96-dimensional vectors as we introduced in IV-A2. Then, the best K value is chosen by comparing varying SSE values, and in the following part, this value is used to deploy the simulation. Third, the trajectory dataset lasting for seven days is selected as the training dataset and the dataset for the other day is the testing set. Fourth, we choose different lengths of feature vectors to train the models in order to compare the accuracy of prediction.
At last, in the RSU communication scheduling process, we compare the number of successfully accessed vehicles between our scheduling algorithm and other scheduling mechanisms when the energy is not sufficient for all the vehicles' communications.

Simulation Result
5.2.1. RSU Clustering. As described in IV-A1, the dimensionality of original traffic flow data needs to be reduced before RSUs can be clustered with t-SNE. For the convenience of data visualization, the 96D data is reduced to 2D in our simulation and the visualization of the original traffic flow is shown in Figure 6. Based on the dimensionality reduction result with t-SNE, the K-means algorithm can be performed. As shown in Figure 7(a), the SSE value decreases with K increasing. When K is 7, the SSE starts to descend slowly. Although the SSE value still falls down, the falling gradient becomes smaller. Therefore, it is believed that the best K value is 7. After the clustering, all the RSUs are divided to 7 clusters.
Then, in Figure 7(b), the clustering result is described with the average value of all the vectors in each cluster. It is found that 3 clusters are shown clearly and each of them has a special evolving rule. The other clusters are not very clear as most of elements in the vector are very small, even almost 0. In Figure 8, we take four clusters as examples, each of which contains several 96-dimensional vectors of RSUs. It is found that the vectors in a cluster show similar evolving trending, which can prove the correctness of our cluster mechanism.

Deep Learning-Based Traffic Prediction.
In the deep learning-based prediction algorithm, the length of feature vectors is set to 5, 10, and 15. Table 2 shows the losses varying with various feature lengths for each cluster, in which the value is the Euclidean distance between the predicted vector and the original vector. When the length of feature is 5, the  Class  Length  5  1 0  1 5   1  3615  3608  3594  2  836  835  834  3  525133  439265  436616  4  27915  27420  27247  5  1282  1282  1282  6  857  857  856  7 63173788 60481962 59170782    loss is the highest compared to the others. For the length of the feature vector being 10 and 15, the losses for each cluster are almost the same. However, considering the computing resource exhausted, we choose the feature length as 10. In order to choose the best prediction model for RSUs in each class, the hyperparameters of the LSTM network are tuned by changing the number of layers and neurons of each layer at first. The activation function and loss function are set as "tanh" and "mse," respectively. Dropout and early stop are used to prevent overfitting, where the value of "dropout" is 0.2. The average MSEs of different models are shown in Table 3, in which "N" represents the number of neurons in each layer and "L" means the number of layers. As shown in Table 3, the model has less MSE loss when the layer is 1 Moreover, the batch size also affects the prediction accuracy of training models. Thus, the models with different batch sizes are trained so that the best prediction model can be found in each class. The average prediction error with different batch sizes of each class is shown in Table 4, where "B" and "C" denote "batch size" and "class," respectively. The smallest error is shown in bold, and the corresponding model is selected as the final prediction model. Next, we show the comparison between the prediction vector and the original vector in four grids as examples. In Figure 9, four RSUs are randomly selected from four clusters, which reveals that each prediction model fits well for the corresponding cluster.

Scheduling Algorithm.
In this part, we simulate the scenario that the RSUs do not have enough power to work through the entire period. Table 5 shows the energy consumption when the service ability is half and three quarters of requirement on different scheduling algorithms, in which "FCFS" denotes "first come first service," "random" means an RSU randomly chooses some vehicles to serve, "average" denotes the service ability of each interval is equal for each RSU, and "priority" represents that an RSU serves the vehicles according to the proposed scheduling algorithm in this paper. It can be seen that RSUs consume minimal energy in the two scenarios with the proposed scheduling algorithm.
Besides, the number of served vehicles under two scenarios are shown in Table 6. It is obvious that RSUs serve the maximized number of vehicles using minimized energy consumption with the proposed algorithm. Therefore, the performance of our proposed algorithm is proved.

Discussion and Conclusion
RSUs are playing a more and more important role in the IoVs. In terms of energy and deployment cost, renewable energy-powered RSUs will be used widely in the future. In this paper, we study the energy scheduling problem on renewable energy-powered RSUs. We propose a deep learning-based communication scheduling algorithm for RSUs in the IoVs, which is composed of RSU clustering, deep learning-based traffic prediction, service ability distribution algorithm, and priority based RSU accessing algorithm. At last, we conduct extensive simulation and the simulation results indicate that our algorithm can achieve a higher fairness rate while keeping a proper performance than the no scheduling algorithm.

Data Availability
The original data is the GPS data of taxis in Shenzhen, China, which were continuously sampled by the GPS devices during time period 2011/04/18-2011/04/26. Considering the large amount of original data, the processed data is submitted at the website https://http://github.com/pultoW/ trajectory. If there is any question, you can contact the author via ljt_it@163.com.

Disclosure
A part of this manuscript has been related in the Conference "2020 92nd IEEE Vehicular Technology Conference, Oct. 4, 2020" with technique parts extended, and the simulation has been done with extended algorithms and mechanisms proposed in this paper [28].

Conflicts of Interest
The authors declare that they have no conflicts of interest.