DeepCF: A Deep Feature Learning-Based Car-Following Model Using Online Ride-Hailing Trajectory Data

International School, Beijing University of Posts and Telecommunications, Beijing 100876, China School of Software, Dalian University of Technology, Dalian 116620, China Computer Science Department, Community College, King Saud University, Riyadh 11437, Saudi Arabia College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China Mathematics and Computer Science Department, Faculty of Science, Menoufia University, Shebin-El-Kom 32511, Egypt


Introduction
Vehicle following, the most common drivers' behavior in traffic, exerts more important influence on many factors including traffic flow characteristics, traffic safety, and traffic simulation results. The car-following model serves as a basic algorithm of traffic simulation tools (such as SUMO and VIS-SIM) and an indispensable control algorithm for automated vehicles [1,2]. The model is aimed at replicating drivers' car-following behavior. The kinematics-based car-following model attempts to describe the kinematic mechanism of vehicle-following maneuver [3][4][5][6][7][8][9][10][11][12]. Most of the parameters have obvious physical meaning. The output of the model can be easily controlled by adjusting the model parameters, so as long as the appropriate parameters, it can perform better in car safety. As the car-following model based on machine learning attempts to learn the human drivers' vehicle-following motion from a large number of human drivers' vehicle-tracking data [13][14][15][16][17], this category of model has a high accuracy in simulating the human drivers' vehicle following.
However, as the existing car-following models have become increasingly accurate in predicting driving decisions, they overlook the dynamic time for the drivers to execute the decision [18]. They set the drivers' reaction time to a fixed value [19]. This setting will greatly affect the car-following simulation performance. For example, in the car-following case, the front car brakes suddenly, causing the rear drivers to decide to slow down [20]. If this decision is 1.3 seconds late, it is likely to collide with the car ahead. Khodayari et al. use the performance characteristics of the drivers' stimulus and reaction while driving to calculate the drivers' reaction time in NGSIM data [21]. And they add the reaction time as known information to the existing car-following model for simulation experiments [22]. Finally, they confirmed that the drivers' reaction time existing in the carfollowing model will greatly improve the accuracy of the model simulation. However, they failed to build a model capable of calculating the drivers' reaction time. In addition, in predicting the driving behavior of the drivers, the existing car-following model does not involve two crucial factors. One is the impact of driving fatigue on driving decisions; the other is that driving fatigue causes the drivers' sensitivity and judgment to decline [23]. For example, driving for 1 hour continuously, a driver who faces the sudden acceleration of the vehicle ahead may be inclined to follow the same behaviour. However, when driving continuously for 3 h, in the face of the same scenario, the drivers' decision may be conservative acceleration, that is, the throttle will be much lighter than that before 2 h.
In this paper, we propose Deep Feature Learning-based Car-Following Model (The model frame diagram is shown in Figure 1). First, we study the influence of driving time and the acceleration of the preceding vehicle on the drivers' reaction time and establish a coarse-grained drivers' reaction time model. Secondly, grounded in the impact of driving time on car-following decisions, we generate a driving decision database based on GAN and use Euclidean distance as a decision indicator. Then, the decision search algorithm is proposed. Finally, we conduct contrast experiments on a real trajectory data, and the performance of DeepCF is evaluated.
The main contributions of this paper are listed as follows: (1) A car-following decision algorithm based on Generative Adversarial Networks is proposed, and a method for establishing a driving decision database based on Euclidean distance as a decision index is proposed (2) We design the framework based on the evidence that driving for a longer time will lead to a longer reaction time of the drivers (3) Analyze the model using China's online car-hailing trajectory data The paper is organized as follows: Section 2 introduces the related work of the car-following model. Section 3 designs the car-following model. Section 4 provides experimental results and compares them with traditional regression models. Section 5 summarizes the whole article.

Related Work
In this section, we first introduce the existing car-following model based on kinematics and the car-following model using machine learning algorithms. Then, we illustrate that the fatigue state will affect the drivers' reaction time.
2.1. Car-Following Model. Car-following models can be divided into kinematics-based models and models with machine learning algorithms. In the kinematics-based carfollowing models, Chandler et al. [3] first proposed the General Motors (GM) model. This model puts forward the relative speed of the front and rear vehicles to calculate the acceleration of the rear vehicle [4,5]. The Gipps model takes the safety distance into account [6]. The optimal speed model obtains the expected rear vehicle speed based on the distance between the front and rear car heads [7,8]. The action point model divides car following into different stages and sets space or speed thresholds separately [11,12]. The carfollowing models based on machine learning algorithms are data-driven models. Wewerinke [13] uses neural networks to model the car-following behavior and achieves high performance. Khodayari et al. [14] propose an improved neural network car-following model with response time as input and verified it using NGSIM data [21]. The results show that the error is significantly less than other neural network models. Wei et al. [16] established a car-following model based on least squares support vector machine (LS-SVR) and used the microscopic traffic simulation system dataset [17] to verify the model. The experimental results show that LS-SVR car-following model is more accurate than the Gipps car-following model [6] and neural network car-following model.

Drivers' Reaction Time.
In driving tasks, the drivers' brain nerve activity caused by exogenous stimuli is correlated with the drivers' reaction time [24], and there are many factors that can significantly affect the drivers' reaction time. Weng [23] divide the drivers' reaction behavior into three stages of perception, determination, and action. And this   Wireless Communications and Mobile Computing paper verified that the drivers' sensitivity and judgment ability would weaken on a long road under fatigue state, and it would take longer time to make driving decisions. Visual distraction can further cause drivers to lose their attention on the road and affect their reaction time [25,26]. Ru et al. [27] confirm that the subjective conditions that interfere with the drivers' response include driving experience, mental and physical conditions, and adaptability. Petermeijer et al. [28] explore the interaction between nondriving task types and take-over request methods. The test experiment of 101 volunteers confirm that nondriving tasks will increase the response time of the corresponding take-over request, and the initial response time of tactile and auditory take-over requests is lower than that of visual. In a traffic accident scenario, the drivers' reaction is related to the speed of obstacles before the accident [29], and the drivers' reaction time is linearly related to the collision time [30]. Xue et al. [31] analyze the simulation data of 47 volunteers in simulated carfollowing scenarios and found that in the case of high traffic flow density, the response time of drivers is usually shorter than that of low to medium traffic density [32,33], while the response time of male and nonprofessional drivers tends to be slightly longer [34].

Methodology
This section introduces the DeepCF model in detail, including driver reaction time model and car-following decision algorithm based on GAN.

Drivers' Reaction Time Model Based on Fatigue Driving
Phase Combination Model. It is intuitively obvious that the drivers' reaction time of fatigue driving is longer than that of normal driving. And in the assumption of this paper, the car-following decision is closely related to the drivers' reaction time. We need to reveal the relationship between the drivers' reaction time and driving time to build drivers' reaction time model. The reaction time model can quantitatively represent the relationship between driving time and reaction time. They investigate 294 drivers and test their reaction time [35]. The final statistical results showed a strong correlation between duration of driving and reaction time. On the basis of their experimental results, we use the cubic function to fit the relationship between duration of driving and the slowest reaction time. Our experiments show that cubic polynomials are the best choice to fit this relation. Due to the lack of original experimental data, the accuracy of the fitted function remains to be verified. The formulas are shown in Equation (1) and function visualization is shown in Figure 2.
According to the results of our experiment, in Equation (1), α 3 is -0.067, α 2 is 0.9769, α 1 is 15.27 and, α 0 is 434.4. These parameters can fit the duration of time and drivers' reaction time well. And there is no overfitting in this model.
3.2. Car-following Decision Algorithm Model Based on Generative Adversarial Network. We need to design the drivers' car-following decision algorithm. First, we need to establish a decision library. We will use GAN (Generative Adversarial Network) to generate the decision library rules. GAN, an unsupervised deep learning model, is composed of two parts: generator and discriminator. The generator generates data close to the characteristics of the training set as much as possible, and the discriminator should attempt to determine the authenticity of the generator. It can find out the internal statistical law of the given observation data and   (2).
where G is the differentiable function of the generator, D is the differentiable function of the discriminator, P data is a real sample, x is the sample taken from P data , and DðxÞ means to distinguish x; we hope that the result of this discrimination is closer to 1, as possible the bigger the loss function log ðDðxÞÞ. P z is the sample generated by G, z is the sample taken from P G , and DðGðzÞÞ means to distinguish GðzÞ; we hope that the smaller the result, as possible the bigger the loss function log ð1 − DðGðzÞÞÞ. The structure of the training set is shown in Table 1.
The network is divided into two parts: generator and discriminator. The generator is composed of input layer, hidden layer, activation layer, and output layer. The dimension of the input and output layers is 1 * 4. The hidden layer contains 2 * 2 * 11 neural network units. The activation layer is composed of the Maxout activation function and sets the k value to 2. Maxout has a strong fitting ability; it can fit any convex function. Maxout has the advantages of ReLU, such as no linear saturation, but also does not have the disadvantage that the ReLU unit is fragile and may die.
Let the distribution generated by the generator (G) be P G ðx ; θÞ, where θ is the parameter of the distribution. And let x i be derived from the true distribution in the generator to calculate the likelihood P G ðx i ; θÞ as shown in Formula (3).
Then, you need to find a θ * to maximize the likelihood, as shown in Formula (4).
The discriminator (D) is composed of an input layer, a hidden layer, an activation layer, and an output layer. The activation function is the sigmoid function. The discriminator's data consists of two parts, the first part is the real data set P data , and the second part is the fake data P G generated by the generator. If x comes from P data , DðxÞ should be as close to 1 as possible. If x comes from P G , DðxÞ should be as close to 0 as possible. The pseudocode of GAN algorithm is shown in Algorithm 1. The structure diagram of Algorithm 1 is shown in Figure 3.
The decision index is used to evaluate the similarity of the input features and the data of the decision database, that is, to calculate the similarity of the two feature vectors. There are many ways to calculate vector similarity, such as Pearson correlation coefficient, cosine similarity, and Manhattan distance. The decision index is mainly used to calculate the

Input:
Training set data distribution P data ðxÞ; random noise distribution P g ðzÞ; Total training times epochs; the number of iterations of the discriminator k; the learning rate of the discriminator s 1 ; the learning rate of the generator s 2 ; the amount of training data per batch n.

Output:
The network parameters of the discriminator θ d ; network parameters of the generator θ g . Begin 1. Initialize θ d , 2. For epochs do 3. For k do 4. Sample n samples fz ðjÞ g n j=1 from the random noise distribution P g ðzÞ 5. Sample n samples fx ðjÞ g n j=1 from the real data distribution P data ðxÞ 6. Update θ d by boosting the stochastic gradient: 7. ∇ θ d ð1/nÞ∑ n j=1 ½log Dðx ðjÞ Þ + log ð1 − DðGðz ðjÞ ÞÞÞ 8. End for 9. Sample n samples fz ðjÞ g n j=1 from the random noise distribution P g ðzÞ 10. Update ∇ θ g with gradient by decreasing: where y represents a feature vector in the decision library, x represents the input feature vector, and the first three features of x and y correspond to the first three features of the data structure shown in Table 1.
Due to the relatively large amount of rule data in the decision database, traversing the entire decision database to find the most similar vectors may affect the algorithm performance, so the retrieval of the decision database should be optimized. The statistics show that, aside from extreme cases, the value range of y 1 is approximately [-5,10], the value range of y 2 is approximately [0,70], and the value range of y 3 is [1,80]. Then, there are about 80,000 combinations of these three feature vectors. The following defines the retrieval algorithm s¯su:  (6) and (7) describe the model details.
where x is the feature vector.

Experiments
This chapter mainly introduces the experimental part, first introduces the comparative experiment, then defines the modulus evaluation index, and finally, analyzes the experimental results.

Experiment Data.
The data set used in this article is the trajectory data of Didi drivers in Xi'an on October 26 and 27, 2016, provided by Didi Travel (please visit: https:// outreach.didichuxing.com/research/opendata/). The data set tracks approximately 18,000 vehicles, including the vehicle number, location, and time. The trajectory tracking time interval is 1 s. We screened the vehicles driving continuously for more than 3 h between 4:00-22:00, and finally extracted 5748 following scenes. The carfollowing duration is 9 s. We further add attributes such as speed, acceleration, driving duration, and distance to each piece of data. We divided the data set into two groups, the first group is the following vehicles in the scene where the driving time of the rear vehicle is less than two hours, and the second group is the vehicle that exceeds two hours.  Besides that, we select 100 follow-up scenes in each group as the test set. Data attribute list refers to Table 2. 4.2. Comparative Experiment. T. Cover and P. Hart proposed k-nearest neighbor in 1967. The working principle is there is a sample data set, and each data in the sample set has a label, that is, we know the correspondence between each data in the sample set and its classification. After inputting new data without labels, we compare each feature of the new data with the features corresponding to several types of data in the sample, and then, the algorithm extracts the classification label of the most similar data (nearest neighbor) of the sample. It uses the following Formula (8) to calculate the distance D. x and y are the two features that need to be calculated.
Random forest is a model composed of many decision trees. In the training process, each tree in the random forest will learn from randomly sampled data points. The idea is to train each tree on different samples. Although for a specific training data set, the variance of each tree may be high, but in general, the variance of the entire forest will be very low without increasing the offset. In the test, predictions are made by averaging the predictions of each decision tree. This process of training a single learner on different self-sampled data subsets and averaging predictions is called bagging. The cart tree is used in the random forest algorithm in Sklearn.
The Gini index reflects the probability that two samples are randomly selected from the data set, and their category labels are inconsistent. Therefore, the smaller the Gini index, the higher the purity of the data set. The Gini index (Formula (9)) can be used to measure any uneven distribution. It is a number between 0 and 1.0 that is completely equal, and 1 is completely unequal.
where k represents the category. The CART classification tree uses the size of the Gini coefficient to measure the division points of features. In the regression model, we adopt the common sum variance measurement method. For any partition feature A, the corresponding arbitrary partition point s is divided into data sets D 1 and D 2 on both sides, and the mean square error of each set of D 1 and D 2 is minimized. The feature and feature value division point corresponding to the minimum sum of mean square error of D 1 and D 2 . The expression is Formula (10): Among them, c 1 is the sample output average of the D 1 data set, and c 2 is the sample output average of the D 2 data set.  Wireless Communications and Mobile Computing 4.3. Model Evaluation. In this paper, mean absolute error (MAE), mean squared error (MSE), and root mean squared error (RMSE) are used for measurement and evaluation [36]. The indicator formulas are shown in Equations (11), (12), and (13).
where N is the total number of samples, x i represents real data value of car i, andx i means the predicted value of the model.

Results and Discussion
This paper proposes a car-following model based on driving fatigue and generative confrontation network, namely, dynamic car-following model (DeepCF). We use 474 real car-following scenes for experiments, and the experiments combine the DeepCF model proposed in this article with the regression model regression tree (RT), polynomial kernel function (PKF), radial basis kernel function (RBKF), boost tree (BT), extreme forest (EF), linear kernel (LK), k-nearest neighbor (KNN), and random forest (RF) for comparison. We use MAE, MSE, and RMSE as evaluation indicators. We use 474 real car-following scenes as the test set. First, we use all real 8000 car-following scenes as a decision library. Besides, we adopt the decision database retrieval algorithm s¯su; the acceleration of the following vehicle after the reaction time is matched according to the speed of the following vehicle, the distance between the front and rear vehicles, and the speed difference between the front and rear vehicles in the test set. And we calculate the evaluation index of the comparison between the real car-following scene and the matching result (for the convenience of subsequent experiments, we will name the experiment CF here). Then, we use the above regression model to obtain the acceleration of the following vehicle after the reaction time. It also calculates the evaluation index comparing the real car-following scene with the regression result. Finally, we compare the evaluation index of the matching result with the evaluation index of the regression result. The results are shown in Figures 4, 5, and 6.
According to Figures 3, 4, and 5, we can find that when using a limited number of real car-following scenes as the decision-making database, using the decision-making database retrieval algorithms, the matching results are better than RT, EF, KNN, and other regression models. However, compared with RBKF, LK, and other regression models, there is still a certain gap.
We use 8000 real car-following scenes as raw data to use the GAN to generate 80,000 car-following scenes, and combine the real and generated car-following scenes as a decision library. Using the decision database retrieval algorithm s¯su, the acceleration of the following vehicle after the reaction time is matched according to the speed of the following vehicle, the distance between the front and rear vehicles, and the speed difference between the front and rear vehicles in the test set. And we calculate the evaluation index of the comparison between the real car-following scene and the matching result (our model DeepCF). We compare the evaluation indicators obtained after the DeepCF model experiment with the above CF model Figure 7.
We find that after using the GAN to expand the decisionmaking database, the matching results are significantly better than before data generation. This proves the necessity of using GAN to generate car-following scenes.
We compare the evaluation indicators obtained after the DeepCF model experiment with the regression model, and the results are shown in Figures 8,9,and 10. In the premise of using the confrontation generation network to generate 80,000 car-following scenes with 8000 real car-following scenes as raw data, combining the real and generated car-following scenes as a decision-making database and adopting the decision-making database retrieval algorithms, the matching data performs better than most data in the regression model. Thus, the effectiveness of our DeepCF model is proved.

Conclusions
In view of the fact that existing car-following models fail to consider the impact of driving fatigue on driver reaction time and decision-making, this paper proposes a car-following model (DeepCF) based on driving fatigue and generating a confrontation network. A car-following decision algorithm based on a generative confrontation network is proposed, and we build a driving decision database with GAN. Besides a comparative experiment, we further conduct a comparative evaluation of the model. The results demonstrate that our car-following model (DeepCF) is closer to the real scene than the regression model.

Data Availability
The data used to support the findings of this study were supplied by Didi Travel under license and so cannot be made freely available. Requests for access to these data should be made to the corresponding author.  Wireless Communications and Mobile Computing