Development of Driver-Behavior Model Based on WOA-RBM Deep Learning Network

Human drivers’ behavior, which is very diﬃcult to model, is a very complicated stochastic system. To characterize a high-accuracy driver behavior model under diﬀerent roadway geometries, the paper proposes a new algorithm of driver behavior model based on the whale optimization algorithm-restricted Boltzmann machine (WOA-RBM) method. This method establishes an objective optimization function ﬁrst, which contains the training of RBM deep learning network based on the real driver behavior data. Second, the optimal training parameters of the restricted Boltzmann machine (RBM) can be obtained through the whale optimization algorithm. Finally, the well-trained model can be used to represent the human drivers’ operation eﬀectively. The MATLAB simulation results showed that the driver model can achieve an accuracy of 90%.


Introduction
Driver models can be applied to (1) vehicle dynamics [1] including vehicle component design, vehicle dynamics analysis, overall vehicle stability analysis, and design of onboard controls; (2) intelligent transport systems (ITS) [2,3] including simulation of traffic flow based on the control theory models of driver behavior and modeling drivers' risk taking behavior; (3) driverless vehicle systems [4]; and (4) traffic energy consumption systems [5]. Traffic energy consumption systems are different from the vehicle dynamics simulation. e traffic energy consumption system will be affected by the road, so our research focuses on how the road grade impacts on the driver's behavior characteristics. Driving behaviors, including acceleration behavior, deceleration behavior, and uniform behavior, have impacts on the driving safety [6,7], vehicle fuel consumption [8][9][10], and air pollution [11]. Designing a drive cycle of the vehicle requires investigating and collecting the practical driving data, analyzing the experimental data, and establishing the road vehicle driving conditions using relevant mathematical theoretical methods. e vehicle speed of this paper is collected based on the distance, and we also considered how the road grade influences driving speeds in its operating conditions. e operating conditions of the resulting vehicles can be used to determine the vehicle's fuel consumption and the technical development as well as evaluation of new models. It is very important to establish an accurate driver behavior model. e main factors that affect the accuracy of the driver behavior model include road geometry and weather condition [12,13]. Since the driver behavior is a very complicated stochastic system [14][15][16][17], designing driver behavior modeling is a very challenging task.
At present, the main achievements of research on driver behavior modeling technology are as follows. Cai et al. [18] developed a new concept of the driving fingerprint map to represent driving characteristics. Miyajima and Takeda [19] proposed a driver behavior modeling method by using on-road driving data. e method is realized through statistical machine-learning techniques, such as hidden Markov models and deep learning. Angkititrakul et al. [20] proposed a stochastic driver behavior model based on Gaussian mixture model framework. is proposed method allows adaptation scheme to enhance the model capability to better represent particular driving characteristics of interest. Shi et al. [21] proposed to evaluate driving styles by normalizing driving behavior based on personalized driver modeling. An aggressiveness index is proposed to quantitatively evaluate driving styles in this method, which can be applied to detect abnormal driving behavior. Yamada and Takahashi [22] proposed a driver behavior modeling method based on real traffic data under varying environmental conditions. In this method, the driving speed is assumed to be a function of several factors such as overall traveling schedule, speed, and road surface conditions. Taniguchi et al. [23] proposed an unsupervised learning method, which is established on the basis of the original double articulation analyzer model. is method predicts possible scenarios of driving behavior by segmenting and modeling incoming driving behavior time series data. Okuda et al. [24] proposed a probability-weighted autoregressive exogenous model wherein the multiple autoregressive exogenous models are composed of the probabilistic weighting functions. is model can represent the actual driving behavior. ere are plentiful publications on this topic using different optimization approaches, e.g., the instantaneous optimization algorithm [25], wolf pack algorithm [26], and genetic algorithm [27,28].
However, methods in [19,20] are very complex and strongly depend on historical data. Shi et al. [21], propose to quantitatively evaluate driving styles by normalizing driving behavior based on personalized driver modeling. e results show that the prediction accuracy of driving behavior modeling will be affected by complex environment. e establishment of methods in [22][23][24]29] requires a large amount of actual driving data as measurement data, which is also strongly dependent on historical data.
To solve these problems, the paper presents a new method of driver behavior model based on WOA-RBM. is method establishes an objective function, which contains the training process of RBM based on real driver behavior data. en, the best training parameters of RBM are obtained through WOA. Finally, the RBM after training based on the best training parameters can be used to build the driver behavior model. is paper is organized as follows. Section 2 describes the driving data collection. Section 3 presents the process of driving behavior modeling based on WOA-RBM. Section 4 shows the experimental results, which prove that the proposed method in this paper can achieve a better performance. Conclusions are offered in Section 5.

Driving Data Collection
In this section, the driving data come from the measurement data along the highway 120 near Manteca, CA, USA. In order to build a highly accurate driver behavior model, more than 2000 different drivers' driving behavior data were collected for each route. e measurement data were collected in Manteca during June 17th-July 28th (six weeks), 2018 [30]. Vehicle speeds are measured at 9 points identified in Figure 1. From the map (Figure 1), this highway is an approximately straight road. When modeling the driver, only the influence of the road grade on the speed of the driver is considered, so the curvature of the road is not displayed. In future research, we will take curvature of the road into building the driver model. e road grade is small, but it still has a great impact on the energy consumption of the vehicle. e energy reduction comes from two reasons. First, if a vehicle has constant speed or accelerates, less energy is consumed by the power plant (the engine and/or the electric motor) to drive the vehicle downhill than on a flat road because the gravity contributes positive work to overcome the negative work by aerodynamic drag and tire rolling resistance. e vehicle's potential energy is partly converted into its kinetic energy. Second, if a vehicle decelerates on a flat road or a downhill, the reduced kinetic energy is wasted by the brake as heat. On the other hand, if the vehicle decelerates on an uphill, part of the decreased kinetic energy is converted into the vehicle's potential energy and less kinetic energy is wasted by the brake. e gained potential energy can later be converted back into kinetic energy during a downhill. In summary, road slope change turns the vehicle's potential energy into an energy buffer to store the kinetic energy. e data are collected in 5-minute interval between 00:00 and 23:55 every day at each measurement point. Vehicle speeds at positions other than the 9 points are estimated by linear interpolation. e studied route stretches along 6.1 km of highway driving. e altitude varies from approximately 9 to 17 m. And the origin altitude is approximately 13 m while the terminal altitude is approximately 17 m. e road is sampled by 305 even steps with the step length Δs � 6100/305 � 20 m. e recorded speed trajectories for the first week and the altitude of the road are shown in Figures 2 and 3. e drivers generally increase the velocity between 4 and 6 km because this section of the road sets speed limits. Figure 4 shows the slope information of road environment. When the slope is larger than 0, the road is an uphill road; when the slope is lower than 0, the road is a downhill road; when the slope is equal to 0, the road is a flat road. e driving data are shown in Table 1. e data size is 2004 × 306, the data in each row represent the driving behaviors of 2004 different drivers in the same road section, and the data in each column show the driving behaviors of each driver at different sample points.

Driver Behavior Model Based on WOA-RBM
In this paper, a new driver behavior model based on WOA-RBM is proposed, which imitates human driving behavior during real-world driving.
is new method is designed based upon the theory of RBM and WOA.

Deep Learning Network Based on RBM.
e deep learning network based on RBM can solve the problem of the multilayer network training, which is also easy to realize [31,32]. e structure of deep learning neural network based on RBM is shown in Figure 5.
ere are two RBMs and one backpropagation (BP) network in this structure, with a hidden layer and a visible layer in each RBM, both of which are connected through a two-way connection between layers, as shown in Figure 6.
As in references [31,32], the equation of RBM can be defined as follows: where E(v, h) is the energy function between the input v vector and the hidden layer output vector h, W ij is a connection weight matrix, v i is the visible layer, h i is the hidden layer, a i is the bias of visible node i, and b j is the bias of hidden node j. e probability of each visible v and hidden layer h can be defined as e logarithmic gradient of weight in (2) can be calculated as where 〈v i h j 〉 data is the mean value of data and 〈v i h j 〉 model is the mean value of model. erefore, the learning rules of RBM can be computed as where Lr is the learning rate of RBM. Similarly, the parameter of indexes in equation (1) can be calculated as follows: From the above analysis, it is noted that the parameters affecting RBM training performance include the initial value of Lr, a i , b j , W ij , the number of hidden layer h, and the number of visible layer v. erefore, it is significant to choose the appropriate parameters.
According to the above principle, we assume that the training process of RBM is represented as follows: where err represents the training error. e smaller the training error is, the closer the driving behavior model similar to the actual driving behavior is.

Calculation of Optimal RBM Training Parameters.
Obviously, the calculation of optimal RBM training parameters is an NP-hard problem. It difficult to calculate the global optimal value directly. To resolve this problem, the paper proposes the whale optimization algorithm (WOA) [33][34][35]. e main steps of WOA are as follows: Step 1. Walking and foraging stage: the humpback whale can find the position of food, and the behavior of humpback whale is defined as where C → is a coefficient vector, D → is the distance between the whales and food, X → rand is the random position, and X → t is the current position. e position of the next moment can be defined as e vectors A → and C → in equations (7) and (8) can be presented as where r → is a random vector whose values lie in the range of [0, 1] and a → is the value which is decreased linearly from 2 to 0. In this paper, the whale population X → t is presented as Step 2. Encircling and contracting stage: when the humpback whale finds the target food, the other whales will go to the position of the humpback whale and surround the food. e equation of this stage can be presented as follows: where X → * t is the vector of random position, which is chose to the current whale population.
Step 3. Spiral predation stage: all the whales will move in a spiral direction of the optimal position of the humpback whale, and then the whales will generate many bubbles to surround the food for predation; the equation of this stage is where D ′ is the distance between the whale and the optimal solutions, b is a shape of logarithmic spiral, and l is a random number in the range of [−1, 1].
Finally, we can define a random value p to distinguish the contraction-bounding stage from the spiral predator. e equation is as follows: To sum up, the flowchart of WOA is shown in Figure 7.

Driver Behavior Model Based on WOA-RBM.
Upon the completion of the optimization process, we obtain the optimal parameters: Subsequently, the optimized RBM model can be expressed as e learning rules of optimized RBM can be calculated as According to the theories introduced above, the driver behavior model based on WOA-RBM is shown in Figure 8. e driver behavior model and the vehicle model constitute a closedloop control system. e driver behavior model is actually an inverse model of the controlled object (vehicle). In this paper, based on road test data, the learning control method is used to build a driver model through machine learning. Assume that the vehicle speed at a certain position is V(s), and the driver presses down the accelerator pedal or brake pedal which as a consequence will affect the vehicle speed at the next position V(s + 1). e expected speed of the next position is V e (s + 1), i.e., the road's speed limit. e collected vehicle speed data are used to train the optimal control parameters which are input into the driver behavior model. At the same time, the road grade, the actual speed of the vehicle, and the deviation between the actual speed and the expected speed are also input into the driver behavior model. Subsequently, the driver behavior model outputs acceleration or deceleration to control the driving of the vehicle. Figure 8 shows the proposed driver behavior model. In this model, the sampled driver data are used to train the WOA-RBM model. e optimal parameters Lr opt , v opt , h opt , a i,opt , b j,opt , and W ij,opt of deep learning network are obtained after WOA optimization. e WOA-RBM model after training with the optimal training parameters is the driving behavior model. Finally, the driving behavior model is used in a vehicle control system. In Figure 8, the "road information" specifically refers to the road grade. ΔV indicates the difference between actual speed and desired speed. e whole closed-loop control system output is the vehicle speed. Drive operation is acceleration or deceleration.

Experiment
e proposed driving behavior model in this paper is simulated and validated through MATLAB 2017b platform. e performance of the proposed driving behavior model Calculate the fitness according to formula (7) Next iteration Walking and foraging stage Encircling and contracting stage Spiral predation stage Update the fitness according to formula (7) N Y Finish?
Obtain the optimal whale individuals X t,opt = [Lr opt , v opt , h opt , a i,opt , b j,opt , W ij,opt ] The initial parameters

e Parameters.
e initial parameters of the proposed driver behavior model are shown in Table 2.
In this experiment, 100 different road sections were collected, and about 2000 drivers' driving data were collected for each route. e driving behavior data of each driver in each route were collected at 306 points.
In this paper, 70 groups are randomly selected as training samples, and the remaining 30 groups are selected as testing samples.

WOA Optimization Process.
e optimization process of WOA is illustrated in Figure 9. Figure 9 shows that when the number of iterations is 9, WOA minimizes the training error of RBM deep learning neural network. e comparison of RBM parameters before and after optimization is shown in Table 3.   Journal of Advanced Transportation e simulation results show that with the increase of the number of iterations, the training error of RBM deep learning network gradually reduces. When the number of iterations is 4, WOA minimizes the training error of RBM deep learning neural network, which is within a precision of 10 − 4 . e comparison of RBM parameters before and after optimization is shown in Table 3. After optimized by WOA, the RBM optimal learning rate Lr is 0.0039, the number of visual layers v is 2, the number of hidden layers h is 2, the optimal values of parameters a and b are 0.1748 and 0.6883, respectively, and the optimal initial average of the initial network weights w is 0.088. Under these parameters, the training effect of RBM can obtain the optimal result.

e Performance of Driver Behavior Model.
e information of the road is already shown in Figure 4. e actual driving behavior and the driving behavior predicted by the driver behavior model are simulated in Figure 10. Figure 10 shows that the six drivers' behaviors from the output of the WOA-RBM-based driver behavior model are consistent with the actual driving behavior. erefore, the driving behavior model proposed in this paper exhibits high prediction accuracy.
Furthermore, the standard error of estimate (SEE) is used to evaluate the proposed models by more driving behaviors, which is defined as where dat actual (i, n) are the actual driver behavior data, i.e., the vehicle speeds from the i − th driver at the n − th sampled point, and dat WOA−RBM (i, s) are the predicted driver behavior data from the i − th driver at the n − th sampled point. e simulation result of SEE is shown in Figure 11. Figure 11 shows that the SEE of the driving data and the actual data is obtained by the WOA-RBM model. e simulation results show that the SEE of more than 2000 different drivers is lower than one. erefore, the driver behavior model based on WOA-RBM can correctly simulate the behavior of the driver.
rough training the 100 driving operation data in different routes, a high level of prediction accuracy is obtained. e corresponding test structure is shown in Figure 12: From the results of simulation, it can be seen that for 100 different routes, the driving behavior prediction algorithm proposed in this paper can be obtained with lower SEE. Compared with the simulation results in Figure 13, for different routes, the SEE of the driving behavior prediction algorithm proposed in this paper is always close to 0.2. It shows that the accuracy of the algorithm is high.
Overfitting means the training error is very small, while the generalization error is very large. Because the model may be too complex, it "remembers" the training samples, but its generalization error is very high. In the algorithm proposed in this topic, the dropout mechanism is used to prevent overfitting.

Comparison.
To evaluate the performance of the WOA-RBM-based driver behavior model, some other driver behavior models are presented for comparison and analysis, including the model based on database of personal mobility driving [36], stochastic driver pedal behavior model [16], driver behavior modeling using on-road driving data [19], and driver behavior modeling using hidden Markov model based on genetic algorithm [28]. e simulation results of the four methods are shown in  e driving behavior predicted by the driver behavior model is shown in Figure 14. e simulation result indicates that the driving data obtained by the proposed WOA-RBM model are closer to the actual driving data. e SEE value of the outputs of different driving models and the real driving data is calculated according to equation (17). It can be seen that the performance of the WOA-RBMbased driving model proposed in this paper is better than  that of the driving models proposed in [16,19,28,36]. is is due to the fact that the WOA-RBM-based driving model proposed in this paper can obtain the optimal training parameters of RBM deep learning through WOA and can obtain a model more in line with actual driving data by learning driving training data. Figure 13 shows that the SEE of the WOA-RBM model is the lowest, while the model in [28] is the largest. Figure 13 shows the SEE of all the driver models. e driving data are added to verify the influence of different training data, which is shown in Table 4. Comparing the five driving models, the corresponding SEE values are calculated by equation (17) and the WOA-RBM driving behavior prediction model proposed in this paper has the highest prediction performance which has better performance than other algorithms with less training samples. Table 5 shows the influence of different roads, including smooth road, less undulating road, and large undulating road. e simulation results show that the proposed WOA-RBM model has the best performance among all kinds of    roads. Several remaining algorithms have larger prediction errors. Especially on road with large undulations, the difference becomes more and more obvious.
On the other hand, the correct and optimal driving behavior can also reduce the energy consumption of the vehicle. In our previous paper [29], the energy consumption model has been discussed. In this paper, the same energy consumption estimation method is used. e energy is calculated as the mechanical energy required for propelling the vehicle. e energy consumption of a vehicle can be estimated from its longitudinal dynamics, as illustrated in Figure 15. By Newton's law, we have the following equation: where M veh is the vehicle mass, a is the vehicle acceleration, v veh is the vehicle speed, F roll is the rolling friction, F aero is the aerodynamic drag, F grade is the force caused by the road slope, F trac is the traction force generated by the powertrain, g is the gravity acceleration, α is the road grade angle in radiant, c roll is the rolling friction coefficient, ρ air is the density of the ambient air, c d is the air resistance coefficient, and A f is the frontal area of the vehicle. Figure 16 compares the performance of energy consumption. e correct and optimal driving behavior can also reduce the energy consumption of vehicle. Figure 16 shows that the proposed WOA-RBM-based driver behavior model has lower energy consumption than other four models.
When operating a motor vehicle, providing the proper driving operation is essential, as it will reduce energy consumption.
is means that frequent acceleration and braking is an improper driving method and will increase energy consumption.

Conclusion
In this paper, a new algorithm of driver behavior model based on WOA-RBM deep learning network is proposed. e model establishes an objective function, which contains the training of deep learning network. e RBM after training based on optimal training parameters is used to predict the output action accurately through the road information. In the future, it is of great interest to collect huge amounts of real driving data from different roads and different drivers.

Data Availability
All raw data in this study are available free of charge. Readers who wish to repeat this study can do so through the following link: http://pems.dot.ca.gov/.

Conflicts of Interest
e authors declare that they have no conflicts of interest.  [36] Model in Ref [16] Model in Ref [19] Model in Ref [28]