Learning-Based Lane-Change Behaviour Detection for Intelligent and Connected Vehicles

Detection of lane-change behaviour is critical to driving safety, especially on highways. In this paper, we proposed a method and designed a learning-based detection model of lane-change behaviour in highway environment, which only needs the vehicle to be equipped with velocity and direction sensors or each section of the highway to have a video camera. First, based on the Next Generation Simulation (NGSIM) Interstate 80 Freeway Dataset, we analyzed the relevant features of lane-changing behaviour and preprocessed the data and then used machine learning algorithms to select the suitable features for lane-change detection. According to the result of feature selection, we chose the lateral velocity of the vehicle as the lane-change feature and used machine learning algorithms to learn the lane-change behaviour of the vehicle to detect it. From the dataset, continuous data of 14 vehicles with frequent lane changes were selected for experimental analysis. The experimental results show that the designed KNN lane-change detection model has the best performance with detection accuracy between 89.57% and 100% on the selected dataset, which can well complete the vehicle lane-change detection task.


Introduction
Over the past few years, with the rapid development of artificial intelligence and communication technology, intelligent vehicles based on intelligence and networking have become a major trend in the development of the automotive industry. From the perspective of technological development, intelligent vehicles are divided into three development directions: connected vehicle (CV), autonomous vehicle (AV), and the integration of the former two, namely, connected and automated vehicle (CAV) or intelligent and connected vehicle (ICV) [1].
ICVs play an important role in improving driving safety and reducing driver burden, contribute to energy conservation and environmental protection, and improve traffic efficiency. Research shows that, in the initial stage of ICVs, advanced intelligent driving assistance technology can help reduce traffic accidents by about 30%, improve traffic efficiency by 10%, and reduce fuel consumption and emissions by 5% [2]. Entering the ultimate stage of the ICVs, that is, the fully automatic driving stage, it could avoid traffic accidents, improve traffic efficiency, and finally liberate people from boring driving tasks.
Driving behaviour detection plays a significant role in ICVs' decision-making system. During the driving of the vehicle, due to the driver's lack of attention or the obstruction of the surrounding large vehicles, it is likely that the driver will not be able to fully understand the driving conditions of the surrounding vehicles, thus causing great safety risks. Many methods of lane-change behaviour detection have been proposed by researchers in recent years, including hidden Markov model (HMM) [3][4][5], multi-view convolutional neural network model (MV-CNN) [6], and vision-based deep residual neural network (RNN) [7]. Detection of lane-changing behaviour in different scenarios, including highways [8][9][10][11][12] and signalized intersections [13][14][15][16], has also been studied by many researchers. Steering behaviour recognition [17] and prediction [18] methods have been proposed, too. Besides, some new deep learning and machine learning methods have also been proposed in recent years. Xie et al. comprehensively modeled lanechange using deep learning approaches including deep belief network and long short-term memory [19]. Xing et al. proposed an ensemble bi-directional LSTM model for driver intention inference [20]. Gao et al. proposed a data-driven lane-change detection system using deep learning techniques [21]. Zhang et al. modeled the car following and lanechanging behaviours simultaneously using hybrid retraining constrained long short-term memory neural networks [22]. Zhao et al. proposed a new quantitative discriminant model based on deep belief networks algorithm and the classification analysis method based on support vector machine [23]. Dang and Dai established a lane-change model based on improved Bayesian network [24]. These methods, however, need prior knowledge, or the structure is complex and the real-time performance can be improved. In practical application scenarios, there is usually lack of prior knowledge of data distribution, and a simpler classification method is easier to implement.
In this paper, we proposed a method and designed a learning-based detection model of lane-change behaviour on highways, which only needs the vehicle to be equipped with velocity and direction sensors or each section of the highway to have a video camera. The main contributions of this paper can be summarized as follows: ( The rest of the paper is organized as follows. Section 2 explains the details of the dataset. Section 3 describes the methodology of lane-change behaviour detection, including feature selection and lane-change detection method. Section 4 presents the experiments and results of lane-change behaviour detection. Section 5 concludes this paper and discusses the future work.

Data Processing
In order to verify the lane-change detection method, NGSIM Interstate 80 Freeway Dataset initiated by the United States Department of Transportation (US DOT) Federal Highway Administration (FHWA), which is freely available at the NGSIM web site at http://ngsim.fhwa.dot.gov, is used and processed. The dataset contains 45 minutes, divided into three periods: 4:00 p.m. to 4:15 p.m.; 5:00 p.m. to 5:15 p.m.; and 5:15 p.m. to 5:30 p.m., which represent the buildup of congestion, the transition between uncongested and congested conditions, and full congestion during the peak period, respectively [25]. As shown in Figure 1, the six-lane study area with a length of 1650 feet is divided into seven sub-areas. In each sub-area, a video detector is installed on the high-rise building near the lane, and the traffic of the sub-area is photographed and recorded.
The original dataset contains many attributes, including some attributes that are not highly relevant to the lanechange detection. In order to establish a dataset suitable for vehicle lane-change detection, the attributes in the dataset that are not highly relevant to lane-change detection were deleted, Vehicle_ID, Lane_ID, V_Length, and V_Width remained the same as those in the original dataset, LX_m, LY_m, Vel_m/s, and Acc_m/s 2 changed the unit in the original dataset from feet to meters (1 foot � 0.3048 meters), the average lateral velocity of vehicle and instantaneous lateral acceleration of vehicle were, respectively, calculated by LX_m and Acc_m/s 2 and added to the dataset, and the lane-change behaviour of the vehicle was calculated by Lane_ID, forming a new dataset. In the Lane_changing attribute, 0 means to keep the current lane, 1 denotes a single lane change to the right, −1 stands for a single lane change to the left, 2 represents sequential lane change to the right, and −2 represents sequential lane change to the left. The composition of processed data is shown in Table 1.
The instantaneous lateral acceleration of vehicle Acc_X can be calculated as Acc_X t � Acc_m/s 2 t * sin acr tan where Acc_X t represents the value of Acc_X at time t, Acc_m/s 2 t denotes the value of Acc_m/s 2 at time t, LX_m t and LX_m t−1 , respectively, mean the value of LX_m at times t and t − 1, while LY_m t and LY_m t−1 stand for the value of LY_m at times t and t − 1, respectively.
The average lateral velocity of vehicle Vel_X can be calculated as 2 Computational Intelligence and Neuroscience where Vel_X t represents the value of Vel_X at time t, LX_m t means the value of LX_m at time t, LX_m (t−1) denotes the value of LX_m at time t − 1, and t is the sampling period of the dataset, which is 0.1 seconds. Current lane-change behaviour of vehicle Lane_changing can be calculated as where Lane_changing t represents the value of Lane_changing at time t, Lane_ID t means the value of Lane_ID at time t, and Lane_ID t denotes the value of Lane_ID at time t − 1.

Feature Selection.
In order to accurately detect vehicle lane changes, the relationship between various attributes and vehicle lane changes was analyzed. By analyzing the changing trends of various attributes when the vehicle changes lanes in the dataset, we found that the vehicle's lateral velocity has the most obvious correlation with the lane-change behaviour. The relationship between lateral velocity and lane change is shown in Figure 2, from which we can see that when the vehicle changes lanes, there will be a very obvious change in lateral velocity.
To further analyze and verify the relationship between each attribute and the lane-change behaviour of the vehicle, machine learning models, including KNN [26], extra trees   Computational Intelligence and Neuroscience 3 [27], and random forest [28], were performed on each attribute. A total of 27287 lane-changing data were extracted from the dataset. At the same time, in order to balance the number of samples of lane-changing (the moment when the vehicle changes lanes) and non-lane-changing (the moment when the vehicle does not change lanes) data, 27287 nonlane-changing data were selected to form a feature selection dataset. The result of feature selection is shown in Table 2, from which we can see that the detection accuracy with Vel_X as the feature is significantly higher than other features, which can reach more than 90%. In the detection using Vel_X, the accuracy of KNN, extra tree, and random forest is 94.85%, 91.73%, and 92.31%, respectively; KNN has the highest accuracy. In order to represent the contribution of each feature to the lane-change behaviour detection more intuitively, feature importance analysis, which can be applied to random forest and extra trees, was performed on the dataset. The Gini index was used to measure the feature importance, which can be defined as where K means that there are K categories and p mk denotes the proportion of category k in node m. VIM is used to represent the variable importance measures. The importance of feature X j at node m, that is, the change in Gini index before and after the branch of node m, can be defined as where G l and G r represent the Gini index of the left and right nodes after the m branch. If the node where the feature X j appears in the decision tree i is in the set M, then the importance of X j in the i-th tree is Assuming there are n trees, then Finally, supposing there are c features, all the obtained importance scores are normalized: The feature ranking based on feature importance is shown in Table 3, which illustrates that Vel_X has the highest feature importance scores in both random forest and extra tree and is significantly higher than the other six features. Therefore, Vel_X can be selected as a feature of lane-change detection for ICVs. changes to the left and right are denoted as −2 and 2, respectively. The lateral velocity of single lane change intercepted from selected data is shown in Figure 3, from which we can see that there is a significant peak/valley when the vehicle changes lanes, and the threshold of peak/valley can be learned to determine if the vehicle is changing lanes. The lateral velocity of sequential lane change intercepted from selected data is shown in Figure 4; similar to single lane change, there is also a significant peak/valley when the vehicle changes lanes. Besides, there will be a continuous peak/valley or a larger peak/valley of lateral velocity in the sequential lane change, which also can be learned to determine if the vehicle is changing lanes sequentially.

Detection
Model. KNN is a simple classification method that can perform effective classification in the absence of prior knowledge of data distribution, and the same are the ET and RF. RF is an algorithm that integrates multiple trees through the idea of integrated learning and its basic unit is the decision tree [29]. The construction process of the random forest model is mainly divided into four steps: (1) First, a tree needs to be constructed. If there are N samples, there are randomly selected N samples to be replaced (randomly select one sample at a time, and then return to continue selection). The selected N samples are used to train a decision tree as the sample at the root node of the decision tree.  The ET is very similar to RF; they are both composed of many decision trees. The difference is that the RF obtains the best bifurcation attribute in a random subset, while ET obtains the bifurcation value completely randomly, so as to achieve the fork of the decision tree [30].
From the results of feature selection, we can see that when lateral velocity is used as a feature for lane-change detection, KNN has achieved good result, which is better than ETand RF. Therefore, the KNN model is designed using lateral velocity as feature for lane-change detection and the result is compared with ET and RF.
KNN makes predictions using the training dataset directly. Predictions are made for a new data point by searching through the entire training set for the K most similar instances (the neighbors) and summarizing the output variable for those K instances.
To determine which of the K instances in the training dataset are most similar to a new input, a distance measure is used. For real-valued input variables, the Euclidean distance is used as a distance measure method. Euclidean distance is calculated as the square root of the sum of the squared differences between a point a and point b across all input attributes i.
When KNN is used for classification, the output can be calculated as the class with the highest frequency from the Kmost similar instances. Each instance in essence votes for their class and the class with the most votes is taken as the prediction.
Class probabilities can be calculated as the normalized frequency of samples that belong to each class in the set of K most similar instances for a new data instance. For example, in a binary classification problem (class is 0 or 1), .
The KNN algorithm can be described as follows: (1) Initialize training sets and categories (2) Calculate the Euclidean distance between the test set sample and the training set sample (3) Sort the training set samples in ascending order according to the Euclidean distance (4) Select the first K training samples with the smallest Euclidean distance and count their frequency in each category (5) The category with the highest return frequency, that is, the test set sample, belongs to this category Table 4 shows the step of KNN algorithm, in which the list I z of its nearest neighbors is determined by calculating the similarity distance between the training object (x, y)∈I   , y), where x represents the training object, y represents the class to which it belongs, and x and y represent the test object and the class to which it belongs.

Evaluation Indicators.
When performing machine learning, the confusion matrix of prediction results can be described as shown in Table 5.
The number of pairs of samples divided by the number of all samples is the accuracy (ACC), which can be defined as Generally, the higher the accuracy, the better the classifier. However, in the case of imbalance between positive and negative samples, the accuracy is a big flaw as an evaluation indicator. It is not scientific and comprehensive to evaluate a model based on accuracy alone.
To evaluate the performance of machine learning more scientifically and comprehensively, precision (P), recall (R), and F1 score can be used.

Input:
Training object (x, y) ∈I and test object z � (x, y) Processing: Compute distance d � (x, y) between z and every object (x, y) ∈I. Select I z ⊆ I, the set of k closest training objects to z.

Output:
Computational Intelligence and Neuroscience The precision, which represents the proportion of positive examples that are actually classified as positive examples, can be defined as The recall, which measures how many positive examples are classified as positive examples, can be defined as P and R indicators sometimes conflict, so they need to be considered comprehensively. The most common method is F-Measure, which is the weighted harmonic average of P and R, and can be defined as When the parameter α � 1, it is F1: F1 combines the results of P and R. When F1 is higher, it can indicate that the model is more effective.
In addition, receiver operating characteristic (ROC) curve, in which the abscissa is False Positive Rate (FPR) and the ordinate is True Positive Rate (TPR), is also an important evaluation indicator. The definition of TPR is the same with P, and the FPR can be defined as The area under the ROC curve is called AUC. The prediction effect of a classification model can be evaluated based on the AUC value; the larger the AUC value, the better the performance of the model.

Experiments and Analysis.
The model is built on vscode using python and uses the scikit-learn framework. The experiments were performed on a server with a single-core CPU, 2.6 GHz, 2G memory, and Ubuntu 18.04.
KNN was performed on the selected dataset first. In order to choose the most appropriate number of neighbors, we trained and tested different numbers of neighbors on the dataset consisting of the data of all 14 vehicles and obtained their accuracy, respectively. The result of varying number of KNN neighbors is shown in Figure 5, from which we can see that the accuracy of the training set decreases as the number of neighbors increases, and at the same time the accuracy of the test set increases as the number of neighbors increases.
When the number of neighbors increases to 9, the accuracy of both the training and test sets remains stable. Therefore, 9 is appropriate to be determined as the number of neighbors.
After determining the number of neighbors, KNN model was designed and performed on the dataset consisting of the data of all 14 vehicles, compared with ET and RF. The ROC of designed models including KNN, ET, and RF is shown in Figure 6 and the AUC values of the three models are shown in Table 6. Obviously, from the ROC curves, the performance of KNN is better than RF and obviously better than ET. It can be seen more clearly in Table 6 that the AUC value of KNN is 97.73%, while the AUC values of ET and RF are 92.55% and 96.69%, respectively, showing that the performance of KNN is the best in these three models.
After experimental testing on the dataset of all 14 vehicles, in order to analyze the effect of continuous lanechange detection of vehicles in real scenes, the designed KNN model was used to perform experiments on the respective datasets of the 14 vehicles and compared with ET and RF.
The detailed sample sizes of lane-change behaviour on 14 selected vehicles are shown in Table 7 The dataset combining all 14 selected vehicles is divided into training and testing sets according to a ratio of 0.75 to 0.25. The confusion matrix of the detection results is shown in Figure 7, which illustrates that the KNN model has the highest detection accuracy, followed by RF, and ET has the lowest detection accuracy. In addition, detection errors mainly occur between the non-lane-change and the single lane-change behaviour, the probability of false detection between sequential lane-change behaviour and other behaviours is small, and the probability of false detection between left lane-change and right lane-change behaviour is also small. It is worth noting that, in the KNN and RF models, there is no misdetection between the left lane-change, right lane-change, and non-lane-change behaviour.
The experimental results of lane-change detection performed on 14 selected vehicles are shown in Table 8

Conclusions
This paper proposed a lane-change detection method for intelligent and connected vehicles. Based on the feature selection of vehicle lane-change behaviour, the detection model based on machine learning was designed, and the effect verification and comparison were performed on the selected dataset. The dataset based on NGSIM Interstate 80 Freeway Dataset was processed for lane-change detection first. After that, feature selection for lane-change detection was performed on the processed dataset, and the lateral velocity was selected as the feature for lanechange detection. Then, the lane-change model was analyzed based on the real data in the processed dataset and the detection model was designed. Finally, the number of KNN neighbors was determined based on experiment, and the performance of KNN, ET, and RF was analyzed by the evaluation indicators. From the experimental results, the designed KNN model performed best in all datasets of the selected 14 vehicles, with detection accuracy ranging from 89.57% to 100%, indicating that it can well complete the task of lane-change behaviour detection for ICVs.
As for future work, the lane-changing scene can be extended by the measured data from the vehicle sensors to establish a more widely adaptable dataset, and the detection model can be further optimized and then implemented on embedded hardware to achieve a lane-change real-time detection system for ICVs.

Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.