Unidirectional and Bidirectional LSTM Models for Short-Term Traffic Prediction

,

e strong interest in these models is that they can be used to provide road operators with predictive intelligence tools to help them optimize freeway operations and avoid traffic breakdowns.ese models have been developed from a variety of theoretical backgrounds including statistical techniques and artificial intelligence (AI) methods based on neural networks [9].With the development of big data and complex computational intelligence, AI methods can predict future traffic more accurately than statistical models.In particular, deep learning networks can represent traffic dynamic behaviour and have recently achieved massive success in time series modelling.An example of recent models is the unidirectional long short-term memory (Uni-LSTM) recurrent neural network and its extension bidirectional long short-term memory (BiLSTM).Previous research has shown that the Uni-LSTM model has an effective prediction in handling long-term dependencies as it remembers useful information from inputs that has already passed through using "additional gates" incorporated in its architecture [10][11][12].However, in more recent years, a bidirectional LSTM (BiLSTM) model has been investigated which offers an additional training capability as the output layer receives information from past (backwards) and future (forward) instances simultaneously and it provides better prediction accuracy [13][14][15][16].In this paper, we assess the performance of Uni-LSTM and BiLSTM for different time horizons using speed and flow field data for multiple freeways in Australia.
e main research questions are as follows: (1) Will results be improved if speed and flow data are trained from both directions (forward direction and backward direction)?(2) How adding layers or mixing both LSTM and BiLSTM as one model affects the model performance?(3) If the model is trained and tested for one freeway, will it achieve a good accuracy if validated only (without retraining) on an independent dataset from a different freeway?
is paper aims to address these questions and demonstrate the feasibility of using advanced AI techniques based on deep learning Uni-LSTM and BiLSTM models to predict speed and flow for multiple prediction horizons.It provides a comparative performance analysis of both Uni-LSTM and BiLSTM based on a common dataset of field measurements.e models are developed using historical data extracted from sensors embedded in pavements on three freeways in Australia: the Pacific Motorway between Brisbane and the Gold Coast in Queensland, Tullamarine Freeway in Melbourne, and South Eastern Freeway in Melbourne.is paper also investigates whether additional layers of training improve prediction accuracies for both speed and flow.To our knowledge, there have been limited papers targeting the BiLSTM model for future traffic prediction, and this paper shows the robust performance of this extension of the Uni-LSTM model.Also, this paper validates the performance of a developed model on different freeways which makes this work a valuable contribution to knowledge in the intelligent transport systems and network operations fields.is provides road operators and transport agencies with confidence that they can apply these models on different freeways even if they have not embarked on comprehensive historical data collection efforts for the target freeways.
is also helps them with reducing the cost of deployment of these algorithms by avoiding the need to preprocess new data and calibrate and validate new models which is a time-consuming undertaking that requires substantial resources and experienced and well-trained AI staff and specialists.
is paper is organized as follows: Section 2 provides a scan of previous research work.Section 3 presents the methodology including data collection and modelling frameworks.Section 4 presents the results of the comparative evaluation of different models.Section 5 presents the performance of stacked and mixed Uni-LSTM and BiLSTM models.Section 6 shows the model validation results, and finally, Section 7 presents the conclusions and future research directions.

Literature Review
e prediction and forecasting of short-term (1 to 60 minutes into the future) traffic conditions plays an important role in the success of intelligent transport systems (ITSs) such as travel information systems, adaptive traffic management systems, public transportation scheduling, and commercial vehicle operations [17][18][19].Due to the wide body of literature on this topic, we focus the literature scan in this section on traffic prediction using LSTM models, which used field traffic data collected from inductive loop detectors, CCTV, probe vehicles, and incident reports.A comprehensive review of other models including those that used simulated data can be found in [17].Increasingly, road operators have more confidence in models that have been developed using real-life data and hence our focus in this work on model development and evaluations using data from real-world environments.
ese parametric methods perform poorly with dynamic traffic patterns, which limits their application in complex traffic prediction compared to nonparametric methods.Nonparametric methods are more capable of predicting a stochastic pattern of input traffic data and are better at handling noisy data.
With the recent advancements in machine learning, many models have shown a promising potential in solving nonlinear problems and handling long-term dependencies.Examples include LSTM and BiLSTM models.ese models were previously used to forecast future traffic speeds [10], travel times [18], and traffic flows [11].In one study, the long short-term memory (LSTM) structure was applied for future speed prediction and showed that it provides higher performance compared to classical methods [10].Another study showed that using LSTM models is promising for irregular travel time prediction models as the error for 1-step-ahead prediction error is relatively small [18].Other studies have shown that flow prediction using LSTM achieved high accuracy compared to other models for different prediction horizons [11].Also, LSTM models have been developed in other studies on car-following models to predict acceleration and deceleration on different road hierarchies [26].
Short-term traffic flow using LSTM has also been investigated where the dependency relationships of time series data were fully considered, and experimental results showed a very good performance with an error of 5.4% when compared with other models [27].In other studies, an endto-end deep learning model has been investigated to predict future traffic flows [28] where one BiLSTM layer was added, and the results showed that the model was capable of solving stochastic flow characteristics and overcoming overfitting problems [28].Similarly, stacking BiLSTM and Uni-LSTM models were developed in another study to predict networkwide traffic speeds.
e results showed that the stacked architecture outperforms both BiLSTM and Uni-LSTM models [29].
In another study, different models were developed that showed superior performance when using deeper BiLSTM layers for urban traffic prediction [30].Other researchers have also used LSTM and RNN approaches for speed prediction models under various urban driving conditions with 2 Journal of Advanced Transportation credible and accurate results [31].LSTM and gated recurrent units (GRUs) were also applied in recent studies to predict the general condition of driving speed in consideration of the road geometry and temporal evolution of traffic demand.e results showed superior LSTM model performance compared to regression models [32].Similarly, superior model performance has been shown from using LSTM and GRU models when compared to ARIMA and support vector regression (SVR) models for the track flow prediction [33].
In other studies, a variational long short-term memory encoder was examined to predict traffic flow which provided better prediction in comparison to other conventional methods used [34].Similarly, a long short-term memorygenetic algorithm support vector regression (LSTM-GASVR) short-term traffic flow prediction algorithm was reported to predict future traffic flows with better accuracies than LSTM, GRU, convolutional neural networks (CNNs), stacked autoencoder (SAE), ARIMA, and support vector regression (SVR) models tested in the same study [35].
Furthermore, LSTM models have also been developed for momentary traffic stream forecasts, which aim to help transport authorities in decision-making during rush hour for gridlock prediction since the model remembers information for longer periods of time than other models [36].Also, the validity of LSTM models has been verified in studies on prediction of short-term traffic flow and were found to provide high prediction accuracies for flow data [37].Other studies have documented superior performance when ARIMA and long short-term memory (LSTM) neural networks were combined for short-term traffic flow prediction [38].In another recent study, type-2 fuzzy LSTM (T2F-LSTM) was developed for long-term traffic volume prediction and extraction of spatial-temporal characteristics of traffic volumes and was found to achieve high prediction accuracies [39].
In summary, a substantial number of studies in the literature have addressed short-term traffic prediction with robust LSTM models.However, only few studies have addressed the promising potential of BiLSTM models for traffic time series future prediction that consider the backward temporal dependencies.Another important contribution in this work is the comparison of stacked and mixed BiLSTM and LSTM layers for model accuracy improvement.In addition, this paper discusses the model applicability when being developed on parameters of one location and validated only (without retraining) on different locations.Furthermore, the models are developed using field data that comprised diverse and complex traffic characteristics (including peak, nonpeak, weekday, weekend, incident, and nonincident data).Another important factor in this work is that the data used for model development have been methodically screened, preprocessed, and validated in a large number of previous studies [40,41].

Study Methodology
is section of the paper presents the study methodology including data collection, model development, evaluation tests, and analyses.

Data for Model Development.
Neural network applications require large amounts of data that are needed for model development [41,42].e data are typically divided into a training dataset used for model calibration and a testing dataset used for model verification.e validity of the model is tested on an independent dataset not used in model training, referred to as the testing dataset.In this research, the data used for model development included traffic speed and flow measurements collected from sensors installed on a number of freeways in Australia.e data were collected over a number of years and time periods including peak, nonpeak, weekday, and weekend conditions.Another unique characteristic of the data is that they include incident traffic conditions which are usually difficult to capture for model development.Such incidents, which include road crashes, broken down vehicles, and similar nonrecurrent events, typically result in a significant capacity reduction of the freeway and last for long durations.Including these data in model training and validation improves the robustness of the prediction models.

Dataset 1: Pacific Motorway, Queensland.
is dataset was collected from a section of the Pacific Motorway between Brisbane and the Gold Coast in Queensland [40].e length of the section is around 1.5 km.Speed and flow data were gathered from 4 detection stations (S0-S3) which include inductive loop sensors installed at approximately 500 m interval as shown in Figure 1.ese data were collected for a period of 5 hours (2 hours peak and 3 hours offpeak traffic conditions).e data comprised normal traffic conditions that did not include any incidents.A total of 1,667 observations were gathered.For this study, the data were divided into 1000 observations for training (60% of the total dataset) and 667 observations for testing (40% of the total dataset).

Dataset 2: Tullamarine Freeway, Melbourne.
is dataset was collected from a section of the Tullamarine Freeway in Melbourne as shown in Figure 2. e freeway connects the city to the airport and the northern suburbs and is considered one of the busiest roads in Melbourne.e data comprised lane-by-lane loop-detector data consisting of speed, flow, and occupancy measurements in 1-minute cycles [41].Detector station spacing ranged between 450 m and 1070 m, with an average spacing of about 580 m for the 14 detector stations.A total of 21,123 observations were gathered, and for this research, they were divided into 60% for training (12,674 observations) and 40% for testing (8,449 observations). is dataset was unique in that it comprised a total of 75 incidents that occurred on the freeway under variable traffic conditions.While such data make it very challenging to be able to predict future conditions (because of the randomness of incidents), they are most useful as they allow the prediction algorithms to be trained on a wide range of traffic data including both incident and nonincident conditions.

Dataset 3: South Eastern Freeway, Melbourne.
is dataset was collected from a section of the South Eastern Freeway in Melbourne as shown in Figure 3. e data consisted of speed and flow measurements in 1-minute cycles, also collected from inductive loops embedded in the pavement.A total of 3,147 observations were gathered.For this work, the data were divided into 60% for training (1,888 observations) and 40% for testing (1,259 observations).Similarly, these data included a total of 25 incidents which would allow the prediction algorithm to be trained on a variety of traffic data including incident and nonincident conditions.e data from the two Melbourne freeways are also important in that each freeway carries more than 100,000 vehicles per day.
e incident data collected from these freeways (100 in total) had varying characteristics that included a representative range of incidents on freeways.For example, four incidents resulted in blocking one lane of traffic, 77 resulted in blocking 2 lanes, and 19 resulted in blocking three lanes.Five of the incidents occurred during low-flow conditions (below 700 vphpl), 58 during heavyflow conditions (above 1550 vphpl), and 37 during moderate flow conditions.Twenty-five incidents also occurred during peak-hour traffic conditions.As for the distribution of incident duration, 26 incidents lasted for less than 30 minutes; 32 lasted between 30 and 60 minutes; 30 lasted between 60 and 90 minutes; and 12 lasted more than 90 minutes [41].
Samples of speed and flow data for the three freeways are shown in Figures 4 and 5, respectively.ese figures represent a small portion of the data for illustrative purposes.Figure 4 shows speed patterns in km/h during AM peak between 5:30 and 8:00 AM. e figure demonstrates that the South Eastern Freeway is the most congested freeway with a speed lower than 20 km/h followed by the Pacific Motorway and Tullamarine Freeway.As for Figure 5, the flow patterns are illustrated in veh/h for the period from 9 AM to 12 PM.e figure shows that each freeway behaves differently during the same period of time as the flow ranges between 7 and 56 vehicles per hour for the three freeways.In summary, the reallife datasets used in this study are considered to be one of the most diverse and representative field traffic datasets available particularly in the Australian context.ey are also unique in that they have been meticulously screened, cleaned, preprocessed, and validated in a large number of studies [15].

Modelling Framework.
Unidirectional LSTM has received considerable attention in recent years for its superior performance compared to the state-of-the-art recurrent neural networks (RNNs).Even though RNNs provide good accuracy, they have been found to underperform for longterm memory as RNNs are unable to use information from distant past.Also, LSTM can learn patterns with long dependencies when compared with traditional RNNs that are not able to function for long-term patterns.erefore, LSTM has generally been found to outperform RNNs in time series data forecasting [43].e inclusion of additional data training has resulted in some model extensions of LSTM, now known as bidirectional LSTM (BiLSTM).is model trains the input time series data twice through forward and backward directions.e architectures of these models are presented in Figures 6 and 7.
In these models, the following formulae are used to calculate the predicted values: where ⊙ denotes the Hadamard product (element-wise multiplication of vectors).In this work, the unidirectional and bidirectional LSTM networks were implemented in MATLAB R2020b.First, the data were arranged as two column values: the first column corresponds to speed/flow at time t and the second column corresponds to the expected output (t + n) where n ranges from 5 minutes to 60 minutes into the future.en, the data were partitioned into training and testing sets.e models were trained on the first 60% of the sequence and tested on the last 40%.To prevent model overfitting, the training/ testing data were standardized to have zero mean and unit variance.e LSTM networks were created using four layers: sequence input layer (number of features � 1), Uni-LSTM/ BiLSTM layers (number of hidden units � 300), fully connected layer (number of responses � 1), and a regression layer.e model hyperparameter settings are presented in Table 1.Multiple sets of hyperparameters were tested with the aim to find the right combination of values which result in the best accuracy.Table 1 shows the parameters that provided the optimal results.e tanh and sigmoid functions were used for state and gate activation functions,

Comparative Evaluation of Uni-LSTM and BiLSTM
e first set of results in this paper was for speed and flow data for the Tullamarine Freeway in Melbourne (Table 2) and the Pacific Motorway in Brisbane (Table 3).e data from both freeways were divided into 60% training data and 40% testing data.
e mean absolute percentage error (MAPE) is used to calculate the accuracy of the model prediction for different time horizons.MAPE calculates the average absolute difference between the predicted output from the model (Y1) and expected true output (Y): e speed prediction results showed that BiLSTM and Uni-LSTM achieve high prediction results up to 60 minutes into the future.BiLSTM outperforms Uni-LSTM with accuracies above 92.6% up to 60 minutes for the Tullamarine Freeway.For a prediction horizon up to 60 minutes, accuracy improvements over Uni-LSTM were 7% for 5 minutes, 6% for 10 minutes, 7% for 15 minutes, 13% for 30 minutes, and 15% and 16% for 45 and 60 minutes, respectively.For the Pacific Motorway, BiLSTM outperforms Uni-LSTM up to 15 minutes, and then Uni-LSTM presents better results up to 60 minutes; however, the two models produce similar results for the 60-minute prediction horizon (e.g., 93.6% versus 92.7% as shown in Table 2. For the Pacific Motorway, the results showed that BiLSTM outperformed Uni-LSTM for up to 45 minutes with an accuracy improvement of 14% for 5 minutes; 14% for 10 minutes; 9% for 15 minutes; 8% for 30 minutes; and 2% for 45 minutes.For 60-minute prediction horizons, the percentage differences in accuracies between Uni-LSTM and BiLSTM were found to be minimal (0.01%) as reported in Table 3.In Tables 2 and 3, the cells highlighted in green denote best-performing models, and cells highlighted in yellow denote second best-performing models.

Deep and Mixed Unidirectional and Bidirectional LSTM
In this section, the results for multiple Uni-LSTM and BiLSTM layers to improve the results for both speed and flow are presented.Also, results for combining both LSTM and BiLSTM layers are presented for the 15minute horizon for the Tullamarine Freeway and Pacific Motorway.To our knowledge, limited publications have tested deep architectures of BiLSTM and mixed models to measure the backward dependency of traffic speed and flow prediction.e results provided in Tables 4 and 5 show that deep BiLSTM with combined layers outperforms Uni-LSTM and   4 and 5.
Figures 8 and 9 present the speed and flow results for the 15-minute prediction horizon using the best-performing 4-layered BiLSTM model.ese figures compare the target or expected values (blue trendline) with the predicted values from the model (orange trendline).Figure 8 shows the superior performance of the model for predicting both speed and flow with accuracies of 98% and 92.50% for 15-minute prediction horizons on the Tullamarine Freeway.Similarly, Figure 9 shows a remarkable prediction performance for the 4-layered BiLSTM model on the Pacific Motorway with a prediction accuracy of 99.99% for both speed and flow for 15-minute prediction horizons.

Deep BiLSTM Model Validation and Transferability
is section of the paper presents results for model validation and potential for transferability to other freeways without the need for recalibration and retraining.If this can be achieved even at the expense of a depreciated accuracy, can provide road operators and transport agencies with confidence that they can apply existing models on different freeways even if they have not embarked on comprehensive historical data collection efforts. is also helps them with reducing the cost of deployment of these algorithms by avoiding the need to preprocess new data and calibrate and validate new models which is a time-consuming undertaking that requires substantial resources and experienced and welltrained AI staff and specialists.e model validation experimental design is shown in Figure 10.e learning obtained from the previous comparative evaluations was used to develop robust speed and flow prediction models using data combined from the two e validation dataset included 1,667 observations from the third freeway (Pacific Motorway in Brisbane).e model development results are provided in the first set of columns in Table 6 (Tullamarine and South Eastern Freeways).For speed, the model accuracy ranged from 99.7% for 5-minute forecasting horizons to 91.8% for 60-minute forecasting horizons.For flow, the model accuracy ranged from 99.6% for 5-minute forecasting horizons to 71.2% for 60-minute forecasting horizons.e model was then validated (without retraining) on the third independent dataset from the Pacific Motorway in Brisbane.e validation results are shown in the right side columns of Table 6.For speed, the model's accuracy ranged from 99.7% for 5-minute forecasting horizons to 90.2% for 60-minute forecasting horizons.For traffic flow, the model's accuracy ranged from 97.2% for 5-minute forecasting horizons to 82.1% for 30-minute forecasting horizons.e performance degrades to 79.19% for 45-minute and 73.45% for 60-minute prediction horizons, as shown in Table 6.
ese findings are also depicted in Figures 11-16.
In Figures 11-16, the blue trendline represents the targeted real data compared to the orange trendline which represents the results generated from the model.In Figure 13, the difference between targeted and predicted results for both speed and flow is minimal as the MAPE percentages between the two are 0.27% and 2.83%, respectively.In Figure 14, the speed results for 10-minute prediction     Journal of Advanced Transportation

Conclusions, Contributions, and Future Research Directions
In this paper, unidirectional and bidirectional LSTM networks were developed to predict speed and flow on freeways for forecasting horizons up to 60 minutes into the future.e models were evaluated based on historical field data collected from inductive loop sensors on a number of freeways in Australia.A comprehensive and rigorous procedure was adopted to evaluate the suitability of different architectures and modelling parameters.e results showed a superior performance for the bidirectional compared to unidirectional LSTM.
e results also demonstrated the challenges of predicting traffic flow, compared to speed. is was a result of the noisy nature of flow measurements compared to speed observations.For the Tullamarine Freeway, the BiLSTM model was able to achieve speed predictions up to 60 minutes into the future with an accuracy above 90%.For the flow prediction, the accuracy was above 80% up to 45 min forecasting horizons, outperforming the Uni-LSTM model.For the Pacific Motorway, BiLSTM also outperformed Uni-LSTM with accuracies above 88% for speed and above 80% for flow up to 60-minute prediction horizons.
is study also extended the models and evaluated their performance when adding multiple Uni-LSTM and BiLSTM layers or mixing both LSTM and BiLSTM as one model for 15-minute prediction horizons.
e experiments showed that the 4-layered BiLSTM outperformed other models for both speed and flow on Tullamarine and Pacific Motorway datasets.Another contribution of this work was to examine model validation and its potential for transferability.e evaluation was undertaken on a combined dataset from the Tullamarine and South Eastern Freeways in Melbourne.is approach enabled us to train the models on a large dataset with different patterns and variable traffic conditions, including peak, nonpeak, weekday, weekend, and incident data.Once optimized, the model was validated by testing only (without retraining) on an independent dataset from a third freeway.e validation results showed speed prediction accuracies ranging from 99.7% for 5-minute forecasting horizons to 90.2% for 60-minute forecasting horizons.e flow validation prediction accuracies were lower and ranged between 74% and 97%.While it is acknowledged that more comprehensive testing is required on much larger numbers of freeways, this contribution demonstrates the potential to develop transferable models provided sufficient data are available to represent more diverse traffic conditions from different cities around the world.
Future directions in this research include collection of more field data from other real-life freeways in different cities both in Australia and overseas such as those reported in [44].e use of microsimulation to generate edge case data that are difficult to measure in the field is also  recommended [45,46].As was shown in this paper, training the models on different patterns and variable traffic conditions enabled us to develop robust models that can perform well on an independent dataset.With more data used for training and model development, it is expected that the accuracy will improve.e AI field is also witnessing a fast pace of developments and breakthroughs providing future opportunities to test new architectures to further improve the model performance and accuracy.

Figure 3 :
Figure 3: Schematic of the South Eastern Freeway.

Figure 9 :
Figure 9: Pacific Motorway speed and flow prediction results for 15-minute horizons using the 4-BiLSTM model.

Figure 14 :
Figure 14: Speed and flow validation results for 30 minutes.

Figure 16 :
Figure 16: Speed and flow validation results for 60 minutes.
Figure 4: Samples of speed data for the three freeways.

Table 2 :
Speed performance for different prediction horizons for the Tullamarine Freeway.

Table 3 :
Flow performance for different prediction horizons for the Tullamarine Freeway.Journal of Advanced Transportation largest datasets (Tullamarine and South Eastern Freeways in Melbourne).e data used in model development included 24,270 observations and were divided into two sets: training set comprising 60% of the data (14,562 observations) and testing set comprising 40% of the data (9,708 observations).

Table 4 :
Speed and flow performance for different prediction horizons for the Tullamarine Freeway.

Table 5 :
Speed and flow performance for different prediction horizons for the Pacific Motorway.Figure 8: Tullamarine Freeway speed and flow prediction results for 15-minute horizons using the 4-BiLSTM model.

Table 6 :
Speed and flow validation results for different prediction horizons.Figure 11: Speed and flow validation results for 5 minutes.Figure 12: Speed and flow validation results for 10 minutes.Figure 13: Speed and flow validation results for 15 minutes.
Figure 15: Speed and flow validation results for 45 minutes.