An improved Bayesian fusion algorithm (BFA) is proposed for forecasting the blink number in a continuous video. It assumes that, at one prediction interval, the blink number is correlated with the blink numbers of only a few previous intervals. With this assumption, the weights of the component predictors in the improved BFA are calculated according to their prediction performance only from a few intervals rather than from all intervals. Therefore, compared with the conventional BFA, the improved BFA is more sensitive to the disturbed condition of the component predictors for adjusting their weights more rapidly. To determine the most relevant intervals, the grey relation entropy-based analysis (GREBA) method is proposed, which can be used analyze the relevancy between the historical data flows of blink number and the data flow at the current interval. Three single predictors, that is, the autoregressive integrated moving average (ARIMA), radial basis function neural network (RBFNN), and Kalman filter (KF), are designed and incorporated linearly into the BFA. Experimental results demonstrate that the improved BFA obviously outperforms the conventional BFA in both accuracy and stability; also fatigue driving can be accurately warned against in advance based on the blink number forecasted by the improved BFA.
1. Introduction
Fatigue driving is one of the major causes of serious accidents in transportation. Statistics show that traffic accidents caused by driver fatigue account for about 20% of the total number of accidents and more than 40% of serious traffic accidents [1, 2]. Experts agree that the actual contributions of fatigue driving to road accidents may be much higher [3]. Frequent road traffic accidents, serious casualties, and property losses caused by fatigue driving of drivers bring a heavy burden to the society and families. Thus, the accurate and efficient detection of driver fatigue is very important [4].
Many methods have been developed for detecting the driver’s fatigue recently, including the measurements of physiological features detection like EEG [5, 6], heart rate variability (HRV) [7, 8], and electrooculogram (EOG) [9], facial features detection such as blink [10, 11] and yawning [12, 13], and vehicle behaviors detection, for instance, lane deviation [14, 15] and steering angle movement [16, 17]. Among those methods, the physiological features based methods are intrusive for drivers, because these methods need to place detection electrodes on the skin, drivers will feel uncomfortable if these electrodes are in touch with the skin for a long time. Vehicle behaviors based methods are easily affected by different sizes and shapes of vehicles, as well as different driving habits of drivers [18]. Facial characteristics based methods especially blink based detection methods using machine vision are attracting more and more researchers [18–20]. The objective of the study is to propose a new forecasting algorithm of blinking number and further verify the effectiveness of the proposed algorithm according to the forecasting results.
For the fatigue driving detection methods based on eye blinking, almost all researchers pay more attention to real time detection algorithms that can fast detect opening or closing state of the eyes and correctly make recognition of whether a driver is fatigued currently. However, the detection methods usually are not able to meet the requirement of real time processing. It is computationally costly in detecting and recognizing fatigue characters, especially for fusion detection methods based on multiple fatigue features [21] which require detecting and recognizing many more characteristic parameters [22]. Moreover, statistics show that if drivers’ responses are only half a second faster when traffic accidents occurred, sixty percent of traffic accidents can be avoided. Therefore, if fatigue driving can be warned against in advance before traffic accidents happen using a certain detection instrument or forecasting method, mostly traffic accidents can be avoided [22, 23]. Furthermore, physiological researches show that fatigue is a gradual process where drivers experience from consciousness to drowsiness [24]. Therefore, if we develop a predictor that can forecast the blink number of the driver in a given time interval according to previous time intervals, then we can recognize the fatigue driving in advance and also save more time for fatigue feature extraction and detection algorithm enhancement. In a word, the advantage of using predictor method is that we can know and estimate the future driving state in advance according to the past and current states as long as we set appropriate time intervals. Meanwhile, we do not need to detect the driving state of drivers in real time, which can save more computation time for other more important processing tasks.
Currently, many successful applications of single predictor based forecasting algorithms have been reported from various fields, for instance, in the fault diagnosis [25, 26], transportation flow forecasting [27], time-series prediction [28], and so on. Based on this, a variety of methods has been put forward for goal state prediction, including the Kalman filtering (KF) model [29], nonparametric regression model [30], and autoregressive integrated moving average (ARIMA) model [31]. Generally, these prediction methods can be categorized as statistical time series analysis methods, which conduct their predictions based on historical data analysis. One benefit of statistical time series analysis methods is that they can make very good predictions when the goal state varies temporally. However, due to linear properties, these methods are inadequate for capturing the rapid variations of goal state. To overcome this problem, numerous studies have used machine learning methods such as artificial neural networks (ANN) [32, 33] and support vector machines (SVM) [34] as alternative predictors. A machine learning method is able to approximate goal state in any degree of complexity without prior knowledge of problem solving. In addition, because of its ability to learn from data, it can capture the underlying relationships of goal state even when they are not apparent.
Though the previously mentioned prediction approaches are powerful and useful methods for goal state prediction and can generate accurate results for certain patterns, they each have their own drawbacks in dealing with predictions. None of them can maintain excellent prediction performance under all application conditions [35, 36]. Generally, driver’s fatigue is related to not only the duration time of continuous driving but also the current time period. For example, drivers feel fatigued more likely at 3:00–5:00 and 14:00–16:00 than at other time periods, and drivers usually do not feel fatigued until after three hours of continuous driving. Additionally, the fluctuation of blink number of drivers from nonfatigue to fatigue is very obvious and abrupt when detected by a device or instrument. Therefore, forecasting algorithm based on a single predictor is hardly suitable for the abrupt fluctuation of blink number when drivers continuously drive for a long time. This result occurs because goal state generally exhibits a spatiotemporal behavior characterized by irregular randomness, and it is very difficult for a single prediction method to capture such a disturbed pattern.
In view of these deficiencies, some researchers have turned to multivariate modeling in which models are developed by combining multiple methods to take advantage of the merits of each algorithm. A feasible fusion method that could effectively combine the predictions of single predictors is the Bayesian fusion algorithm (BFA) proposed by Petridis et al. [37]. The BFA generates a prediction by a weighted fusion of forecasts of its all component predictors based on posterior probabilities and the Bayesian rule. However, the BFA pays no attention to the relevance between the historical data flows and the current data flow. It assumes that, at a particular interval, the weights of the component predictors in the BFA depend on the cumulative prediction performance of all past intervals. This may result in making the prediction quite impervious to the greatly fluctuated prediction accuracy of the component predictors. To overcome this problem, an improved BFA has been proposed in this paper. The underlying assumption is that the data flow of blink number at a particular prediction interval is only affected by the data flows from only few of the previous time intervals, which have a comparatively higher relevancy with it. Based on this assumption, the prediction errors of only a few intervals of the component predictors are required to be considered when calculating their weights.
Simulation experiments of fatigue driving find that the improved BFA is more sensitive to the fluctuation of the component predictors and can adjust their weights more rapidly compared to the conventional BFA and also can obtain better forecasting accuracy and stability. Therefore, it can be used to judge in advance whether drivers are fatigued by the blink number forecasted in next time period.
2. Shortcoming of Conventional BFA
The conventional Bayesian fusion algorithm was originally proposed by Petridis et al. [37]. Its general idea is summarized as follows.
Let yt denote the actual blink number at time interval t. Then, we have(1)yt=ytnyt-1,yt-2,…,y1+etn,where ytn is the nth predictor, ytn(yt-1,yt-2,…,y1) is the predicted blink number at time interval t by the nth predictor, and etn is the corresponding prediction error.
For a certain time interval, we can use Z to denote the uncertainty. The posterior probability of the nth predictor to be the best model at time interval t is defined as(2)ptn=probZ=n/yt,yt-1,…,y1.
Then, according to Bayesian rule, we can obtain(3)ptn=probyt,Z=n/yt-1,…,y1∑m=1Nprobyt,Z=m/yt-1,…,y1,where N is the total number of component predictors; notice that(4)probyt,Z=n/yt-1,…,y1=probyt,Z=n,yt-1,…,y1probyt-1,…,y1=probyt,Z=n,yt-1,…,y1probyt-1,…,y1,Z=n×probyt-1,…,y1,Z=n∑m=1Nprobyt-1,Z=m/yt-1,…,y1=probyt/yt-1,…,y1,Z=npt-1n.
Assuming that the prediction error etn=yt-ytn is a Gaussian white noise time series with zero mean and standard deviation σn, we have(5)probyt/yt-1,…,y1,Z=n=probetn=yt-ytn/yt-1,…,y1,Z=n=12πσnexp-yt-ytn2σn2.
Combining (3), (4), and (5) yields (6)ptn=1/2πσnpt-1nexp-yt-ytn/2σn2∑m=1N1/2πσmpt-1mexp-yt-ytm/2σm2.
Then, at time interval t+1, the prediction of the conventional BFA can be written as the linear combination of the output of all the predictors as follows:(7)y^t+1=∑n=1Nptnyt+1n,where y^t+1 is the prediction generated by the conventional BFA and yt+1n is the result estimated by the nth predictor.
Similar to (6), pt-1n, n=(1,2,…,N) is formulated as(8)pt-1n=1/2πσnpt-2nexp-yt-1-yt-1n/2σn2∑m=1N1/2πσmpt-2mexp-yt-1-yt-1m/2σm2.
Combine (6) and (8) and we have(9)ptn=1/2πσn2pt-2nexp∑i=01-yt-i-yt-in/2σn2∑m=1N1/2πσm2pt-2mexp∑i=01-yt-i-yt-im/2σm2.
Similarly, we can formulate explicit expressions for pt-2n,pt-3n,…,p1n. Substituting them into (9), we have(10)ptn=1/2πσnt-1exp∑i=0t-1-yt-i-yt-in/2σn2∑m=1N1/2πσmt-1exp∑i=0t-1-yt-i-yt-im/2σm2.
From (10), we can see that, rather than only depending on the prediction error at prediction interval t, the weight ptn depends on the prediction errors of all past intervals. This characteristic makes the conventional BFA very inert to the fluctuating accuracy of component predictors. If the dominant predictor is no longer the most accurate, it will take many intervals to reduce the dominant status of that predictor, thus imposing a negative impact on the predictions of the conventional BFA.
This problem arises because the conventional BFA does not consider the correlation between the historical data flows and the data flow at the prediction interval. Generally, only the data flows of blink number at the latest few intervals may strongly correlate with the data flow at a given prediction interval. For the data flows of blink number at earlier time intervals, they may have less impact on the current data flow.
3. Improved Bayesian Fusion Algorithm3.1. Selection of Correlative Dada Flow
For most of the forecasting methods, the selected blink data flows from only a few intervals are considered. Inspired by this idea, this paper hypothesizes that, at a particular prediction interval, the data flow of blink number is only affected by the past few intervals’ data flows which have comparatively higher relevance with it. With this assumption, the weight of each component predictor in the BFA at a given prediction interval would just depend on the accumulative prediction performance of a few selected intervals rather than all intervals. Based on this assumption, (1) can be rewritten as (11)yt=ytnyk+etn;∀k∈K,where K represents the set of the previous intervals at which the data flows of blink number have a comparatively higher correlation with the data flow at prediction interval t.
Following the same inferential procedures from (2) to (6) and from (9) to (10), we have(12)ptn=1/2πσnRKexp∑k∈K-yk-ykn/2σn2∑m=1N1/2πσmRKexp∑k∈K-yk-ykm/2σm2,where R(K) represents the dimension of set K. From (12), we can see that the weights of the component predictors just depend on their prediction errors at the previous R(K) intervals. Therefore, theoretically, the weights calculated by (12) are more sensitive to the fluctuated accuracy of component predictors, and the number of data flows to be used to calculate is also reduced greatly. Substituting it into (7), we can obtain the prediction of the improved BFA.
3.2. Grey Relation Entropy-Based Analysis
According to the discussion in the previous section, one challenge to implementing the improved BFA is how to identify the time interval set K where the data flows of blink number have comparatively higher correlation with the data flows at the prediction interval. Because (12) includes a nonlinear, complex exponential function, the regression method is not applicable here. An alternative method is the grey relation entropy-based analysis (GREBA) [38, 39]. It has been extensively applied for relevance analysis in various disciplines, such as logistics [40], economics [41], and engineering [42]. In this paper, we use this method for relevance analysis between current and historical data flows of blink number.
Assuming that, at the prediction interval t, blink number yt is affected by its previous w intervals’ data flows, listed as Y(t)=(yt-1,yt-2,…,yt-w+1), w should be large enough to cover most of the important data flows. We use Yt(j)=(yt-1,yt-2,yt-j,…,yt-N) and Yt-i(j)=(yt-i-1,yt-i-2,yt-i-j,…,yt-i-N) express the target data flow time sequence at the tth interval and the alternative data flow time sequence at the (t-i)th interval, respectively; i={1,2,…,M}, M is the amount of the alternatives data flow time sequence; j={1,2,…,N}, N is the length of the target and alternative data flow time sequences; Yt(j)⊂Y(t) and Yt-i(j)⊂Y(t). Here, the sequences of Yt(j) and Yt-i(j) are comparable because they are obtained from the same data sequence Y(t); therefore, it is unnecessary to normalize the original data. The grey relation coefficient between Yt(j) and Yt-i(j) is then calculated as(13)ψYtj,Yt-ij=miniminjYtj-Yt-ij+ρmaximaxjYtj-Yt-ijYtj-Yt-ij+ρmaximaxjYtj-Yt-ij,where ρ is the distinguishing coefficient in the range of [0,1] and it can be adjusted to make a better distinction between the target sequence and the alternative sequence.
To satisfy the rule of grey entropy, the grey relation coefficients should be transformed into a grey relation density γ(t,t-i), calculated by(14)γt,t-i=ψYtj,Yt-ij∑j=1NψYtj,Yt-ij;i=1,2,…,M;j=1,2,…,N.
Once the grey relation density γ(t,t-i) is obtained, the entropy of the grey relation coefficient of each alternative Yt-i(j), i=(1,2,…,M), is then computable, which represents the relevance degree of the grey relation coefficient. The calculation is shown below as(15)Et-i=-∑j=1Nγt,t-iInγt,t-iInN;i=1,2,…,M,where -∑j=1Nγt,t-iIn(γ(t,t-i)) is the grey entropy between the target data flow time sequence Yt(j) and the alternative data flow time sequences Yt-i(j) and In(N) is the maximum grey entropy, which guarantees 0≤E(t-i)≤1.
Finally, by multiplying the entropy E(t-i) and the average grey relation coefficient of alternative Yt-i(j), we can obtain the grey relevancy grade (GRG). In this study, the GRG is defined as the numerical measurement of the relevancy between the alternative data flow time sequence Yt-i(j) and the target data flow time sequence Yt(j), which is calculated as(16)Dt-i=Et-iN∑j=1NψYtj,Yt-ij,where D(t-i) is the GRG of the alternative time sequence Yt-i(j) with respect to the target sequence Yt(j). It is distributed between 0 and 1. Based on the definition of the GRG, it can be seen that the higher the D(t-i) is, the more relevance there is between the alternative data flow time sequence Yt-i(j) and the target data flow time sequence Yt(j).
3.3. Procedures of the Improved BFA
Based on the above discussions, the procedures of the improved BFA are described in Figure 1 and detailed steps are as follows.
Flowchart of improved BFA.
Step 1 (predictor calibration and validation).
Use the historical data flow of blink number to calibrate the coefficients for each component predictor and then forecast the data flows that will be tested with the trained predictors.
Step 2 (set K identification).
Assume that, at the prediction interval, the data flow is potentially correlated with the latest wth intervals’ data flows. Then, use the GREBA method to calculate the GRG of each alternative data flow time sequence Yt-i(j). Ranking the alternative data flow time sequence Yt-i(j) according to their GRGs in decreasing order, the intervals of the first R(K) alternative data flow time sequences are identified as the set K.
Step 3 (BFA prediction).
Predict the data flows using the improved BFA where the weights of the component predictors are calculated by (12).
4. Component Predictors Selection4.1. ARIMA Model
In theory, ARIMA model is the most common models for time series forecasting. It formulates its mathematical model to fit the time series based on the Box-Jenkins autoregressive moving average (ARMA) model [43]. It is often referred to as ARIMA(p,d,q) model, where p and q are the factors of AR (autoregressive) and MA (moving average), respectively, while d is the difference frequency to make time series stationary [44–46]. The model can be written as(17)ΦB1-Bdy^t=ΘBεt,where y^t denotes the predicted data flow at tth interval. B is the backshift operator defined by B(y^t)=yt-1; Φ(B)=1-φ1(B)-φ2(B2)-⋯-φp(Bp) is the AR operator of order p, where φ1, φ2, …, and φp are the autoregressive coefficients; Θ(B)=1-θ1(B)-θ2(B2)-⋯-θq(Bq) is the MA operator of order q, where θ1, θ2, …, and θq are the moving average coefficients; εt is the random error which meets normal distribution with mean zero, variance σ2, and cov(εt,εt-m)=0, for all m≠0.
The procedures to formulate ARIMA model are summarized in Figure 2: firstly, AR and MA operator are used to identify whether the time series is a stationary sequence. If not, we difference the data to make it stable. Secondly, the maximum likelihood estimation method is used to evaluate the parameters in ARIMA. Then the resulting model is tested. We diagnose whether the residual error of the time series is white noise or not. If it is not, we reestimate the parameters until the generated residual error turns out to be white noise. Finally, with the suitable parameters, the ARIMA could be applied for prediction [47].
Flowchart of ARIMA modeling.
4.2. Radial Basis Function Neural Network Model
ANN models are formulated by emulating the processes of the human neurological system. Among numerous ANN models, the radial basis function neural network (RBFNN) is perhaps the best model, which is often considered as an alternative for data flow forecasting [48]. The RBFNN is a feed-forward neural network with strong local nonlinear approximation ability and fast learning rate. It contains three layers, including the input layer, hidden layer, and output layer. Its structure is presented in Figure 3 [49]. There are k neurons in the input layer; therefore, input data X can be expressed by a k-dimension column vector, X={xt,xt-1,…,xt-(k-1)}, where xt is the blink number at current interval t. There are i neurons in the hidden layer, which can be expressed using radial basis function R(X), R(X)={R1(X),R2(X),…,Ri(X)}. There is one neuron in the output layer, which is used to express the prediction value x(t+1) at next interval t+1. x(t+1)=f(X)=∑p=1iwp,qRp(X), where, p={1,2,…,i}, q=1, wp,q is the weight value between the pth neuron in the hidden layer and the neuron in the output layer.
Structure of RBF neural network.
4.3. Kalman Filtering Model
KF is an estimator that provides a set of mathematical equations to estimate the state of a process by minimizing the mean of the squared error. The filter is very powerful in estimating past, present, and even future states. With its high computational efficiency, it has been applied to numerous fields such as process control, flood prediction, tracking, and navigation [50–52]. Assuming that t is the time interval index, y^t and yt represent the predicted and observed data flow at the current interval, respectively. Let Χt=[yt,yt-1,yt-2,…,yt-(h-1)]T. According to KF theory, the predicted data flow y^t can be formulated as measurement equation:(18)y^t=HtΧt+νt,where νt represents the measurement noise with normal probability distributions p(v)~N(0,R) and Ht is measurement matrix. Χt is the state vector at interval t which is updated as(19)Χt=ΦtΧt-1+ωt-1,where Χt-1 is the state vector at interval t-1, ωt-1 is the process noise with normal probability distributions of p(ω)=N(0,Q), and Φt is the transition matrix.
According to KF theory, state equations can be expressed as follows:(20)Χ^t=HtΧ^t-1+Gtyt-HtΦtΧ^t-1,Gt=pt∣t-1HtTHt·pt∣t-1·HtT+R-1,pt∣t-1=Φtpt-1ΦtT+Q,pt=I-Gt·Φtpt∣t-1.The prediction equation is (21)y^t+1=Ht+1Φt+1Χ^t.To simplify the forecasting procedures and improve the accuracy of KF method, the following tips are adopted when it is applied for data flow prediction: (1) the dimension of state matrix is decided by the ARIMA model, and it is set to be the number of the history data flows that is most correlative with the current data flow; (2) the transition matrix Φt is set to be identity matrix; (3) let Ht=[1,0,0,…,0]T; (4) the measurement noise covariance matrices R can change with each time step or measurement, but for simplicity it is assumed to be a constant decided by sample data of blink number; (5) let Χ^0=E[Χ0] and p0=var[Χ0].
5. Experiments and Analysis5.1. Experiment Setup
It is very dangerous for subjects to drive in real highway when they feel fatigued; therefore, the experimental data of blink number are acquired by driving simulation experiments in laboratory. The platform of our driving simulation experiment includes hardware and software. The hardware consists of driving simulation equipment, computer, scene display device, video collection card, CCD camera, and so forth. The software is comprised of virtual driving scene and visual detection program of blink number. Experiments are carried out in bright lighting condition. Twenty subjects without any diseases or drinking before the experiment, including 12 male subjects and 8 female subjects who all have more than three years of driving experience, are allowed to take part in this experiment.
5.2. Blink Number Calculation
In this paper we select RE as an indicator of open-close state of eyes and it can be calculated by (22). Whether driver’s eyes are open or closed is estimated according to the size of RE and the set threshold TRE. If RE≥TRE, then it will be inferred that the driver is conscious; otherwise, the driver is deduced to be blinking. Consider(22)RE=HEWE.
Physiological studies show that blinking number increases and the duration of the blinking is longer when a driver feels fatigued. Therefore, it is very reasonable that fatigue driving is deduced by the blinking number NB in given time intervals, where NB is the frame number of the images where driver’s eyes are recognized as being closed. If NB exceeds the threshold set, then fatigue driving is determined [53].
5.3. Data Collection
In this study, eight hours of blink data each day were collected on driving simulator from 1:00 to 5:00 and from 12:00 to 16:00. Whether drivers are blinking or not is detected and judged by visual detection algorithms based on eye state recognition every 100 milliseconds. A time interval of prediction is set to be five minutes. Four-day data set is separated into two parts. The first part including the data from the first three days is used as historical data. The second part including the data from the fourth day is used for model testing and comparison. The blink data collected in five-minute interval from 1:00 to 5:00 and from 12:00 to 16:00 on the first day are plotted in Figures 4(a) and 4(b), respectively. Figures 4(c) and 4(d) show blink data from 1:00 to 5:00 and from 12:00 to 16:00 on the second day. From Figure 4, we can see that the blink numbers from 3:00 to 5:00 and from 14:00 to 16:00 are much higher than those of other periods. In addition, blink data in midnight and noon contains two peak value, including the midnight peak value (about from 3:30 to 4:20) and the noon peak value (about from 14:40 to 15:20). The observed blink data from 1:00 to 2:30 and from 12:00 to 13:30, however, exhibits no apparently peak value. The testing data is collected from the fourth day, including both peak value data and nonpeak value data. Thus we can compare the prediction performance between the conventional and improved BFA under different data flow pattern.
Two-day blink data in 5-minute interval: (a) blink data collected in five-minute interval from 1:00 to 5:00 on the first day; (b) blink data collected in five-minute interval from 12:00 to 16:00 on the first day; (c) blink data collected in five-minute interval from 1:00 to 5:00 on the second day; (d) blink data collected in five-minute interval from 12:00 to 16:00 on the second day.
5.4. Construction of Three Component Predictors5.4.1. ARIMA
The first three-day data from the first part are used for model identification, which is to determine the orders of AR, difference, and MA. The procedure is discussed in Section 4.1. First of all, the autocorrelation test shows that the partial correlations of the original sample data decay very slowly. Then we difference the sample data to make the sample data become a stationary time sequence. The corresponding test shows the data differenced is stable. Following that, the differenced data is used to identify the orders of AR and MA. With parameter estimation; finally, ARIMA(4,1,0) model is identified to be the most suitable model with statistical packages.
5.4.2. RBFNN
Blink data collected in the first three days is used for network training. RBFNN is selected with the following architecture: five neurons in the input layer, one hidden layer with p neurons, and one output neuron. At a prediction interval, the latest five blink data, xt, xt-1, xt-2, xt-3, and xt-4 are used for the input data. Gaussian function is chosen as the radial basis function in the hidden layer; that is, Rp(X)=exp-X-cp2/2(σp)2, where cp is the center of Gaussian function and σp is the width. During the training RBFNN, cp and p are calculated by using k-means clustering algorithm [54] and σp=dmax/2l, where dmax is the largest distance between each two cp that has been obtained.
5.4.3. KF
KF was used for forecasting by extracting the most recent blink data measured. As is mentioned before, the orders of AR are regarded as the number of previous blink data that the KF is based on. Therefore, the dimension of Ht in (18) is 1×4.
5.5. Evaluation Criterion of Prediction Performance
We select and employ three evaluation criterions, that is, the mean absolute error (MSA), the mean relative error (MRE), and variance of relative error (VRE), to evaluate the prediction performance of predictors. Evaluation criterions are calculated by (23). Among these criterions, MSA and MRE are used to measure the prediction accuracy, while VRE is used to present the prediction stability. Consider(23)MSA=1S∑t=1Sy^t-yt,MRE=1S∑t=1Sy^t-ytyt,VRE=S∑t=1Syt-y^t/yt2-∑t=1Syt-y^t/yt2SS-1,where yt is the observed blink number in time interval t, y^t is the predicted blink number in time interval t, and S is the number of prediction intervals.
5.6. Predictions of Component Predictors
After being trained with the blink data of the first three days, the three component predictors are used to predict the blink number of next observation period on the fourth day. Statistics show that drivers usually feel the most fatigued in the period from 1:00 to 5:00 on each day, even though they have good sleep on the last day, and the blink data flow in the period has an obvious fluctuation. Therefore, we select the blink data from 1:00 to 5:00 to test prediction performance of each predictor. Figure 5 describes the outputs of the three component predictors for blink number in the period from 1:00 to 5:00. The corresponding prediction performances are summarized in Table 1. Table 1 indicates that, despite all component predictors successfully making predictions as a whole, no single predictor can retain the best prediction performance all the time. It can be observed from Table 1 that while the ARIMA predictor made relatively better predictions in the period from 1:00 to 2:30 than RBFNN and KF, it delivers the poorest prediction performance in the period from 2:30 to 3:30. Therefore, it is necessary to find ways to integrate the merits of the single predictors to improve the prediction accuracy and stability.
Prediction performance of the three component predictors.
ARIMA
RBFNN
KF
MSA
MRE (%)
VRE (%)
MSA
MRE (%)
VRE (%)
MSA
MRE (%)
VRE (%)
1:00–2:00
4.25
7.51
8.20
5.32
7.82
8.53
6.26
6.35
7.18
2:00–3:00
19.60
8.23
9.07
10.42
7.51
8.02
15.66
8.17
9.08
3:00–4:00
15.53
8.04
9.12
10.07
7.62
7.99
17.72
8.28
9.24
4:00–5:00
22.71
7.93
8.11
15.87
7.49
7.93
23.72
8.32
8.77
Total
16.13
8.09
9.14
12.17
7.71
8.29
16.33
8.21
9.19
Observed and predicted blink number of the three component predictors on the fourth day from 1:00 to 5:00.
5.7. Comparison of Prediction Performance of the Improved BFA and the Conventional BFA
Prior to applying the improved BFA for blink number forecasting, the set K should be determined by using the GREBA method. We assumed that the blink number at the prediction interval (Yt(j)) is potentially affected by the blink number from the last 10 intervals (50 minutes) (Yt-1(j),Yt-2(j),…,Yt-10(j)). Our aim is to identify a few intervals at which the blink number has comparatively higher relevance to the blink number at the prediction intervals. Once determined, the prediction errors of each of the component predictors at these intervals will be used to decide their weights in the improved BFA. The GRG between each alternative (Yt-1(j),Yt-2(j),…,Yt-10(j)) and the target blink number time sequences Yt(j) is described in Figure 6. It demonstrates that although there are some perturbations between adjacent intervals, the GRG decreases with respect to decreasing time intervals (t-1,t-2,…) in a longer period, which implies that the further the time gets away from the current prediction intervals, the less relevance there is between the corresponding blink number and the current blink number. This result coincides with our hypothesis that the blink number is very complicated and usually exhibits irregular randomness. It is affected by subject’s sleep situation, continuous driving time, road traffic conditions, and subject’s psychological or physiological conditions, and only a few of the latest blink data are competent enough to dynamically observe the blink number in prediction interval. Note that, in Figure 6, the GRG ceased to decrease dramatically after three intervals, the blink number at the past 3 intervals is thus identified to be the value that mostly correlates with the blink number at forecasting interval; that is, K={t-1,t-2,t-3}.
The GRG between the alternative data flow time sequences (Yt-1,Yt-2,…,Yt-10) and the target data flow time sequences Yt of blink number.
The prediction performances of the improved BFA and the conventional BFA are listed in Table 2. Tables 1 and 2 imply that while the conventional BFA improves the prediction performance of single predictors to some extent, it does not compare to the improved BFA. Table 2 indicates that the improved BFA considerably outperforms the conventional BFA in both accuracy and stability. For example, the improved BFA has a performance of 4.16% MRE and 3.52% VRE in the period from 1:00 to 2:00 and a 26.11% improvement for MRE and a 30.01% improvement for VRE over the conventional BFA, which has a MRE of 5.63% and a VRE of 5.03%. Even the improved BFA has its worst performance in the period from 2:00 to 3:00, with 7.99% MRE and 12.11 MSA; it is still better, however, than the corresponding performance of the conventional BFA. Another insight gained from the comparison of conventional BFA and the improved BFA in Tables 1 and 2 is that the performances of both methods depend heavily on the performance of the component predictors that each method combines. Thus, both BFAs perform better if each component predictor is more accurate.
Prediction performance of the conventional BFA and the improved BFA.
Conventional BFA
Improved BFA (K={1,2,3})
MSA
MRE (%)
VRE (%)
MSA
MRE (%)
VRE (%)
1:00–2:00
5.03
5.63
5.03
3.28
4.16
3.52
2:00–3:00
16.33
8.05
9.14
12.11
7.99
8.75
3:00–4:00
13.21
7.03
8.77
9.03
6.87
7.89
4:00–5:00
15.94
8.09
9.26
10.04
7.01
8.37
Note that the prediction performances of both the conventional BFA and the improved BFA rely heavily on the weights of the component predictors calculated by (10) and (12). Because the weight in the BFA characterizes the estimated possibility that a particular predictor best forecasts the observed blink number, whether the BFA can assign more weight to the “actual” best predictor at each prediction interval is thus very crucial to its prediction accuracy.
Figure 7(a) shows that the weight of the ARIMA predictor assigned by the conventional BFA is the highest from 1:00 to 2:30 and then decreases slowly and finally approaches 0 after 2:30. The weight of the RBFNN predictor increases monotonically over time and eventually dominates the predictions of the conventional BFA. It happens because the predictor with the best average performance changed during the day, from the ARIMA predictor to the KF predictor. However, upon inspection, we find that the weight calculated by the conventional BFA is unable to dynamically monitor the disturbed accuracy of each component predictor. For example, Figure 5 shows that the RBFNN predictor delivers the best predictions in the periods from 2:30 to 3:30 and from 4:20 to 5:00. However, during the periods, the weight of the RBFNN assigned by the conventional BFA is not the highest of the three, as shown in Figure 7(a). Nevertheless, the extraordinary prediction performance of the RBFNN predictor is indeed detected by the conventional BFA, given that the weight of the RBFNN predictor increases monotonously during this period. However, because the weights of the component predictors at every prediction interval in the conventional BFA are accounted for by their prediction errors from all past intervals, the increased scope is very limited. Once the weight of RBFNN predictor finally has a leading role, it is also difficult to make a change even if there are comparatively larger prediction errors for RBFNN predictor during a period. This effect could be justified by the predictions in Figure 5 in the period from 3:30 to 4:20, during the period the RBFNN predictor makes much worse predictions than the KF predictor. However, Figure 7(a) shows that the weight of the RBFNN predictor assigned by the conventional BFA is dramatically larger than that of the KF predictor, which may lead to inaccurate predictions. For our improved BFA, there is no such deficiency because the weight of the component predictors at the current interval is only affected by their prediction errors from the latest three intervals. Therefore, the improved BFA is more sensitive to the fluctuated accuracy of the component predictors and can more quickly adjust their weight. Figure 7(b) shows that, in the periods from 2:30 to 3:30 and from 4:20 to 5:00, when the RBFNN predictor makes the best predictions, its weight assigned by the improved BFA is always the highest of the three, and when it is no longer the best predictor in the period from 3:30 to 4:20, its corresponding weight decreases dramatically; thus, better predictions are generated compared to the conventional BFA (Figure 7(c)).
Forecasted results of the conventional BFA and the improved BFA in the period from 1:00 to 5:00: (a) evolution of the weights assigned to each component predictor by the conventional BFA; (b) evolution of the weights assigned to each component predictor by the improved BFA; (c) comparison of the predictions of the conventional BFA and the improved BFA.
Figure 7(a) also reveals that, in most cases, none of single predictors can achieve the best performance during all prediction periods. However, our improved BFA is sensitive to the disturbed accuracy of each component predictor and can assign more weight to the best predictor. Therefore, it is definitely more preferable than the conventional BFA.
Based on the forecasted blink number, further experiments that can be used to identify whether drivers are fatigued or not show that correct rate of fatigue driving detection can reach 98.4%; false-alarm rate is only 1.9%. Therefore, fatigue driving can be accurately warned against in advance based on the blink number forecasted by the improved BFA.
6. Conclusions
For forecasting fatigue state of drivers in advance, we propose an improved BFA to enhance the accuracy and stability of blink number forecasted in next time interval. By analyzing the correlation between historical data flows and current data flows, the improved BFA only considers the prediction errors from the latest few intervals when calculating the weights of each component predictor. Thus, it is more sensitive to the disturbed accuracy of the component predictors and can adjust their weights more quickly than the conventional BFA. To make full use of the advantages of various methods, ARIMA, BPNN, and KF are designed and fused linearly into the BFA. Simulating experiments demonstrate that the improved BFA outperforms the conventional BFA in both accuracy and stability considerably. We found that the conventional BFA is quite impervious to the disturbed accuracy of the component predictors. Most of the time, it assigns nearly all weight to one component predictor. Compared with the conventional BFA, our improved BFA can more rapidly adjust the weights of the component predictors and generate better predictions. Therefore, the improved BFA is more preferable for blink number prediction than the conventional BFA.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The research has been supported in part by the National Nature Science Foundation of China (nos. 61304205, 61203273, 61103086, and 41301037), Nature Science Foundation of Jiangsu Province (BK20141002), the Open Funding Project of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University (no. BUAA-VR-13KF-04), Jiangsu Ordinary University Science Research Project (no. 13KJB120007), and Innovation and Entrepreneurship Training Project of College Students (nos. 201510300228, 201510300276).
HuS. Y.ZhengG. T.PetersB.Driver fatigue detection from electroencephalogram spectrum after electrooculography artefact removal20137110511310.1049/iet-its.2012.00452-s2.0-84879388112LalS. K. L.CraigA.BoordP.KirkupL.NguyenH.Development of an algorithm for an EEG-based driver fatigue countermeasure200334332132810.1016/S0022-4375(03)00027-62-s2.0-0041691376LalS. K. L.CraigA.Driver fatigue: electroencephalography and psychological assessment200239331332110.1017/s00485772013930952-s2.0-0036251023AhlstromC.NyströmM.HolmqvistK.ForsC.SandbergD.AnundA.KecklundG.ÅkerstedtT.Fit-for-duty test for estimation of drivers' sleepiness level: eye movements improve the sleep/wake predictor201326203210.1016/j.trc.2012.07.0082-s2.0-84865833172SimonM.SchmidtE. A.KincsesW. E.FritzscheM.BrunsA.AufmuthC.BogdanM.RosenstielW.SchraufM.Eeg alpha spindle measures as indicators of driver fatigue under real traffic conditions201112261168117810.1016/j.clinph.2010.10.0442-s2.0-79956344170JapB. T.LalS.FischerP.BekiarisE.Using EEG spectral components to assess algorithms for detecting fatigue20093622352235910.1016/j.eswa.2007.12.0432-s2.0-56349131350LiG.ChungW.-Y.Detection of driver drowsiness using wavelet analysis of heart rate variability and a support vector machine classifier20131312164941651110.3390/s1312164942-s2.0-84889019005PatelM.LalS. K. L.KavanaghD.RossiterP.Applying neural network analysis on heart rate variability data to assess driver fatigue20113867235724210.1016/j.eswa.2010.12.0282-s2.0-79951579523HsiehC.-S.TaiC.-C.An improved and portable eye-blink duration detection system to warn of driver fatigue201341542944410.1080/10739149.2013.7965602-s2.0-84882433962JoJ.LeeS. J.ParkK. R.KimI.-J.KimJ.Detecting driver drowsiness using feature-level fusion and user-specific classification20144141139115210.1016/j.eswa.2013.07.1082-s2.0-84888363097D'OrazioT.LeoM.GuaragnellaC.DistanteA.A visual approach for driver inattention detection20074082341235510.1016/j.patcog.2007.01.018ZBL1111.686372-s2.0-34147139802DuY.HuQ.-H.ChenD.-G.MaP.-J.Kernelized fuzzy rough sets based yawn detection for driver fatigue monitoring20111111657910.3233/fi-2011-554MR28950792-s2.0-84863049571WangT. S.ShiP. F.Yawning detection for determining driver drowsinessProceedings of the IEEE International Workshop on VLSI Design and Video TechnologyMay 20053853882-s2.0-23844472675TunçerÖ.GüvençL.CoşkunF.KarsligilE.Vision based lane keeping assistance control triggered by a driver inattention monitorProceedings of the IEEE International Conference on Systems, Man and Cybernetics (SMC '10)October 201028929710.1109/icsmc.2010.56422542-s2.0-78751488444PiluttiT.Galip UlsoyA.Identification of driver state for lane-keeping tasks199929548650210.1109/3468.7841752-s2.0-0032691275HeQ.LiW.FanX.Estimation of driver’s fatigue based on steering wheel angle20116781Berlin, GermanySpringer145155Lecture Notes in Computer Science10.1007/978-3-642-21741-8_17SayedR.EskandarianA.Unobtrusive drowsiness detection by neural network learning of driver steering2001215996997510.1243/09544070115285362-s2.0-0035735676SahayadhasA.SundarajK.MurugappanM.Detecting driver drowsiness based on sensors: a review20121212169371695310.3390/s1212169372-s2.0-84871702859SchleicherR.GalleyN.BriestS.GalleyL.Blinks and saccades as indicators of fatigue in sleepiness warnings: looking tired?2008517982101010.1080/001401307018170622-s2.0-46249112789ErikssonM.PapanikolopoulosN. P.Driver fatigue: a vision-based approach to automatic diagnosis20019639941310.1016/s0968-090x(00)00045-02-s2.0-0035546698LeeB.-G.ChungW.-Y.Driver alertness monitoring using fusion of facial features and bio-signals20121272416242210.1109/JSEN.2012.21905052-s2.0-84861898531DongY.-C.HuZ.-C.UchimuraK.MurayamaN.Driver inattention monitoring system for intelligent vehicles: a review201112259661410.1109/tits.2010.20927702-s2.0-79958144996YangG.LinY.BhattacharyaP.Multimodality inferring of human cognitive states based on integration of neuro-fuzzy network and information fusion techniques2008200837162110.1155/2008/3716212-s2.0-36949004998WangL.PeiY.The impact of continuous driving time and rest time on commercial drivers' driving performance and recovery201450111510.1016/j.jsr.2014.01.0032-s2.0-84893980333GadsdenS. A.SongY.HabibiS. R.Novel model-based estimators for the purposes of fault detection and diagnosis20131841237124910.1109/TMECH.2013.22536162-s2.0-84880571267KimD.KangP.ChoS.Machine learning-based novelty detection for faulty wafer detection in semiconductor manufacturing2012254814823JeongY.-S.ByonY.-J.Castro-NetoM. M.EasaS. M.Supervised weighting-online learning algorithm for short-term traffic flow prediction20131441700170710.1109/TITS.2013.22677352-s2.0-84890433467OuyangY.YinH. J.A neural gas mixture autoregressive network for modelling and forecasting FX time series201413517117910.1016/j.neucom.2013.12.0372-s2.0-84897911959TamuraH.BacopoulosP.WangD. B.HagenS. C.KubatkoE. J.State estimation of tidal hydrodynamics using ensemble Kalman filter201463455610.1016/j.advwatres.2013.11.0022-s2.0-84896922800LorcaP.LandajoM.de AndrésJ.Nonparametric quantile regression-based classifiers for bankruptcy forecasting201433212413310.1002/for.2280MR32330472-s2.0-84894903305Contreras-ReyesJ. E.PalmaW.Statistical analysis of autoregressive fractionally integrated moving average models in R20132852309233110.1007/s00180-013-0408-7MR31073032-s2.0-84884701605EbrahimiE.BayatH.NeyshaburiM. R.AbyanehH. Z.Prediction capability of different soil water retention curve models using artificial neural networks201460685987910.1080/03650340.2013.8372192-s2.0-84892986379AzadehA.GhaderiS. F.SheikhalishahiM.NokhandanB. P.Optimization of short load forecasting in electricity market of iran using artificial neural networks201415248550810.1007/s11081-012-9200-8YanX.ChowdhuryN. A.Mid-term electricity market clearing price forecasting: a multiple SVM approach20145820621410.1016/j.ijepes.2014.01.0232-s2.0-84893872426KirbyH. R.WatsonS. M.DoughertyM. S.Should we use neural networks or statistical models for short-term motorway traffic forecasting?1997131435010.1016/S0169-2070(96)00699-12-s2.0-0031092111SmithB. L.WilliamsB. M.OswaldR. K.Comparison of parametric and nonparametric models for traffic flow forecasting200210430332110.1016/s0968-090x(02)00009-82-s2.0-0036692982PetridisV.KehagiasA.PetrouL.BakirtzisA.KiartzisS.PanagiotouH.MaslarisN.A Bayesian multiple models combination method for time series prediction2001311–3698910.1023/a:10120618142422-s2.0-0035343067ZhangQ. S.DengJ. L.ShaoY.Grey correlation analysis by the method of degree of balance and approach199523119498MR1400328WangZ.WangQ.AiT.Comparative study on effects of binders and curing ages on properties of cement emulsified asphalt mixture using gray correlation entropy analysis20145461562210.1016/j.conbuildmat.2013.12.0932-s2.0-84893474567ZhangY.ZhangR.-C.Study on the third party logistics service providers' performance evaluation based on the weighted entropy and analysis process of grey relationProceedings of the 17th International Conference on Management Science and Engineering (ICMSE '10)November 2010Melbourne, AustraliaIEEE58258710.1109/icmse.2010.57198612-s2.0-79952904440LiuC. F.DangG. F.Related analysis of the evolvement of land utilize structural and social economic development based on the information entropyProceedings of the 8th International Conference on Information and Management SciencesJuly 2009351354XiaX.MengF.Grey relational analysis of measure for uncertainty of rolling bearing friction torque as time series20112321351442-s2.0-79959875177LeeY.-S.TongL.-I.Forecasting time series using a methodology based on autoregressive integrated moving average and genetic programming2011241667210.1016/j.knosys.2010.07.0062-s2.0-77957994377TsuiW. H. K.Ozer BalliH.GilbeyA.GowH.Forecasting of hong kong airport’s passenger throughput201442627610.1016/j.tourman.2013.10.0082-s2.0-84887570221WyattT. M.DejongW.DixonE.Population-level administration of alcoholedu for college: an ARIMA time-series analysis201318889891210.1080/10810730.2011.6265012-s2.0-84881370075KimH.-W.LeeJ.-H.ChoiY.-H.ChungY.-U.LeeH.Dynamic bandwidth provisioning using ARIMA-based traffic forecasting for Mobile WiMAX20113419910610.1016/j.comcom.2010.08.0082-s2.0-78149470958HanP.WangP. X.ZhangS. Y.ZhuD. H.Drought forecasting based on the remote sensing data using ARIMA models20105111-121398140310.1016/j.mcm.2009.10.0312-s2.0-77950862145NiuH. L.WangJ.Financial time series prediction by a random data-time effective RBF neural network201418349750810.1007/s00500-013-1070-22-s2.0-84897580817TelmoudiA. J.TlijaniH.NabliL.AliM.M'HiriR.A New rbf neural network for prediction in industrial control201211474977510.1142/S02196220125001982-s2.0-84865965335BaraldiP.MangiliF.ZioE.A Kalman filter-based ensemble approach with application to turbine creep prognostics201261496697710.1109/TR.2012.22210372-s2.0-84870554246PapadimitriouC.FritzenC.-P.KraemerP.NtotsiosE.Fatigue predictions in entire body of metallic structures from a limited number of vibration sensors using Kalman filtering201118555457310.1002/stc.3952-s2.0-80051519014LoukaP.GalanisG.SiebertN.KariniotakisG.KatsafadosP.PytharoulisI.KallosG.Improvements in wind speed forecasts for wind power prediction purposes using Kalman filtering200896122348236210.1016/j.jweia.2008.03.0132-s2.0-49749138923SunW.ZhangX. R.ZhuangW.TangH.Driver fatigue driving detection based on eye state201151030731410.4156/jdcta.vol5.issue10.362-s2.0-80355147243OhS. K.KimW. D.PedryczW.JooS.-C.Design of K-means clustering-based polynomial radial basis function neural networks (pRBF NNs) realized with the aid of particle swarm optimization and differential evolution201278112113210.1016/j.neucom.2011.06.0312-s2.0-82655162109