GNSS / Low-Cost MEMS-INS Integration Using Variational Bayesian Adaptive Cubature Kalman Smoother and Ensemble Regularized ELM

Among the inertial navigation system (INS) devices used in land vehicle navigation (LVN), low-cost microelectromechanical systems (MEMS) inertial sensors have receivedmore interest for bridging global navigation satellites systems (GNSS) signal failures because of their price and portability. Kalman filter (KF) based GNSS/INS integration has been widely used to provide a robust solution to the navigation. However, its prediction model cannot give satisfactory results in the presence of colored and variational noise. In order to achieve reliable and accurate positional solution for LVN in urban areas surrounded by skyscrapers or under dense foliage and tunnels, a novel model combining variational Bayesian adaptive Kalman smoother (VB-ACKS) as an alternative of KF and ensemble regularized extreme learning machine (ERELM) for bridging global positioning systems outages is proposed. The ERELM is applied to reduce the fluctuating performance of GNSS during an outage. We show that a well-organized collection of predictors using ensemble learning yields a more accurate positional result when compared with conventional artificial neural network (ANN) predictors. Experimental results show that the performance of VB-ACKS is more robust compared with KF solution, and the prediction of ERELM contains the smallest error compared with other ANN solutions.


Introduction
Active safety systems and self-driving cars are a promising solution to reduce the number of traffic accidents [1,2].Some advanced driver assistance systems (ADAS) such as adaptive cruise control (ACC), collision warning system (CWS), and emergency braking system (EBS) that already exist in series of land vehicles are able to warn the driver and even to intervene in the state of the vehicle when a hazardous traffic situation is being developed.With the development of various driver assistance systems using vehicle position, the improvement of positioning accuracy is highly desired.Global navigation systems (GNSS), widely used as a positioning system, can provide accuracy of a few meters in suburban areas in order to provide an accurate trajectory or position of vehicles which is crucial for collision warning/avoidance.However, in urban areas or under dense foliage, this accuracy can be degraded to ten meters or more because of the reflection and blocking of global positioning system (GPS) signals by tall buildings.In these environments, signal may be difficult to acquire or the number of satellites available may be not sufficient to provide position information [2].Thus, a backup or alternative system is required to mitigate the impact of GPS outages.
Hybridization of GNSS with an inertial navigation system (INS) is one of the best solutions for bridging the GNSS gaps.Among the INS devices used in land vehicle navigation (LVN), low-cost microelectromechanical systems (MEMS) inertial sensors have received more interest because of their price and technology enabling compact and portable size [3].MEMS-INS and GNSS are commonly integrated using Kalman filter (KF) to provide a robust navigation solution, overcoming situations of GPS satellite signals blockage.However, it was shown that long-term GPS outages can be crucial and the INS position accuracy degrades also with time.Additionally, most of the methods proposed in the literature point that the traditional Kalman filter is the main 2 Mathematical Problems in Engineering reason of the failure of the integration of GNSS/INS as its prediction model is not adapted for nonlinear stochastic problems.It is also shown that applying adaptive filters and smoothers techniques to the GNSS/INS integrated navigation system could obtain better estimated performance than by using conventional KF [4,5].Vehicular navigation is often characterized with dynamics changes in motion and the use of KF could negatively have an impact on the position accuracy due to its inappropriate stochastic models and its inability to solve nonlinear system problems with variational and colored noise properties [5].As a consequence of the aforementioned KF drawbacks, different approaches based on Bayesian filtering [6][7][8][9] and artificial intelligence (AI) techniques [3,[10][11][12][13][14][15][16] have been proposed to improve the effectiveness of the integration methodology in bridging GPS outages.
(1) Research Objectives.In this research, we propose the implementation of ensemble regularized extreme learning machine (ERELM) based GNSS/MEMS-INS integration methodology to improve the accuracy of predictor and achieve better stability through training a set of ELM models and then combining them for final predictions.Unlike the traditional ANN techniques, the extreme learning machine (ELM) offers significant advantages such as fast learning speed and ease of implementation and the ensemble scheme we use is able to avoid the problem of local minima, stopping criteria, and overfitting.Moreover, in ensemble learning, the overfitting can be useful for ELMs.We use the adaptive boost regression and threshold (AdaBoost.RT) scheme which is one of the most popular techniques for generating ensemble ELM models due to its adaptability, its simplicity of implementation, and its ability to handle strong nonlinearity and random variations of dynamic data series.Therefore, to represent the sequential process that describes the input data of GNSS/MEM-INS integration and deal with variational noise, the combination of a variational Bayesian based adaptive cubature Kalman smoother (VB-ACKS) and ERELM is proposed to estimate INS errors.
(2) Related Works.Recently, several techniques based on artificial neural networks (ANNs) have been proposed to replace the function of KF in order to overcome some disadvantages.As a result of their ability to handle the problem of nonlinearity, some approaches such as those based on multilayer perceptron neural networks (MLPNNs), radial basis function neural networks (RBFNNs) [10,11], and adaptive neural fuzzy inference system (ANFIS) [12] and others based on random forest regression (RFR) [13] and least square support vector machine (LS-SVM) [14] were reported for GNSS/MEMS-INS systems.To overcome the drawbacks associated with the use of GPS and INS in a standalone manner, two recent AI methods were reported in the literature.The first was proposed by Malleswaran et al. combined recurrent neural network (RNN) with two evolutionary algorithms such as particle swarm optimization (PSO) and genetic algorithm (GA) in order to integrate GNSS and INS.Despite the level of prediction accuracy added by the weight optimization techniques, it could be computationally burdensome for navigation systems.Another GNSS/INS technique is based on Bhatt et al. [15] which uses a Dempster-Shafer theory combined with support vector machine (DS-SVM) [16].The DS based theory is used for the INS and GPS data fusion and the SVM for modeling INS error.Although DS-SVM learning algorithm performs very well for GPS outages prediction, a critical limitation relies on the fact that finding the correct pair of position samples as required by the technique could be time-consuming.All of the above techniques and models were shown to be able to have good performance due to their great power to model the nonlinear relationship.However, in dealing with the GPS outages problem, these AI techniques have not considered some important aspects of ANN such as the accuracy of their architectures, the problem of weightdecay, and local minima problem.Furthermore, most of them suffer from slow convergence rates, thus limiting their applicability and capability for predicting accurately land vehicles navigation.Saadeddin et al. [17] and Chen and Fang [18] proposed an adaptive method based on ANN and a hybrid prediction method based on radial basis function (RBF), respectively, to overcome the limitation of Kalman filter and the authors focused on the improvement of the predictors integrated in the kernel of the ANN.Combining several predictors as demonstrated by ANFIS, RFR are generally more powerful than using single predictor.However, the accuracy of these techniques degrades when applying lowcost MEMS-INS due to their architecture that does not assure generalization performance [19,20].The use of ensemble learning models stands as the best solution because of its performance which is much higher in terms of prediction accuracy compared to the global and local individual models.ELM is a freshly emerging learning architecture that provides fast prediction solutions.ELM stands out from other AI methods because of the following particular characteristics: extremely fast training, good generalization, universal approximation capability, and excellent learning accuracy.It has many advantages, and not only avoids many problems encountered by traditional gradient-based neural network learning algorithms such as local minima and various training parameters (training efficiency, stopping criteria, learning epochs, and the hidden layer unit number), but also learns much faster, with higher generalization performance than the established gradient-based learning methods [19].In ELM, the hidden nodes are randomly generated and this is done without an iterative tuning procedure.Unlike the traditional learning techniques, ELM can generate the hidden node parameters before seeing the training data [20].ELM requires lesser training time and it is hence remarkably efficient in terms of computational cost.An important aspect of ELM resides in the activation functions that enable us to achieve universal approximation capability.Unlike traditional AI methods, it does not need to calibrate the parameters.ELM enables us to alleviate several challenging issues in AI such as local minima, trivial human intervention, and time consumption in learning.As mentioned earlier, combining a number of learning machines can reduce the risk of overfitting and lead to better generalization performance.Ensemble neural network methods are more and more desirable due to the basic fact that the selection of the weights represents in itself an optimization problem with many local minima [21][22][23].Ensemble extreme learning machine methods have received much more attention because of their threefold advantages.First, it allows improving the accuracy of predictors.Second, it alleviates the overtraining problem.Third, it enhances the predictive stability of ELM [21].
The remainder of this paper is organized as follows.Section 2 gives a description of the system overview for GNSS/INS integration which includes the INS mechanization process and the computation of the navigation parameters as well as the novel variational Bayesian based adaptive cubature Kalman smoother used for GNSS/INS integration.Section 3 describes the basic RELM and the ERELM based on AdaBoost.RT algorithm.Section 4 describes the methodology for GNSS/INS integration used in this study and the new prediction model used during GPS outages.Section 5 presents the experimental results of the ERELM model and its comparison with the existing ANNs.Finally, Section 6 contains the conclusions and recommendations for future work.

System Model
The proposed GPS outages bridging technique is realized in low-cost 2D land vehicle tracking system using reduced inertial measurement unit integrated with GPS.

Computation of Navigation Parameters.
Before the preprocessing or postprocessing steps, the GNSS and MEM-INS data have to be converted and mechanized by the conventional methods as described in [3].The vehicles position, velocity, and attitude that will be later used as input in the AI algorithms can be obtained by the integration of the INS measurements.These measurements are transformed and converted with the well-known coordinate transformation system earth-centered, earth-fixed (ECEF) reference frame.The measured specific force related to the time rate of change of velocity with respect to the ECEF frame is written as where V is the cosine rotation matrix from body frame to the ECEF frame (-frame) and Ω  is the skew matrix of the earth's angular velocity.  represents the vehicle's position.  is the vehicle's velocity with respect to the ECEF frame and finally   is the local gravitation vector.2Ω    is the coriolis acceleration.Ω 2   is the centrifugal acceleration.The position of the vehicle with respect to the -frame can be obtained by integrating the vehicle's velocity as shown in the following equation: The attitude with respect to the -frame is represented in the quaternion vector as The time rate of change of the attitude of the vehicle is given as follows: and ( 4) is integrated numerically to obtain the attitude of the vehicle at any given time.The vehicle's angular velocity is where   is the angular velocity of the vehicle relative to the inertial frame as measured by the INS.  is the angular velocity of the earth relative to the inertial frame.This equation is represented in the body frame as follows: and its skew-symmetric cross matrix is then represented as The INS position, velocity, and attitude of the vehicle, obtained by integrating ( 1), (2), and (3) quickly, drift away from the correct position and velocity of the vehicle due the bias error associated with the IMU measurements.

MEM-INS Measurements Model. The MEMS-INS mea-
surements' model can be represented as in the following equations: where f is the measured vehicle specific force in the body frame, π  is the measured angular velocity of the vehicle relative to the inertial frame measured in the body frame of the vehicle,    is the bias error in the specific force measurement,   is the bias in the angular velocity measurement,    is the process noise on the specific force measurement, and   is the process noise on the angular velocity measurement [8].

Variational Bayesian Adaptive Cubature Kalman
Smoother.After the mechanization process, both MEMS-INS and GNSS parameters are integrated using the traditional KF.However, since the system in land vehicle navigation presents is nonlinear, the use of a smoother is highly required.Furthermore, land vehicle navigation exhibits dynamics changes in motion and the use of KF could negatively impact the position accuracy due to its inappropriate stochastic models and its inability to solve nonlinear filtering problems with variational or colored noise properties.These nonlinear systems can be solved by applying variational Bayesian adaptive cubature Kalman smoother (VB-ACKS) based on the cubature rule which is a variation of Gaussian approximation based smoothing algorithm [4,5].As depicted in Figure 1 equation describing the nonlinear discrete-time dynamical systems is given as follows: where   is the state vector of the vehicle,   is the measurement vector, (⋅) and ℎ(⋅) are the state and measurement matrices, respectively, of nonlinear system, and V −1 and  −1 are noise samples from two independent zero-mean Gaussian processes with covariances  −1 and  −1 , respectively.In order to get the CKS, let us define the posterior density where   is the vector of measurements at time .Our nonlinear filtering algorithm can be obtained using five steps described as follows.
In order to make our filter adaptive to colored or variational noise, we use the heuristic as in [4], defined by a dynamic model and obtained using an approximation of the product of a Gaussian term and inverse Wishart (IW) term.We consider the dynamic model that can be expressed as follows: where  is a parameter which controls the dynamics 0 <  ≤ 1 and  is a matrix, 0 < || ≤ 1.  = 1 corresponds to stationary covariance and lower values allow for higher time-fluctuation.Assume that the state and the covariance estimate at time  are obtained in advance using Gaussian distributions.

Extreme Learning
x(t − 1) x(t − i) x(t − t  ) where   is the weight vector connecting the input nodes and the th hidden node,   is the bias of the th hidden node,   is the weight vector connecting the th hidden node and the output nodes,   is the th training example,   is the corresponding output of   in the network, and "⋅" denotes the dot product of two vectors.An approximation of the SFLNs can be made.One has ∃  ,   ,   , such that (30) can be written in a simple form of matrix where is often called the hidden layer output matrix.

Regularized ELM.
In the traditional feed-forward network training methods, such as the back-propagation (BP) or MLP algorithms, all the parameters, including the weight vectors and bias values, need to be tuned iteratively, thus the training speed may be very slow.As mentioned previously, what is more, they also suffer from other problems, such as local minima and overfitting.The ELM algorithm is summarized around three important steps which consist of generating random weights and bias for each hidden node, calculating the hidden output matrix of hidden layer , and calculating the output weight .It was shown that RELM can produce much more consistent results than the ELM.In order to get an optimized solution of the output weight, the ELM is often regularized using the residual  between the target value and output of sample and structural ‖‖ 2 .This process can be expressed as follows [19,23]: where  = diag( 1 ,  2 , . . .,   ) is a matrix that can weight error variable   = [ 1 ,  2 , . . .,   ] using weighting factors   to obtain the best generalization performance by the optimal tradeoff between these two risks, and the Lagrangian for (31) can be written as follows: The optimality conditions can be obtained; the partial difference equation group set equals zero as follows: We can get a simplified expression of  by substituting (33) into (32), as follows: RELM is summarized in Algorithm 1. shown that they could perform better than when using individual ELM.In this paper, more recent boosting algorithms for regression problems called modified-AdaBoost regression and threshold (modified-AdaBoost.RT) [21,22] are used to improve the performance of single RELM as an ensemble method.The original sample is divided into many samples and fed to every learner of the ensemble model.The weak learners are generated to evaluate which of them has the best prediction accuracy.The entire process of learning is run using a certain number of iterations .At each step of the iteration, a distribution is assigned to every RELM weak learner that is called to provide it with the distribution, and the associated errors are calculated.A regression model is built within the process to minimize the errors of the output provided by the weak learners.In this work, we chose to apply the SW-RR which enables the usingof the past inputs in order to reduce errors on the current outputs [24,25].Each learner calculates the absolute relative error for each training and updates the distribution.The structure of an ensemble RELM model is shown in Figure 3. Based on the description above, we derive ERELM algorithm for GNSS/MEMS-INS integration as shown in Algorithm 2.

GNSS/INS Integration Scheme.
In this section, the ERELM/VB-ACKS architecture for GNSS/INS integration is described.In order to achieve the integration of INS and GNSS data, we adopted a loosely coupled integration strategy [8].GPS measurements are processed and fused with INS for its error computations [10].The output of the mechanization is depicted in Figure 4; that is, the INS data is fused with the raw GNSS data using the VB-ACKS filter.The output of this fusion can provide accurately the position of a land vehicle in case of nonsignal blockages of GNSS.However, when the GNSS encounters signals blockages, the mechanized output that provides the INS data is trained by the ERELM method.
The INS and GNSS data fused by VB-ACKS are selected as input for predicting the position of the vehicle using the ensemble ELM model.

Prediction Process.
For simplicity of implementation, we denote by  in =  GI the input sample and  out the desired output of the prediction model.According to Algorithm 2, the input and ouput samples can be divided into  samples as follows: , where (  in ,   out ) represents the th input and output samples position, velocity, and attitude of the vehicle.The prediction starts with the training ERELM input samples (position, velocity, and attitude) provided by the past VB-ACKS outputs.These output samples are chosen as the inputs of ERELM while the current VB-ACKS output is considered as the desired output of ensemble RELM model.When GPS outages occur, the trained ERELM is employed as the replacement of VB-ACKS to predict INS errors.The predicting process is dynamic because each ERELM output at current time will be provided for predicting the output of next moment.The position of a land vehicle is predicted by the ERELM as follows: Given a training set consisting of position error samples at time  denoted  in (e.g.,  GI ) which represents the differences of position, velocity, and attitude between INS and GNSS.The training set is partitioned into  samples ( 1 in ,  for  = 1, . . .,  do ( is the total number of samples) (5) Use steps ( 1)-( 12) to compute the final prediction.(6) end for (7) until GPS outages happen.in Figure 3.Each sample is given as input to the initialized weak learners and the number of iterations  is chosen.After initializing the weak learner and the error rate for each learner and setting the distribution for each weak learner, the RELMs are called to train the position, velocity, and attitude of samples.Within the same step, a regression model is built and the absolute relative error of the training is calculated.The error rate of each regression and the distributions are updated using equations ( * * ) and ( * * * ) in Algorithm 2, respectively.The contribution of each training is evaluated and the final hypothesis is calculated.When GPS outages occur, the ensemble model chooses the learner that has the least error rate.The EREM model that combined the AdaBoost.RT is established to predict the position of the vehicle at time.The prediction process is shown in Figure 5. Based on the analytical study above, we derive ERELM algorithm for GNSS/MEMS-INS integration as described in Algorithm 3.

Parameters of the GPS/INS Integrated Navigation Model.
To evaluate the performance of the proposed scheme, the dataset collection was made using OBD-II coupled with a GPS u-blox LEA-6T and G-force data logger based on arduino MEGA2560, which serves as a MEMS-INS device.
The inertial measurement unit is sampled at 50 Hz and consists of three accelerometers and three gyroscopes to get vehicle's three axes angular velocity rate, three axes gyro rate, and three axes acceleration rate.The tests were conducted in land vehicle environments.The equipment used for data collection is shown in Figure 6.The trajectory of a vehicle as depicted in Figure 7 includes all the real-world scenarios, that is, U-turn, S-turn, or straight line and dynamics encountered in a typical LVN environment.

Results and Discussion
. The assessment of the performance of our proposed algorithms for GNSS/MEMS-INS integration was made in two manners.Firstly, we tested the performance of the VB-ACKS and compare it with the cubature Rauch Tung Striebel Smoother (CRTS) [5] and the traditional KF.Figures 8, 9, and 10 show the estimated errors of the position, velocity, and attitude filtered by KF, CRTS, and VB-ACKS.As it can be seen in these figures, the VB-ACKS approach demonstrates its superiority compared with the KF and the CRTS, due to the fact that the VB-ACKS is able to deal with the nonlinear formulation and the variational noise, while the linear model used by the KF does not reflect the actual dynamic behavior when the vehicle is maneuvering.Although the CRTS outperforms the tradition KF, the same figures demonstrate that the VB-ACKS is much better.From Figures 8 and 9, we can see that the KF and CRTS solutions have large drift errors even though the CRTS reduces the error of the KF.The VB-ACKS results are in general satisfactory in terms of errors compared to the CRTS and the conventional KF.
We trained the ERELM model for a fixed number of ELMs, which are 20.The system accuracy depends on the number of ELMs, but for the sake of computational cost the maximum number of ELMs used was 50; we choose to test our algorithm using 300-sample time.This is motivated by the fact that sampling the point will enable a good evaluation of the GPS accuracy and will benefit the low-cost MEMStype device since it has a lower sampling rate compared to the expensive MEMS devices.
Figure 11 shows the East and North position errors estimated by the ERELM, LS-SVR [14], and the VB-ACKS.It can  be clearly observed that our proposed prediction method outperforms the LS-SVR and the VB-ACKS.Identically, Figures 12 and 13 show that our proposed ensemble RELM algorithm for predicting GPS outages outperforms the other prediction algorithm (i.e., LS-SVR) in estimating the North and East velocities errors.The proposed ERELM-VB-ACKS scheme improves as well the errors of roll, pitch, and yaw angles estimated by the conventional KF by 87%.We considered four different outages periods of 10, 30, 70, and 100 s each to evaluate the effectiveness of the proposed hybrid prediction algorithm, to mimic both short-and medium-term GPS outages conditions.As it is observed from Tables 1 and 2, the proposed ERELM model produces less positional error compared to VB-ACKS used in a standalone way and the LS-SVR.Our proposed GPS outage prediction methodology is capable of delivering stable output for the case where vehicle experiences sudden change in its trajectory as demonstrated through outage 2 in Figure 8. Table 2 shows the results obtained by our proposed algorithm during outage 1 and  outage 2 and is compared with LS-SVR algorithm which has shown good performance so far in GNSS/MEMS-INS integration.As it can be seen, our algorithm outperforms the VB-ACKS and LS-SVR.Table 2 confirms as well the performance of ERELM/VB-ACKS which is 30% better than the LS-SVR/VB-ACKS and LS-SVR/KF.From the same tables, we can see that the ERELM/VB-ACKS algorithm improves the position and velocity accuracy of GNSS/INS integration systems.In addition, corresponding RMSEs in these tables demonstrate that the ERELM prediction algorithm is able to provide remarkable position accuracy during the outages.LVN is often characterized by challenging trajectories that comprise curves.In order to test the performance of our algorithm, one outage was introduced during the "" turn trajectory as depicted in Figure 8.An illustration of our algorithm performance is shown in Figure 14 where two consecutive outages were introduced between time slots 200 and 250 on the East position during the training of ERELM and LS-SVR algorithms.This Figure shows that our algorithm outperforms the two other solutions, that is, VB-ACKS and LS-SVR.Compared with the other two solutions, ERELM prediction solution shows the outstanding performance during all the four GPS outages that ascribes to its ability to improve prediction accuracy through the combination of several predictors using AdaBoost.RT and the RELM algorithm.Finally, the performance of ERELM during the four GPS outages introduced in the trajectory of the vehicle is demonstrated during the testing process.From Figure 14, we can clearly see that our algorithm shows a remarkable performance when the four GPS outages are introduced at the same time.As it can be observed, VB-ACKS gives the best results compared to the other two filters (CRTS/KF and conventional KF) solution used.Ten seconds GPS outages can be observed in Figure 14 between time slots 50 and 100.One

Figure 1 :
Figure 1: Model structure of the VB-ACKS fusion filter for GNSS/INS integration.

Figure 2 :
Figure 2: Network structure of an ELM.

Figure 4 :Figure 5 :
Figure 4: Model structure of the VB-ACKS fusion filter for GNSS/INS integration.

Figure 7 :
Figure 7: Trajectory of the experiment and GPS outages.

Figure 8 :
Figure 8: Comparison of position errors during filtering.

Figure 9 :
Figure 9: Comparison of velocity errors during filtering.

Figure 10 :
Figure 10: Comparison of attitude errors during filtering.

Figure 14 :
Figure 14: Prediction of position with GPS outages.
Machine.As a learning algorithm for single-hidden layer feed-forward neural networks (SFLNs), ELM randomly selects weights and biases for hidden nodes and analytically determines the output weights by finding least square solution.The main concept behind ELM lies in the random initialization of the SLFN weights and biases.
AdaBoost-R.T.Many researchers have proposed various ensemble methods that integrate a set of ELMs into a combined network structures, and it was Input: Final hypothesis  fin = ∑  =1     ().