ANN Approach for State Estimation of Hybrid Systems and Its Experimental Validation

A novel artificial neural network based state estimator has been proposed to ensure the robustness in the state estimation of autonomous switching hybrid systems under various uncertainties. Taking the autonomous switching three-tank system as benchmark hybrid model working under various additive and multiplicative uncertainties such as process noise, measurement error, process–model parameter variation, initial state mismatch, and hand valve faults, real-time performance evaluation by the comparison of it with other state estimators such as extended Kalman filter and unscented Kalman Filter was carried out. The experimental results reportedwith the proposed approach show considerable improvement in the robustness in performance under the considered uncertainties.


Introduction
Many researchers focused their attention towards hybrid systems, because almost all the technological systems deal with the variables which are both continuous and discrete in nature [1][2][3][4][5].Autonomous switching hybrid systems (AHS), in which the switching of discrete states is based on the values of continuous states [5], are considered in this work.Tank systems were widely used as the benchmark system for demonstrating the effectiveness of algorithms proposed for modeling, estimation, fault detection, fault isolation, and control operations in hybrid systems [6][7][8][9][10][11].Most of such works were confined to simulation studies only [6][7][8][9][10].But [11] deals with the experimental validation of the algorithms for modeling and control of nonlinear hybrid systems.In this context, the state estimation plays an important role in controlling the process variables of the hybrid systems with advanced control algorithm [9,11,12].
Since 1960, state estimation has become a very important area of study in control engineering, as Kalman [13] had developed a well-known Kalman filter (KF) for the linear filtering and prediction problems.But almost all real systems possess some kind of nonlinearity.So, later on people started working on the extension of Kalman filter to the state estimation problems of nonlinear systems [12,[14][15][16][17]. Jazwinski [14] used an extended Kalman filter (EKF) for the estimation of states of nonlinear systems through local linearization.In this, the local linearization is carried out by Taylor series expansion, which requires the calculation of Jacobians of state transition operator at each time step.So this method is complex for higher-order systems [12].Because of the discontinuity in hybrid systems due to the switching of discrete variables, it was suggested in [12] not to use EKF for the state estimation of AHS, especially in the regions where the discrete variables undergo frequent variations.In 1996, Julier et al. [15] proposed a new approach to the nonlinear state estimation problems and this uses the unscented transform principle, known as the unscented Kalman filter (UKF).Since this method follows a derivativefree approach, it was extensively utilized by the majority of the researchers working with nonlinear dynamics especially in hybrid systems [9,11,16,17].But the applicability of this algorithm in fast dynamic processes has to be investigated in detail in terms of estimation time requirement.Some works introduced different types of artificial neural networks (ANN) for modeling and observer design of nonlinear systems [18][19][20][21][22][23][24][25][26].The same concepts were applied for the identification of a global model for hybrid systems using feed forward ANN by Messai et al. [20] and an adaptive, growing, and pruning radial basis function network by Alizadeh et al. [21].Later these types of ANN models were utilized for the model-based predictive and adaptive control schemes for nonlinear systems.In this regard, a Lyapunov based neural network adaptive control scheme was developed by Hayakawa et al. [22], and this guaranteed an asymptotic stability of closed loop hybrid systems.Al Seyab and Cao [23] had used a continuous time recurrent neural network (CTRNN) model in the nonlinear model predictive control (NMPC) of nonlinear systems.Works were carried out for finding more dynamic and efficient ANN architectures for modeling the different types of nonlinear systems and for the utilization of such models in the online system identification [24], state estimation [25], and tracking control [26] of nonlinear systems.But the potential of ANN architecture for modeling was not extensively explored in the state estimation of hybrid systems, which is investigated in this work.
Also, the effectiveness of the different state estimation algorithms for hybrid systems under various real-time uncertainties and constraints was not significantly investigated [9,12].This work proposes an ANN based state estimator (ANNSE) and carries out a detailed experimental investigation on its performance comparison with EKF and UKF based state estimators (EKFSE and UKFSE) [17] under different real-time uncertainties, such as parameter variation, noise variation, and some valve faults.The efficacies of the algorithms are demonstrated with the help of a benchmark three-tank hybrid system under real-time situations.The performance indices considered for the comparison are ISE criterion and the time required for estimation.
The organization of the rest of the paper is as follows.
In Section 2, the proposed ANNSE is described along with a brief idea about the general state estimation schemes for nonlinear systems.Section 3 provides the details of the hybrid three-tank system used for the implementation of the algorithms.Experimental results of the performance comparison of the proposed ANNSE with EKFSE and UKFSE are presented in Section 4. Finally, Section 5 deals with conclusion and scope for further studies in the state estimation of AHS.

State Estimation in Nonlinear Systems
State estimation has gained a lot of attention from researchers working in the process industry in order to control various process variables efficiently [9][10][11][12].The Kalman filter is one of the strongest tools used for estimating states of linear systems, while its nonlinear extensions like EKF and UKF are commonly used for the estimation problems of nonlinear systems.This algorithm requires a model of the actual system as well as some basic knowledge such as initial values of states and covariance of the system.Figure 1 gives the general block diagram representation of the general state estimator with prediction-correction approach.
The estimator receives input (()) and output (()) sampled from the process at each time step.The prediction model of the process gives a priori estimates of the states Measurement noise, (t) ( In the above hybrid dynamic system equations,  and  represent the unobserved continuous and discrete states of the system, while  is a known exogenous input and  is the only measured output signal.For autonomous switching hybrid systems,  will be a function of  [5].The process noise (()) and measurement noise (V()) shown in Figure 1 are assumed as additive type zero-mean Gaussian white noise sequences (cf.(1)) with covariance matrices  and , respectively.
The  and  in (1) symbolize -dimensional nonlinear state and output functions which are assumed to be known.ANNSE is proposed for estimating unobservable continuous states () of the hybrid process using the only measurable output () and the input measurements () in a recursive manner, so that this algorithm can be implemented online easily.

ANN Based State
Estimation.ANN is one of the strongest tools used for the modeling of different types of systems [18][19][20][21][22].The main advantage of ANN modeling is that it does not require any knowledge about the actual system; but rather the modeling is done on the basis of the data collected from the system.It is a black box modeling method, in which output is expressed as the linear combination of the inputs followed by a nonlinear activation function.Different types of ANN such as feed forward, recurrent, radial basis function, and dynamic neural networks are utilized for modeling of hybrid systems in literature [20,21,23,24].
The proposed ANNSE scheme is based on predictioncorrection approach of the Kalman filter in which the correction part is replaced by a nonlinear autoregressive network with exogenous input (NARX).As a result, this algorithm does not require Jacobian matrix calculations in the correction part, and it becomes a derivative-free state estimation algorithm.Hence, it overcomes one of the main limitations of EKFSE [12].
In the first step, a proper value for initial state vector has to be selected.Using this, a priori estimate of the states and outputs, based on the nonlinear state and output models of the system, are calculated as given below: ( The innovation between the actual output and predicted output is generated as The updated state estimates are obtained using the nonlinear update equation as follows: NN is the nonlinear ANN function that is performed by the recurrent multilayer perceptron with NARX structure.The details of NARX are given as NARX Strucutre.It receives a priori state estimates x( |  − 1) and innovation ( |  − 1) as exogenous input and gives a posteriori state estimates x() as output.
The important and time-consuming step involved in the development of ANNSE is the training and tuning of the ANN.The training data, generated using EKFSE, cover the entire operating region of the system in order to ensure that the functioning of ANNSE is possible in the entire operating region.Tuning of the ANN is done by varying hidden layer neurons, number of output delays (), and number of input delays () by keeping the integral square error (ISE) as the performance measure.The values of  and  and number of hidden layer neurons, for which the minimum ISE is obtained, are selected as the final ANN parameters which are listed in Table 1 along with other specifications.Even though the offline training is performed using the bulk data, its online validation is also carried out.

Kalman Based State Estimators for Nonlinear Systems.
This section reviews the nonlinear extensions of Kalman filter for state estimation problems using EKF [14] and UKF [15].[14].The EKF [14] was attempted to overcome the limitation of KF [13] by using a linearized approximation where the linearization was performed about the current state estimate.As per [12], the usage of EKF for the state estimation of hybrid systems is not allowed only at the point of mode switching.In the current problem of three-tank system explained in Section 3, this condition occurs only when the level in tanks is exactly 0.3 m.But the chances of reaching this situation are negligibly small as the measurements are taken only once in one second.EKF for the AHS is developed by defining different Jacobian functions for each mode of operation based on the discrete variables ( 1 and  2 ).Also, in the case of level exactly 0.3 m, the Jacobian of the previous sampling instant is taken for the correction part of the state estimator.This may lead to negligible estimation error at that particular sampling instant.EKF involves the recursive estimation of the mean and covariance of the state.The function  can be used to compute the predicted state from the previous estimate and similarly the function  can be used to compute the predicted measurement from the predicted state.However,  and  cannot be directly applied for the calculation of covariance.Instead, a matrix of partial derivatives (Jacobians) was computed at each time step with current predicted state and evaluated.This process essentially carries out the linearization of the nonlinear function around the current estimate.

Extended Kalman Filter
The predicted state estimates are obtained using a prediction model as in ANNSE x ( |  − 1) =  ( x ( − 1) , ẑ ( − 1) ,  ()) . ( The covariance matrix of estimation errors in the predicted estimates is obtained as where Φ() is nothing but the Jacobian matrix of partial derivatives of  with respect to  at x( − 1): The measurement prediction, innovation, and covariance matrix of innovation are computed as given below: where () is the Jacobian matrix of partial derivatives of  with respect to : The Kalman gain is computed using the following equation: The updated state estimates are obtained using the Kalman gain and the innovation The covariance matrix of estimation errors is updated as follows: These steps are repeated for the entire process run with new estimates of state and covariance.[15].In order to overcome the limitations of EKF algorithm in the state estimation of nonlinear systems, Julier et al. [15] had proposed UKF.

Unscented Kalman Filter
The main advantage of this estimator was that it does not require any Jacobian matrix calculation at each time step.This utilized the unscented transform principle for computing approximate solutions to the filtering problems.Unscented transform is a method of calculating the statistics of a random variable which undergoes nonlinear transformation.In this algorithm too, the initial values of state x( − 1) and covariance P(−1) have to be chosen properly from the prior knowledge about the system.In the next step, 2 + 1 sigma points have to be identified such that x( − 1) and P( − 1) form their mean and covariance, respectively, as given below: x() = x ( − 1) ± √ ( + ) P ( − 1), where  = 1, 2, . . ., 2 and  is the dimension of the state vector.The  in ( 14) is computed as The spread of the sigma points around the mean, x( − 1), is determined by , whose value may vary between 0.001 and 1.The value of secondary scaling parameter, , is usually 0 for state estimation and (3 − ) for parameter estimation [16].Each sigma point is then passed through the nonlinear process model so that a cloud of 2 + 1 new sigma points is obtained as follows: Now these new sigma points are fed to measurement model as Based on their statistics new mean x( | −1) and covariance P( |  − 1) are calculated: A priori estimate of the output is computed as The associated weights in the calculation of mean, covariance, and output in ( 18)-( 20) are obtained as Here  is employed to include the prior information about the distribution of the state and for Gaussian distribution,  = 2 is selected as the optimum value [16].The innovation covariance and the cross covariance between the innovation and a priori estimates are given below: These values were used for the calculation of Kalman gain as This Kalman gain is used for the measurement update of mean and covariance of the states as These new values of state and covariance are fed as input to the time update part for the next iteration.

Autonomous Switching Hybrid Three-Tank System [17]
The benchmark autonomous three-tank hybrid system, shown in Figure 2, is used for demonstrating the efficacy of the proposed ANNSE and its performance comparison with EKFSE and UKFSE.It consists of three cylindrical tanks with corresponding cross-sectional area denoted by  1 ,  2 , and  3 .These tanks are connected to each other and to sump through seven hand valves with valve coefficients denoted by  1 to  7 (experimental values are given in Table 9) and two variable speed pumps as shown in Figure 2.
The two inflows (Fin 1 and Fin 2 ) make the inputs, , and the levels in the three tanks (ℎ 1 , ℎ 2 , and ℎ 3 ) make the continuous states, , of the system shown in the block diagram of estimator (Figure 1).The discrete variables ( 1 and  2 ) are decided by the direction of flow through the 3rd and 4th hand valves and they in turn depend on the three water levels.Since the switching of discrete state variables is decided by the continuous state variables and further no external input is required for the switching of discrete variables, this system comes under the group of AHS.
The value of continuous variable varies between 0 and overflow height (0.55 m) of corresponding tank continuously.The discrete variables,  1 and  2 , may take the values 0, +1, or −1 depending on whether flows  3 and  4 are 0, towards tank 3 or away from tank 3, respectively [17].Let the flow rate of water through the th valve be   , which is calculated using Bernoulli's equation based on the upstream and downstream water heads and the corresponding valve parameters as below: where , , and  are used to indicate whether water in the three tanks is above or below the threshold level, ℎ 0 .Threshold level is the height of the middle interconnecting pipe from the bottom of the tanks.The dead heights of the 5th, 6th, and 7th valves from the tank bottom are represented by ℎ  1 , ℎ  , and ℎ  2 , respectively.Based on the volume balance equation, nonlinear state space representation of the system is as follows: This state space representation constitutes the prediction model of the state estimator shown in Figure 1.

Results and Analysis
All the three state estimation algorithms explained in Section 2 were simulated using Matlab for 15000 seconds and the sampling time considered was 1 second.Further, it was validated using the data collected from the experimental setup of autonomous three-tank hybrid system and the results are given in this section.LabVIEW and NI DAQ were used for data collection from the system.The discharge coefficients of different hand valves of the three-tank hybrid setup (shown in Figure 14), given in Table 9, were found by activating the actual system with the general input sequence, tabulated in Table 10.Other process parameters of the system are also given in Table 11.
The estimators have been designed such that only one level (middle tank level, ℎ 3 ) is available for measurement, and using this level as well as the two inputs (Fin 1 and Fin 2 ), all the three levels are estimated at each time step in an iterative manner.Two types of noises taken into account are process noise and measurement noise which are considered as additive Gaussian white noise [12].By knowing the characteristics of the output level sensor, the measurement error covariance, , is assumed to be known; the process noise covariance, , used in ( 6) and ( 19) is unknown and this is considered as the tuning factor for the state estimator.Both EKFSE and UKFSE are tuned for finding the optimal value of the tuning factor, such that the integral square error (ISE) between the actual and estimated states is minimized.
Quantitative comparison of all the three estimators is also carried out in terms of ISE and estimation time.Figures 4-12 are the graphical comparison of the estimators under various operating conditions in which the legend ACTUAL stands for the real-time data collected from the experimental setup.For better understanding of the performance comparison, expanded views of the graphs are provided in most of the cases.The abbreviations ISE 1 , ISE 2 , and ISE 3 in Tables 2-8 are the corresponding integral square errors (expressed in m 2 s) in estimating levels in the 1st, 2nd, and 3rd tanks, respectively.Est.Time is the time requirement in seconds for estimating the states, once the data from the process is available.

Performance of the Estimators under Normal Operating
Condition.Under normal operating condition, no other uncertainties except process noise and measurement noise are considered.Prediction model conditions were made as close as possible with that of the actual process.At this stage the performance of EKFSE and UKFSE was found to be better than that of ANNSE as shown in Figure 3 and Table 2. Figure 4 gives the expanded view of the initial portion of Figure 3 for a better understanding of results of different algorithms.

Performance of the Estimators under
Various Uncertainties.Also various uncertainties have been considered in the performance analysis in order to investigate the robustness of these estimators.They are given below: (1) process-model parameter mismatch,  (2) initial state vector mismatch, (3) higher level of process noise,  The uncertainties due to parameter mismatch and hand valve faults are directly getting multiplied with the states, while those due to process noise and measurement noise are considered additive in nature.

Process and Model Parameter Mismatch.
As the parameters (area of tanks and valve coefficients) used for the model are found out by human measurements, experiments, and calculations, the parameters of the prediction model and the actual system will never be identical.These variations have been considered in this work by introducing a parameter mismatch factor in the state estimators.As the magnitude of mismatch factor is increasing, the corresponding ISEs also get increased.Under this state of affairs, ANNSE is giving better estimates than other Kalman based state estimators.Both EKFSE and UKFSE were giving offset in the state estimates under parameter mismatched condition.The cases of both higher and lower model parameter values are considered and the corresponding quantitative performance measures were tabulated in Table 3. Figure 5 shows the sample graph for estimation of states under process-model parameter mismatched condition (  = 1.2 and   = 1.05),where   and   are the mismatch factors in area and valve coefficients.

Initial State Vector Mismatch. It has already been stated
that the level in the third tank is only measurable.But the state estimation algorithm requires the initial value for all the state variables.So it is required to make some assumptions about the levels in the first and second tanks.These assumed values need not be the same as the actual value.In order to include this condition, a different initial state vector was considered for the prediction model of the algorithms, and along with the other two performance measures, the convergence time in seconds is also compared for the three methods as given in Table 4.As we are interested in the initial convergence characteristics only, the estimation has been done for 3000 seconds in this case.
The results show that all the algorithms are good enough in computing the estimated levels to the actual value, as it is clear from the expanded view of the evolution of the three levels, which is shown in Figure 6.From Figure 6 and Table 4, it is clear that EKFSE is giving the fastest convergence, whereas the UKFSE is giving very slow convergence as compared to EKFSE and ANNSE.

Higher Degree of Process Noise.
At the time of implementation stage, the degree of process noise may become higher than that at the time of tuning the estimators.This condition is incorporated by increasing the noise in the actual process and keeping the same tuned value for process noise covariance () of the estimator.The result shown in Figure 7 proves the robustness of the estimators under this condition.Performance indices of the three estimators are compared in Table 5.  model of the estimator.Random noise is added as the uncertainty in the input sequence.The performances of the three estimators are compared in Figure 8 and Table 6, which shows that ANNSE is giving better estimates of the states under higher degree of input uncertainties.

Applying an Input Sequence Different from Input
Sequence of Tuning.In all the above-mentioned conditions, input sequence applied to the process exactly matches with the input sequence in the tuning of the estimators (cf.Table 10).Now, the process was activated with a higher input sequence (Fin 1 = 0.0000360528 m 3 /s and Fin 2 = 0.0000275055 m 3 /s) from the beginning to the end  of estimation (5000 sampling instants).The performance comparisons are given in Figure 9 and Table 7, which shows that all the estimators give satisfactory performance with a different input sequence, as in the case under normal condition.11).In this case, the inflows given to the tank are the same as that in Section 4.2.5 for the entire duration of the estimation.

Conclusion
ANNSE was developed for the state estimation of autonomous switching hybrid system and its performance under different real-time operating conditions and uncertainties was compared with that of EKFSE and UKFSE both qualitatively and quantitatively.Even though a negligible compromise is made with the estimates at the mode switching point of AHS, EKFSE was found to be the best method for the real-time implementation of state estimation under normal operating conditions especially that with faster dynamics, as the estimation of AHS under normal operating condition as the estimation time is considerably small.Considering the estimation time given in Tables 2-8, EKFSE method makes the lowest time whereas the UKFSE makes the highest for all the cases of operating conditions illustrated in this work.But under process-model parameter variations, noise affected input, and hand valve faults conditions the proposed ANNSE gave better estimates than the other two methods.It was observed that even though the ANN is trained using the data set of EKFSE, the better performance of ANNSE when the AHS is under these considered uncertainties, compared with the other methods, is obtained due to the following advantages of ANNSE.
(i) In the correction part of EKFSE and UKFSE, some approximation using linearization around the operating point is carried out, whereas, in ANNSE, nonlinear ANN is used to correct the a priori estimates.
(ii) The correction part of ANNSE is completely parameter independent, which gives the better state estimates under parameter mismatch.
(iii) ANN has built-in noise rejection capability.
(iv) Dynamic ANN structure (NARX) selected is suitable for this application.
(v) Training of the ANNSE is carried out with a very huge set of data which covers the entire operating region of the process.
In the operating situation when the process noise exceeds its normal value used for training of filters, UKFSE was found to be a better estimator, as the ISEs are lowest when compared to EKFSE and ANNSE.In the case of initial state vector mismatch, EKFSE gave the estimates with very fast convergence to the actual value.In these two conditions also, the estimates of ANNSE were converging to the actual values, but the accuracy of estimation was found to be poor compared to other methods.In this work it was strongly recommended to use ANNSE for the state estimation of AHS under parameter uncertainties, noise contaminated input, and hand valve leakage and clogging conditions, EKFSE for the state estimation of AHS under normal operating conditions, initial state vector mismatch, and faster dynamic situations, and UKFSE for the state estimation of AHS with higher degree of process noise than that at the time of training.It is also concluded that the proposed ANNSE is a robust method compared to the conventional Kalman based methods, for the state estimation of AHS operating under different uncertainties (especially parameter uncertainties).
As a scope of the work, since there is a chance for the process parameters, especially the valve coefficients, of getting changed, it is better to estimate the process parameters along with the states so that these estimated parameters can be used in the prediction model which may further lead to better estimates of the states.Present practice of using first principle model can be replaced with ANN prediction model, so that the estimation algorithm becomes a black box state estimation algorithm which does not require any knowledge about the system, except input output data collected from the actual system.Also, based on the proposed ANNSE, some control schemes can be developed, which may overcome the limitations with existing model-based controllers thereby ensuring the robustness in the control of hybrid systems under uncertainties.Instead of benchmark application, this proposed method can be applied for more realistic and killer engineering applications.Works in this connection can open up a new research direction.

Figure 3 :
Figure 3: Evolution of three levels under normal operating conditions.

Figure 4 :
Figure 4: Expanded view of evolution of three levels under normal operating conditions.

Figure 5 :
Figure 5: Expanded view of evolution of three levels under process and model parameter mismatched condition (  = 1.2 and   = 1.05).

Figure 8 :Figure 9 :
Figure 8: Expanded view of evolution of three levels under noise contaminated input measurements (input noise factor = 0.003).

Figure 11 :
Figure 11: Expanded view of evolution of three levels under leakage fault in hand valve 5.

Figure 13 :
Figure 13: NARX structure for the considered example.

Figure 14 :
Figure 14: Experimental setup of three-tank hybrid system.

Table 1 :
Values of different ANN parameters.

Table 2 :
ISE in estimating the three levels under normal operating conditions.

Table 3 :
Performance measure of the state estimators under process-model parameter mismatch.

Table 5 :
Performance measure of the state estimators under higher degree of process noise.

Table 6 :
Performance measure of the state estimators under noise contaminated input measurements.

Table 7 :
Performance measure of the state estimators under different input sequence.

Table 8 :
Performance measure of the state estimators valve fault conditions (clogging and leakage).

Table 9 :
Discharge coefficients of hand valves.

Table 10 :
Values of inflows to the system with sampling instants.

Table 11 :
Values of different system parameters.
Contaminated Input Measurements.In this case, input data with measurement noise was fed to the prediction These faults were introduced by partially closing one of the hand valves (6th valve) and partially opening the fully closed 5th hand valve at the 7500th sampling instants in separate runs.Performance of the ANNSE is much better than that of other two estimators under these hand valve faults (cf.Table8and Figures10 and 4.2.6.Clogging and Leakage Faults.Two types of faults in the hand valves, clogging and leakage faults, have also been considered in this work.