State Estimation for Nonlinear Discrete-Time Systems with Markov Jumps and Nonhomogeneous Transition Probabilities

State estimation problem is addressed for a class of nonlinear discrete-time systems withMarkov parameters and nonhomogeneous transition probabilities (TPs). In this paper, the optimal estimation mechanism of transition probability matrix is proposed in the minimum mean square error sense to show some critical points. Based on this mechanism, the extended Kalman filters are employed as the subfilters to obtain the subestimates with correspondingmodels. A novel operator which fuses the prior knowledge and the posterior information embedded in observations is developed to modify the posterior mode probabilities. A meaningful example is presented to illustrate the effectiveness of our method.


Introduction
Markov jump systems (MJSs) are special systems which involve both time-evolving and event-driven mechanisms and have been used to model a wide variety of dynamic systems with unpredictable system structures.For MJSs, the state of the systems has two components: the finite valued state (denoted as mode) and the state in a finite dimensional Euclidean space.Motivated by a wide spectrum of applications, state estimation for MJSs has received considerable attention in the literatures [1][2][3][4][5][6][7] over the last three decades.It is well known that the optimal estimator for linear MJSs without the information about the underlying mode sequence is obtained by a bank of Kalman filters which requires exponentially increasing memory and computation with time.In order to limit the resource requirement, some suboptimal but executable algorithms have been proposed, including the generalized pseudo-Bayesian (GPB) [2], the interacting multiple model (IMM) [3], the improved IMM method [4,5], and the particle filters (PF) [6,7].Moreover, methods based on guaranteed-cost approach which attempt to guarantee that the covariance of state is upper bounded by a certain constant value are also derived in quantity [8][9][10][11][12][13][14].
It is worth noting that the all the Markov chains considered in the aforementioned estimators are assumed to be homogeneous.More specifically, the transition probabilities (TPs) are constant and are not allowed to be time-variant during the entire estimation process [15].However, since the external practical environment (humidity, temperature, etc.) always exhibits complex and rich dynamics, it is impossible to guarantee the TPs of all likely modes to be time-invariant.In other words, nonhomogeneous Markov process is a common phenomenon, not the exception in practice [16,17].On the other hand, TPs are introduced to depict the uncertainties of transitions between possible system behavior patterns and the accuracy of TPs is critical to the estimation performance of MM-type estimators [18].In this sense, it is wise and meaningful to consider the nonhomogeneous Markov chain in systems.By observing this, nonlinear stochastic hybrid systems with state-dependent TPs were considered in [19], where PF was embedded into IMM method to obtain the IMM-PF.With the same motivation, state-dependent TPs were modeled in terms of Gaussian-shaped functions in [20] and the Gaussian hypotheses were used to reduce the computational cost of the developed algorithms.However, the existing literatures seem to lack a complete approach of state estimation for MJSs with uncertain time-varying TPs, until the H-infinity filter for nonhomogeneous linear MJSs was proposed in [21].In this paper, the character of nonhomogeneous TPs is reflected by a higher-level transition matrix and TPs vary within a finite set in discrete-time domain.Later, the convenience brought by the assumption that the transition probability matrix (TPM) randomly takes value in a finite set was demonstrated analytically in [22] by comparing the optimal Bayesian equation derived under two different conditions.Nevertheless, all these estimators require the information about the TPM.More specifically, all the candidate TPMs and high-level transition matrix are required to be known accurately, making the resulting methods intractable in practical applications.
In this paper, a recursive estimator is derived for nonlinear MJSs with nonhomogeneous TPs.Different from [21,22], the information about the dynamic of TPM including candidate TPMs and high-level transition matrix is not required to be known completely.This is a particular advantage of the framework raised here, and to the best of authors' knowledge, no similar situation has been considered.The Bayesian equation that exploits the likelihoods of TPM to update the prior PDFs is developed.For the output calculation, we are required to find both the likelihoods and the dynamic of TPM which unfortunately turn out to be impossible.Therefore, instead of considering all the possible values of TPM in an admissible region, a constant diagonally dominant TPM which is rudely determined in advance is employed in the entire estimation process.In this sense, the problems involved in the estimation of TPM are solved naturally.The extended Kalman filter (EKF) [23,24] is utilized as the subfilter under different models to obtain the subestimates and posterior mode probabilities.The exponentially increased computational problem is solved by the IMM-type approximation.Once the posterior mode probabilities are available, a novel operator is proposed to modify the posterior mode probabilities.Specifically, the probabilities of dominant modes are modified to approach one automatically according to their dominating degree.This operation is reasonable, as if the probability of one model is much larger than the others and we can almost make sure that it is the real one.
The content of this paper is organized as follows.In Section 2, some definitions and preliminaries are given.Section 3 proposes Bayesian estimation of TPM.The derivation of the suboptimal implementation is given in Section 4, where we propose a novel operator to modify the posterior mode probabilities after time update.The objective of Section 5 is to demonstrate the effectiveness of our method with a simulation example.Finally, Section 6 concludes this paper.

Problem Formulation and Preliminaries
We consider the discrete-time nonlinear JMSs described by the following equations: where (⋅) and ℎ(⋅) are nonlinear transition and measurement functions, respectively;   ∈   is the continuous state in Euclidean space with known initial distribution of state  0 ;   ∈   is the measurement;   and V  are, respectively, uncorrelated white process and measurement noises with known statistics; the sequence {  } is a continuous-time discrete-valued Markov chain and takes values in a finite state space  = {1, 2, . . ., }.In this paper, the Markov chain governing the system dynamics is nonhomogeneous, which means that the time  exists as a guard condition of the probability of a jump from mode  at time  − 1 to mode  at time  as where ,  = 1, 2, . . ., .Then, the TPM can be given as T , where notation [⋅] T denotes the transpose of matrix or vector, T , and  = 1, 2, . . ., .Note that if Π  is known as a prior, the existing algorithms such as IMM-PF method [2,5,6] can be extended to the nonhomogeneous cases easily by using corresponding TPM at each time step.Unfortunately, TPs always change randomly and it is unrealistic to obtain the dynamic trajectories of real TPM in practical applications.Hence, it is assumed in the paper [21] that for arbitrary , Π  is unknown but takes its value in a finite set where ,  = 1, 2, . . .,  and ∑  =1   = 1.In this sense, the discrete distribution of Π  , where  = 1, 2, . . ., , can be considered as a point-mass approximation of Π  over the simplex of valid TPMs.However, the requirement that both the candidate elements in Θ and the high-level transition matrix  should be available in advance is difficult to be satisfied in applications.That is to say, the finite-valued hypothesis is also impractical.In fact, a filter designer often uses a diagonally dominant constant matrix  = [ 1 ,  2 , . . .,   ] T to replace the true values in reality, where   = [ 1 ,  2 , . . .,   ] T and ∑  =1   = 1, for any  ∈ .Intuitively, if no further operator is implemented, using such an inaccurate matrix directly may lead to performance degradation, as it failed to capture the real possible system behavior patterns.Therefore, the objective of this paper is to derive a recursive, filtering algorithm that yields satisfactory estimates of states in the cases that only the approximate matrix  is available.

Bayesian Estimation
It is well known that the recursive Bayesian equation of the JMSs with accurate TPs is given as where   (  −1 |  −1 ) are the mode probabilities at a previous time step.At this point, we can see that using incorrect TPs will lead to the deviation of mode probabilities and reduce the estimation accuracy further due to the following fact: Thus, it is highly desirable and really nice to obtain the true TPM Π  at each time step.Unfortunately, Π  is assumed to be unavailable, and a natural solution of this problem is to estimate the unknown TPM recursively.In the following, we thus focus our attention on the problem of finding the posterior PDF (Π  |   ) of the nonhomogeneous TPM in the Bayesian framework.In view of Bayes' ruler, we have where (  |  −1 ) is considered as a normalization constant, (  | Π  ,  −1 ) is the likelihood, and (Π  |  −1 ) denotes the prior PDF of Π  .Using the law of the total probability, the prior PDF (Π  |  −1 ) can be equivalently written as where (Π  | Π −1 ) and (Π −1 |  −1 ) denote the dynamic of Π  itself and posterior PDF at a previous time step.Using the total probability theorem, the first term in the numerator of the right-hand side of (6) becomes Defining the likelihood vector Λ  and mode probability vector  −1 as T , where By the Chapman-Kolmogorov equation, the denominator of the right-hand of ( 6) can be rewritten as Substituting ( 9) and ( 10) into (6) yields where (Π  |  −1 ) is given by (7).Note that formula (11) is the key of the entire estimation process, as it tells that the known prior distribution (Π  |  −1 ) is updated using the likelihoods of TPM to obtain the posterior conditional PDF (Π  |   ).Unfortunately, calculating (11) analytically is almost impossible as it involves the following problems: (i) the dynamic of Π  in ( 7) is difficult to obtain; (ii) notice that to get the posterior conditional distribution, one has to calculate the likelihood of Π  .However, since Π  is unknown and is also required to be estimated at time , evaluating the likelihood directly is impossible.Due to the above problems, the implementation of ( 11) is intractable and an alternative approach or some necessary approximation will be introduced in the following section.
Remark 1.If we assume that the TPM takes in a finite set Θ, the dynamic of TPM in (7) can be expressed by the highlevel transition probabilities   , where ,  = 1, 2, . . ., .Moreover, we are not required to consider all possible values in an admissible region and only consider the likelihood of the values in Θ that need to be evaluated, making the problem considerable simpler.

Suboptimal Estimator
Since the system under consideration is nonlinear, in this paper, EKFs are utilized as subfilters.It is well known that EKF is one of the most widely used algorithms in the framework of nonlinear filtering.Let   −1|−1 , Δ  −1|−1 denote the estimate of state, covariance matrix computed by the  EKF at time −1, respectively.Let   −1 denote the mode probability of mode  at time  − 1.Then, the PDF of state at time  − 1 conditioned on   −1 is denoted as where (⋅) denotes the normal distribution.As done in IMM method, the optimal Bayesian equation is decomposed into a series of basic transitions.In this sense, the PDFs ( −1 |    ,  −1 ) can be equivalently given as follows: Note that we have Here, the IMM approximation is employed.More specifically, -component Gaussian mixture is replaced by a single Gaussian density to solve the computational sensitive problem as where  = 1, 2, . . ., , where    is the filter gain given as where   and   are the covariance matrices of the process noise and measurement noise, respectively.At this point, the posterior mode probabilities can be updated as where  = 1, 2, . . ., .Considering ( 14) and ( 22), show that only the approximate TPs   are employed to compute the mode probabilities which are used as "weights" of subfilters in the filtering process.That is to say, the estimation performance will be unsatisfied, as the mode probabilities deviate from their real trajectories.On the other hand, it is known that the ideal case of estimation of MJSs is that the underlying mode sequence is known.In such case, only one subfilter is required to calculate the estimate at every time step and the multiple-model estimation algorithm will degenerate to be the Kalman filter with corresponding model.In this sense, the information of TPs is not required any more in filtering process.By observing these, a novel operator is developed to modify the dominating mode probabilities after time update to make them approach to one according to their dominating degree.Without loss of generality, we present (   ) at time  as where  = 1, 2, . . ., ; the index parameter   is given as follows.For the cases with  > 2 and   +    > 2/, we have else   = 1, where   and    are the maximum mode probability and minimum mode probability at time , respectively.Note that it is not difficult to verify that   ̸ = 1/ in the cases   +    > 2/.Therefore, the denominator of the righthand side of (23) will not be zero and can be used safely in computer simulation.For the cases with  = 2 and   −   > 0.1, the index parameter can be evaluated by else   = 1.At this point, we can see that the index parameter   which governs the modification degree in our method is adaptive and varies automatically according to the dominance of posterior mode probabilities.Specifically, a large parameter is evaluated when the dominance is evident, otherwise either 1 or another relatively small value is employed.It is also necessary to point out that the assumption that the real TPs are constants is not required, as the compensation parameter is updated using online information at each sampling time step.The proposed method works effectively as long as the dominance is obvious.This is an intuitively meaningful property and can be considered a particular advantage of the method introduced here.After modifying the posterior mode probabilities, the final estimate is calculated by merging the posterior estimates   | and covariance Δ  | using the modified mode probability υ  as The algorithm based on the above illustrations is detailed below.

Numerical Example
To illustrate the performance of our method, a nonlinear dynamic system modeled which is a slightly modified version of the one given in [25] was carried out.Consider where   ∼ (0, 1) and V  ∼ (0, 1).The mode-dependent parameters  are aggregated into three possible modes shown as  1 = 30 and  2 = 5.The nonhomogeneous TPM Π  takes its value in a finite set Θ = {Π 1 , Π 2 , Π 3 }, where Π 1 , Π 2 , and Π 3 are presented as follows: Two different algorithms were performed: (i) the IMM-EKF with ; (ii) the proposed algorithms (denoted as OA) with .
To assess the performance of algorithms, the average root mean square (RMS) error based on 5000 times Monte-Carlo simulations is employed.All the filters are initiated with the prior distribution ( 0 ) ∼ (0, 2).Results from 5000 Monte Carlo runs are shown in Figures 1 and 2 and Table 1.In Figure 1, the average RMS error of state based on 5000 Monte Carlo runs of different algorithms is shown.Together with Table 1 and Figure 1, we can clearly see that OA yields improvements in estimation performance significantly over IMM-EKF as expected.This is the result of using more reasonable mode probabilities, which are mainly caused by the operator developed.That is, the information embedded in the measurements is exploited to counteract the negative effect brought by the wrong TPs.At each time step, the posterior mode probability of dominating mode is modified to approach one according to the dominating degree.The index parameter in a typical run is given in Figure 2. We see that the index parameter varies with the time instants and constantly adjusts and realigns itself according to the dominance of the posterior mode probabilities.In this sense, the operator can be considered as an instrument that can increase the probabilities of dominating modes automatically.Another important issue in practical application is the computation time.Note that in OA, the index parameter is required to be calculated at every time step.However, the computational time of OA is almost the same as IMM-EKF.This is mainly because the computation of OA is quite simple, no circulation or inversion is required, making the proposed method more competitive in online applications.

Conclusions
This paper proposed a novel state estimation method for a class of nonlinear JMSs with nonhomogeneous TPs.In our work, the optimal Bayesian estimation of TPM is proposed and EKF is employed to handle the nonlinear problem.Moreover, an operator is developed to modify the posterior mode probabilities to compensate the effect caused by wrong TPs used.Simulation implicates that our method yields better estimation performance than IMM-EKF and shows more robustness against the wrong TPs.

Figure 1 :
Figure 1: Average RMS error of state.

Figure 2 :
Figure 2: Variation of the index parameter in a typical simulation.
(3)] normalization constant, (  |   ,    ) and (  |    ,  −1 ) denote the likelihoods and system dynamic under mode , respectively,   (   ,   −1 |  −1) are the joint probabilities of system modes at current and previous time steps, and ( −1 |   −1 ,  −1 ) denote the prior PDFs of state under mode , where  = 1, 2, ..., .To see the derivation in detail, one can refer to one of our recent papers[22].Considering(3), show that the effect of TPs in filtering process is mainly exhibited by the terms   (

Table 1 :
Average results of different algorithms.