Estimates for Discrete-Time Markovian Jump Linear Systems

This paper deals with the problem of filtering for discrete-time Markovian jump linear systems. Predicted and filtered estimates are obtained based on the game theory. Both filters are solved through recursive algorithms. The Markovian system considered assumes that the jump parameters are not accessible. Necessary and sufficient conditions are provided to the existence of the filters. A numerical example is provided in order to show the effectiveness of the approach proposed.


Introduction
Filtering of Markovian jump linear systems (MJLSs) has been subject of intensive study in the last years.Different and creative approaches to deal with this class of filter have been considered in the literature.One of the main alternatives to solve this kind of problem is based on Kalman filter algorithms, see for instance [1,2].Filters proposed in these references are based on linear minimum mean square error estimates of discrete-time MJLS.They present an interesting feature related with recursiveness; however, they are not robust in nature.
When the robustness is relevant to the filtering process and demands an extra performance of the filtering approach, H ∞ techniques are always considered as one of the best solutions to be adopted.In this way, a closed-loop transfer function from the unknown disturbances to the estimation error is designed in order to satisfy a prescribed H ∞ -norm constraint.In general, algorithms developed to deduce H ∞ Markovian filters are based on linear matrix inequalities (LMIs), see for instance [3][4][5][6][7][8][9][10][11][12].In particular, [4,6] assume that the jump parameter of the Markov chain is not available.
In this paper, we propose H ∞ filters for discrete-time MJLS which are calculated in terms of recursive algorithms based on Riccati equations.Following the approach considered in [13], we develop predicted and filtered estimates based on the two-players game theory.The idea of game theory was also adopted in sthochastic H 2 /H ∞ control, see for instance [14].
We define the status of the players in the filtering problems considered in this paper in order to reach an equilibrium between two contradictory objectives.The first player can be interpreted as the maximizer of the estimation cost, whereas the second player tries to find an estimate that brings the quadratic cost to a minimum.A solution exists for a specified -level if the resulting cost is positive.We assume in these Markovian filtering problems that the jump parameter is not accessible.The recursiveness of the approach we are proposing is the main advantage of these Markovian filters if compared with the filters aforementioned.As a byproduct of this approach, we provide necessary conditions for the existence of them based on only known parameters of the Markovian system.This paper is organized as follows: in Section 2, we present the problem statement; in Section 3, H ∞ filters for discretetime MJLS are presented; and in Section 4, a comparative study, based on numerical example, between the approach we are proposing and the filter developed in [4] is performed.

Problem Definition
The H ∞ recursive filters developed in this paper are based on the following discrete-time MJLS: where   ∈ R  is the valued state,   ∈ R  is the valued output sequence,   ∈ R  is the valued signal to be estimated,   ∈ R  ,   ∈ R  , and V  ∈ R  are random disturbances; Θ  is a discrete-time Markov chain with finite state space {1, . . ., } and transition probability matrix The random disturbances {  }, {  }, and {V  } are assumed to be null mean with finite second order moments, independent wide sense stationary sequences mutually independent with covariance matrices equal to the   ,   , and   , respectively. 0 1 {Θ 0 =} are random vectors with E{ 0 1 {Θ 0 =} } =   (where 1 {⋅} denotes Dirac measure) and E{ 0   0 1 {Θ 0 =} } =   ;  0 and {Θ  } are independent of {  }, {  } and {V  }.A scalar  > 0 and a sequence of sets of observations are defined to find at each instant  a filtered estimate  | of   such that sup is satisfied for  = 0, and is satisfied for  > 0. For the predicted estimate, given  > 0 and a sequence of observations (2), the problem is to find at each instant  a prediction  +1| of  +1 such that sup  4) and (6).In general, to synthesize recursive filtering algorithms for this class of problems is not an easy task.Thanks to the augmented model of (1) proposed in [1] we can redefine these functionals in order to develop H ∞ recursive filters similar to those we find in the literature for standard state-space systems, without jumps.The augmented model of ( 1) can be written as follows: where the parameter matrices are given by ] , whose dimensions are defined by F  ∈ R × , H  ∈ R × , and L  ∈ R × ; and the state variable is defined as We define for  ≥ 0 and  ∈ {1, . . ., }, where  , is given by the following recursive equation: We define also in the next lemma, weighting matrices Λ  , Π  , and Γ  as variances of random disturbances given by   ,   , and   , respectively.Lemma 1.Let   ,   , and   defined by for  ≥ 0 and ,  ∈ {1, . . ., }, where The variances of   ,   , and   can be obtained by the following equations: respectively, where Proof.The proof follows the arguments proposed in [15], Chapter 3, and in [1].

H ∞ Estimates for DMJLS
In this section, we propose recursive H ∞ estimates for discrete-time MJLS (DMJLS).It is known that for standard state-space systems, this kind of filtering approach is difficult to be implemented  due to the fact that we do not know the minimum  for each step of the recursion.In general, it depends on the estimate error variance matrix which should be calculated at the same instant of time.
In this sense, the existence condition of this filter is not known a priori.To ameliorate this limitation we provide, for this Markovian problem, necessary conditions to tune this parameter depending on the variances of the random disturbances   ,   , and   and on the known parameter matrices of the augmented model of (1) given in (7).
3.1.H ∞ Filter.Consider (3) and ( 4) for the augmented model (7): for  = 0, and for  > 0. We can rewrite this H ∞ filtering problem in terms of the following optimization problem: where ) > 0 has a minimum {ẑ | }  =0 .Therefore, in order to deal with only the minimization problem (19), for each  ≥ 0 it is easy to show that (21) can be rewritten as where ] , For all  ≥ 0, we can conclude that the following recurrent relations are valid: According to [16] With this fundamental lemma in mind, we can obtain an existence condition for the filter we are proposing in the following, based on the known parameter matrices of the Markovian model.For  > 0, the term U   R  U  of X| can be written as and a necessary condition for U   R  U  > 0 is the positiveness of E   R  E  , which in terms of the augmented Markovian model of (1) can be established as follows: Remark 3. A lower bound for   can be calculated through (27) based on the augmented model ( 7) and Lemma 1.Notice that ( 27) is a natural and interesting extension of the filtering of standard state-space systems [13], without jumps, to the DMJLS we are dealing with in this paper.Now we are in a position to deduce the H ∞ recursive filtered estimate for DMJLS.The next theorem provides the solution for this problem based on the augmented model (7) and on the sequence of measurements (2) of the original Markovian system (1).

Theorem 4. Consider the augmented Markovian model (7).
There exists a recursive H ∞ filter for this system, defined by the following recursive equations: if and only if Z| > 0,  = 0, 1, . .., and the attenuation level   is positive.
Proof.This proof follows standard arguments used to deduce Kalman filters and basically consists in checking the positiveness of (26) and in finding the equation of the filter through the Lemma 2. In order to have U   R  U  > 0, we can rewrite this term as (26).The (2, 2) sub block of (26) must be positive definite.Assuming that the positiveness of U  −1 R −1 U −1 is guaranteed and the (2, 2) term of ( 26) is positive definite and U   R  U  > 0 if and only if the Schur complement of the (2, 2) block, With  −1 defined in (24), we obtain Considering Z | :=  | in terms of the original data (23), we obtain (28).From Lemma 2 we have the solution for the minimization of ( 22).Based on the recurrent relations ( 23) and ( 24), considering  −1 X −1| = A −1  −1| for  ≥  − 1, and introducing holds (29) taking into account that  | = ŝ| := L  ẑ| .

H ∞ Predictor.
In this subsection, we present the recursive H ∞ predictor filter for DMJLS.The original H ∞ problem ( 5)-( 6) can be rewritten in terms of the following optimization problem: where The H ∞ predictor filter exists at the instant  if and only if there exists a sequence {ŝ | } +1 =0 such that The prediction optimization problem is equivalent in nature to the filter problem, aforementioned.In the next theorem we present the H ∞ predictor filter whose proof follows the line of the filtered version.

Theorem 5. The Markovian H ∞ prediction problem (35) is solvable if, and only if,
where the sequence { Z+1| } and the predictor filter are calculated by the recursions In order to recover the original predicted and filtered state estimates of System (1), we calculate the following summations: where ẑ|−1 : x| is defined in the same way.
Remark 6.To establish the stability of the stationary H ∞ prediction filter, it is assumed that all matrices of (1) and the transition probabilities   are time invariant; System (1) is mean square stable (MSS) and its Markov chain {Θ  } is ergodic.There exists a unique positive-definite solution Z (with  → ∞) to the algebraic Riccati equation providing that Z−1 − −2  L  Λ −1 L > 0, for any   fixed, which guarantees that  ∞ > 0 and thus the above inverse is well defined.Following the guidelines defined in [1], the stability of the predictor filter is assured with where   (⋅) denotes spectral radius of the dynamic matrix of the filter with stationary gain.The asymptotic stability of the filtered version can also be assured following this line of argumentation.
Remark 7. Similar to the filtered case, we can also find a necessary existence condition to the Markovian H ∞ prediction filter given by Remark 8.If we consider   → ∞, the H ∞ filter proposed in this paper reduces to the Kalman filter for MJLS proposed in [1].For the case with no jumps ( = 1), the predicted and the filtered H ∞ estimates reduce to the H ∞ filters for statespace systems given in [17].

Numerical Example
In this section, we compare the filtering approach proposed with the filter developed in [4].With two Markovian states, the model ( 1) is defined based on the following parameters: We compare both filters in the predicted form, written in the following way: where The filter of [4] is considered in the strictly proper form as whose solution is given in terms of linear matrix inequalities.We show in Figure 1 the root-mean-square errors (rms) of both filters.We performed 1000 Monte Carlo simulations from  = 0, . . ., 6 with the values of Θ  generated randomly.The initial condition  0 is considered Gaussian with mean [0.196 0.295]  and variance [ 0.0384 0.0578 0.0578 0.870 ]; Θ  ∈ {1, 2},   ,   , and V  are independent sequences of noises with   ,   , and   identity matrices with appropriate dimensions,  1 (0) = 0.05, and  2 (0) = 0.95.We obtained   = 1.9523 for the filter proposed in this paper and  = 0.2886 for the filter proposed in [4].The parameter matrices of our filter were computed as In spite of the smaller  provided by the filter of [4], the rms of both filters are equivalent.This difference in  is due to the fact that the functionals of both approaches are different in nature.However, if we consider the same numerical value of all parameters but with matrices  1 and  2 multiplied by 10 6 , we obtain for the proposed recursive filter   = 0.001.On the other hand, we obtain an infeasible solution with the LMI proposed in Section IV of [4] (computed through Matlab 7.4.0).The mode-independent filter provided in [4] deals with only sufficient conditions.Considering offline computations, our approach provides necessary and sufficient conditions for the existence of the filter.
It is important to recall that the proposed approach enables all parameter matrices of (1) to be time varying.It also provides a practical estimate for the minimum admissible  through the necessary condition (27), which depends on only known parameters of the Markovian system.

Conclusion
This paper developed H ∞ predicted and filtered estimates for DMJLS.They were deduced based on the assumption that jump parameter is not accessible.The numerical example showed the effectiveness of this approach.Through an augmented model of the standard Markovian system, these filters were deduced based on the game theory.It allows us to deal with recursive algorithms to solve this kind of filtering problem where the parameter matrices can be time varying for each operation mode of the Markovian system.As future works, we intend to solve nonlinear filtering and control problems for DMJLS based on the problems raised by [12,18].

Figure 1 :
Figure 1: Root-mean-square errors for H ∞ filters based on Riccati recursive equation and linear matrix inequalities.
The sequence of solutions  | and  +1| is outputs of the respective H ∞ filters.Due to the hybrid nature of this class of system, at each instant of time a new model of a countable set of models is used to calculate the functionals (