Fault Detection of Markov Jumping Linear Systems

In this paper, the fault detection FD problems of discrete-time Markov jumping linear systems MJLSs are studied. We first focus on the stationary MJLS. The proposed FD system consists of two steps: residual generation and residual evaluation. A new reference model strategy is applied to construct a residual generator, such that it is robust against disturbances and sensitive to system faults. The generated residual signals are then evaluated according to their stochastic properties, and a threshold is computed for detecting the occurrences of faults. The upper bound of the corresponding false alarm rate FAR is also given. For the nonstationary MJLS, similar results are also obtained. All the solutions are presented in the form of linear matrix inequalities LMIs . Finally, a numerical example is used to illustrate the results.


Introduction
The complexity and automation degree of modern technical systems are continuously increasing.In order to guarantee the system safety and reliability, the model-based fault detection and isolation FDI technology has been developed since the early 1970s 1-3 .It is fully integrated into many industrial processes and automatic control systems, which can provide valuable information about system faults.The basic idea of model-based FDI is to generate a residual signal based on the system model and measurements and to determine a residual evaluation function to compare with a threshold regarding to all possible model uncertainties and unknown inputs.Exceeding the threshold indicates a fault in the system.Hence the FDI technology consists of two steps: residual generation and residual evaluation.As mentioned in the survey papers 4, 5 and book 2 , the robustness issues have been studied in the purpose of designing of the FDI system under a cost function that expresses a tradeoff between the system robustness against unknown inputs as well as model uncertainties and the system sensitivity to faults.A unified solution for the fault detection system design was presented in 5, 6 for continuous and discrete time linear systems, respectively.In 7, 8 , an H ∞ -filtering formulation of the robust FDI design for uncertain system has been proposed and widely accepted, which tries to make the difference between residual and so-called weighted fault as small as possible by considering model uncertainties and unknown inputs.
Markov jump linear systems have also attracted a great deal of attention.The MJLS is one of the hybrid systems in which a state takes values in a countably finite set, referred to as the state.It can be used to represent a class of linear systems subjects to abrupt changes in their structures due to random components failures, repairs, sudden environment disturbance, change of the operation point of a linearized model of nonlinear systems, for example, electric power systems, aircraft flight control and especially networked control systems for reasonable model of the packet delivery characteristic in communication channels .There are many theoretical works contributed in the field of MJLS.In 9, 10 the results on stability of MJLS were presented.The linear quadratic Gaussian control problem was studied in 11, 12 .The bounded real lemma for MJLS has been fully developed by 13 in the form of LMIs.The H ∞ -control problems were discussed in 14, 15 , where a controller stabilizing a linear system ensures that the l 2 induced norm from unknown inputs to the outputs is bounded.In 16,17 , the H ∞ filtering for MJLS was studied as the dual problem of control.For networked systems, recently there were also many new results obtained by applying the MJLS theorem, for example, in 18-20 .Although there is intensive research in FD and MJLS, the design of FD system for MJLS has just begun.In 21 , the packet loss in networked control systems was modeled as Markov process, and an FD system has been designed in terms of LMIs.In 22 , the fault detection system over noisy communication channels was also described via a MJLS.In 23 , the design of FD for MJLS was formulated as a H ∞ -filtering problem.In those works, observer-based fault detection filters were designed to minimize the influence of unknown inputs 21, 22 or to minimize the difference between residual signals and weighted faults 23 , and the threshold is simply computed based on the expectation of the norm of residual signals.However, by only considering the unknown inputs or faults in the design, the residual generator usually cannot achieve an optimal performance in sense of the tradeoff between the system robustness and the fault sensitivity 2 .Besides, further statistic properties of residual signals were not analyzed and considered in the system design, and a proper residual evaluation scheme for MJLS is still missing to the best of the authors' knowledge.We will show that the usual norm-based residual evaluation method, in which the norm of residual signals is used as evaluation function and compared with the threshold, cannot be directly applied in FD of MJLS.Without taking the variance of residual signals into account, the evaluation will result in a possible high FAR.
In this paper, the fault detection system design of MJLS is formulated as a set of optimization problems.The residual generator is designed to stochastically match an deterministic reference residual model which can achieve an optimal tradeoff between the system robustness and the fault sensitivity.The reference residual model is selected according to the statistic properties of the MJLS.For the stationary MJLS, in which the distributions of Markov state at different time instances remain the same, a new bounded real lemma is also derived, and the constraints of the expectation and variance of residual signals are considered in the residual generator design.The stochastic model matching problem is then solved by optimizing the H ∞ -norm of an MJLS in terms of LMIs.In the residual evaluation, the absolute value of each residual signals is selected as the evaluation functions, and the corresponding threshold is computed by considering their expectations and variances.Those statistic properties of evaluated residual signals are calculated with the help of LMIs based on convex optimization problems.An upper bound of the FAR is also derived for this evaluation scheme.The FD problem of nonstationary MJLS is then addressed in a similar way.Finally a numerical example is given to illustrate the feasibility and effectiveness of the proposed design approach.
The rest of this paper is organized as follows.The problem of fault detection of MJLS is formulated in Section 2. Section 3 presents the design of residual generator and the new residual evaluation approach.In Section 4 the FD system design for nonstationary MJLS is discussed.An example is given in Section 5 and we conclude the paper in Section 6.
Notation.The notation used throughout the paper is fairly standard.The superscript "T" stands for transposition.R n denotes the n-dimensional Euclidean space and the notation S > 0 means that S is real symmetric and positive definite.diag stands for a block-diagonal matrix.E ϑ means the expectation of ϑ.For convenience we denote ϑ E ς .The notation • j is used to represent the jth row of a matrix or a vector.When w k ∈ l 2 space, its norm is given by and if w k is a stochastic value and its expectation is in l 2 space, then we denote

Problem Formulation
The considered MJLS is defined as follows: where x ∈ R n denotes the state vector, u ∈ R p denotes the control inputs, y ∈ R m denotes the measured output vector, d ∈ R n d denotes the unknown inputs, and f ∈ R n f is the fault to be detected.{θ k } is a discrete homogeneous Markov chain taking values in a finite state space ψ {1, 2, . . ., N} with transition probability matrix Φ λ ij i,j∈ψ , and λ ij is defined as which are subjected to the restriction λ ij ≥ 0, N j 1 λ ij 1 for any i ∈ ψ.For notation, define Θ k : {θ 1 , . . ., θ k } as a sample of possible sequences of Markov states and P k as the vector of state probabilities

2.3
In this paper we consider the Markov chain with the following assumption.
This assumption means that, in such a Markov chain, it is possible to get to any state from any state.

Residual Generator Design
Residual generation is the first step of FD.We propose the following residual generator for the system 2.1 : where x k and y k are the estimated state vector and output vector, respectively.r k is the residual vector.The matrices L θ k and W θ k are to be designed.For the convenience, we denote the matrices associated with θ k i ∈ ψ by

2.5
With e k x k − x k , the residual dynamics of 2.4 can be written as with

2.7
The objective of the design is to generate the residual signals which is robust to unknown inputs and sensitive to faults.It is clear that 2.7 itself can be an MJLS instead of a deterministic one.The Markov state θ k is not known as a perquisite, while only its probability is known.Hence we propose a reference residual model which achieves an optimal tradeoff between system robustness and fault sensitivity, and then the residual generator is designed to match the reference residual model.In this approach the matrices L θ k and W θ k in 2.4 should be selected such that min where α > 0 and r ref denotes the residual vector of a reference residual model in the form of

2.10
Here L o and W o are chosen by applying the unified solution proposed in 6 , such that sup Since r k is stochastic vector, usually the expectation, r ref −r E , is not enough to characterize its behavior.Hence the constraint 2.9 is applied to ensure that the summation of variances of each residual signal is bounded with an expected value α 2 .In this approach the expectation and variance of r k are both considered in the model matching problem for MJLS, such that the stochastic r k could approach a deterministic r ref k .
Remark 2.1.A significant difference between the reference model for the purpose of FD adopted here and the one in most of the literatures is that unknown inputs d k are included in our model, such that an optimal tradeoff between system robustness against unknown inputs and sensitivity to faults can be achieved.Simply reducing the influence of d k or increasing the sensitivity to system faults f k does not automatically lead to an optimal tradeoff between system robustness and fault sensitivity.For instance, with a residual generator which decouples d k from r k , some f k may also be decoupled from r k and thus cannot be detected.Hence it is necessary to take unknown inputs into account in the reference model.

It is well known that
where P ∞ is a unique constant vector called the stationary state distribution of a Markov chain with the assumption A1 , and P ∞ can be computed according to This means that as time goes by, the Markov chain forgets its initial condition and converges to its stationary distribution.Hence it is reasonable to set the following matrices: so that the reference residual model describes the optimal stationary expected behavior of the MJLS 2.7 .
In fact, the MJLS 2.7 consists of two groups of system state: e k and θ k .In the robust control system design, the initial condition of system state is usually assumed to be zeros socalled zero initial condition .Comparably, we make three assumptions in MJLS for the rest of the paper: A3 e 0 is deterministic and e 0 0, A4 θ k is independent of e 0 .
Assumption A2 can be regarded as the "zeros" initial condition of θ 0 .It is worthy to mention that, with assumption A2 , we have P 0 That means the state probabilities is independent of time, that is, the Markov chain is in the stationary state.We call MJLSs with assumption 2.4 as stationary MJLSs.For the FD purpose, we can generally assume that the MJLS under consideration is operating in its stationary state before a fault occurs.We denote p i k p i for all k for later use.The problem of residual generator design of a stationary MJLS is summarized as follows.

Problem RGFD [Residual Generator for Fault Detection]
Consider system 2.1 .Given the reference residual model 2.10 , determine the matrices L i and W i , i ∈ ψ of the residual generator in the form of 2.4 under assumption A1 -A4 , such that the residual dynamics described by 2.7 satisfy 2.8 and 2.9 .

Residual Evaluation Design
The second step in FD is the evaluation of residual signals, where r k is evaluated and compared with a threshold.The following logic is applied to detect the occurrences of faults r e ≤ J th ⇒ fault-free, r e > J th ⇒ fault alarm,

2.15
where r e denotes the evaluated residual signals and J th denotes a threshold.The residual evaluation problem of deterministic systems has been extensively studied.One important evaluation strategy is the so-called norm-based residual evaluation 2 .Assume the dynamics of a residual generator for some deterministic systems are governed by the following time invariant system where d k is unknown inputs, and f k the system faults.Then the residual evaluation function is chosen as with s being the length of the evaluation window.It is clear that the residual signals r k are corrupted with d k .Hence threshold is set to distinguish the faults from the unknown inputs.As widely accepted in the literatures, the threshold is set as such that false fault alarms can be prevented and meanwhile missing detection of faults can be reduced as much as possible.
The residual signals of MJLS are stochastic values.Their statistic properties are associated with the Markov process, and they are determined by the dynamics of the system 2.4 .It is possible to compute γ such that in a similar way as stated in 13 for deterministic systems and to set J th γ η k 2 as in many literatures.In this case only the expectation of the l 2 -norm of r k is considered for the computation of J th .Due to the variance of r k , there could be false fault alarms and the probability of those false alarms also called false alarm rate FAR is not known.

Mathematical Problems in Engineering
Even a fault is detected, it is difficult to say with how large percentages the fault alarm is correct.Therefore, we apply the residual evaluation approach proposed in 24 .Define a set of residual evaluation functions as follows: where j 1, . . ., m and r j k is the jth measurement.That means the absolute value of each residual signal is selected as the evaluation function.For the evaluation function 2.20 , we suggest the following threshold: where r j k is the absolute value of the expectation of r j k and is its variance, where β > 0 is some constant.
The residual evaluation problem is summarized as follows.

Problem REFD [Residual Evaluation for Fault Detection]
Consider system 2.1 .Given the residual generator 2.4 and assumptions A1 -A4 , determine the threshold J j,th , j 1, . . ., m for each residual signal, that is, sup k | r j k | , sup k σ j r k and β > 0, such that FAR is smaller than a given constant.

Residual Generation
In this section the dynamics of r k − r ref k are described at first.Then the existing bounded real lemma BRL for MJLS is reviewed, and a new BRL is derived for MJLS under assumption A2 .Based on those lemmas, the solution of RGFD is presented.
According to 2.4 and 2.10 , the dynamics of r k − r ref k can be written as

3.3
Before giving the solution, we first introduce the following useful lemmas.The first one is the standard BRL for MJLS given in 13 .if there exist Q i > 0, i ∈ ψ satisfying the following LMIs:

Lemma 3.1. Consider the system
where Proof.Here we give a short proof which is slightly different from the one in 13 .Define the function Mathematical Problems in Engineering Given x 0 0, V θ 0 , 0 0 for any initial mode θ 0 ∈ Ψ, we have Then, where With R θ k < 0, for any θ 0 ∈ Ψ and d k ∈ l 2 , we have E e 0 ,θ 0 y 2 < γ d 2 .
Recall that Θ k is any possible sequences of Markov state.By taking expectation over Θ k , we can obtain 3.6 which implies R θ k < 0 for each k.Thus the lemma is proved.Remark 3.2.Lemma 3.1 assumed that θ 0 is deterministic.However, it can also be used when there is no assumption made on θ 0 as shown in the proof.
The second lemma gives an equivalent expression of 3.6 .Lemma 3.3.Consider the system 3.4 .Given a constant γ > 0, x 0 0 and any θ 0 ∈ ψ, then 3.6 with Q i > 0, i ∈ ψ are feasible, if and only if there exist matrices Q i > 0 and G i > 0 such that the following LMIs By observing that θ k is independent of θ k−1 under assumption A2 , we can obtain the following bounded real lemma.Lemma 3.4.Consider the system 3.4 , where x k , y k , A θ k , B θ k , C θ k , D θ k , and θ k are defined as in 2.1 , d k ∈ R m is the l 2 -norm bounded input sequence.Given a constant γ > 0, x 0 0 and under assumption (A2), then 3.5 is satisfied, if there exist S > 0 satisfying the following LMIs: Proof.Under the assumption A2 , the expectation terms in 3.10 are

3.14
Hence R θ k in 3.10 can be rewritten as Applying Shur complement and congruence transformation with diag{S, I, . . ., S, I, I, I}, R < 0 can be formulated as 3.13 .Remark 3.5.When the number of modes of {θ k } is 1, Lemmas 3.1 and 3.4 reduce to the standard bounded real lemma for deterministic systems 25 .It is clear that inequality 3.6 in Lemma 3.1 implies 3.13 .But Lemma 3.4 not only requires less computational efforts, but also provides less conservative results as shown in the following example.
The fourth lemma is given to compute the bound of variances for a stationary MJLS.Lemma 3.7.Consider the system 3.4 , where x k , y k , A θ k , B θ k , C θ k , D θ k , and θ k are defined as in 2.1 , d k ∈ R m is the l 2 -norm bounded input sequence.Given a constant γ > 0, x 0 0, γ > 0 and under assumption (A2), then 3.17 if there exist S > 0 satisfying the following LMI:

3.19
Proof.The expected behavior of 3.4 under assumption A2 is described by

3.20
Then the dynamics of r k − r k can be written as

3.21
Following the similar procedure in Lemma 3.4, the result can be obtained.Now we are in the position to give the theorem for the solving RGFD.
Theorem 3.8.Given the system 2.1 , a constant α and under assumptions (A1)-(A4), the optimal L θ k and W θ k in the residual generator 2.4 in the sense of minimizing 2.8 and satisfying 2.9 can be obtained by solving the following optimization problem

3.25
for i ∈ ψ and

3.26
The optimal L i is then given by S −1 1 Y i .
Proof.With Lemmas 3.4 and 3.7, the proof is straightforward, and thus omitted.

Residual Evaluation
In this section, the solution of REFD is given.The bounds of the expectation and variance of 2.20 are firstly computed by using the peak-norm and l 2 -norm of unknown inputs.Then the threshold is determined, and the upper bound of the guaranteed FAR is obtained.
The following lemma gives the computation of | r j k | in terms of norm.

3.31
The expected residual dynamics are governed by 3.30 , which is time invariant system.The results can be easily obtained following the idea in 25 .Thus the rest of the proof is omitted.
The variance of r j k can be computed in terms of peak-norm using the following lemma.

3.35
Proof.From 2.7 and 3.30 , we have

3.36
Define for some Q i > 0 and assume that is bounded by 3.38 .By denoting θ k i, the inequality p i μ i , 3.43 ensures 3.39 and thus 3.38 .Noticing that Mathematical Problems in Engineering can be obtained which implies

3.47
Applying Shur complement and congruence transformation, 3.41 and 3.46 can be reformulated as 3.33 and 3.34 , respectively.
We propose also methods to compute the expectation and variance of r k based on l 2 -norm.The following lemma gives the computation of |r j k |.Lemma 3.11.Given system 2.7 , γ j,1 > 0, γ j,2 > 0 and under assumption (A1)-(A4), then if there exist S > 0, i ∈ ψ satisfying the following LMIs: Proof.The proof is similar with Lemma 3.9.
The computation of σ j k based on l 2 -norm is given in the following lemma.

Mathematical Problems in Engineering
By denoting θ k i, the inequality 3.57 is equivalent with

3.60
Then 3.52 is guaranteed, if
Based on above results, the following theorem gives the solution of REFD.

3.66
The false alarm rate is upper bounded as 3.67 Proof.The theorem can be proved in a similar way as in 24 .
Now we have the threshold for each residual signal.The fault can be detected if one evaluated residual exceeds its threshold, for example, r j k ≤ J j,th ⇒ fault-free, r j k > J j,th ⇒ fault.

3.68
A false alarm occurs when there is no fault but |r j k | > J j,th .Its probability is upper bounded by 3.67 .Remark 3.14.If the number of Markov state is 1, then the computed σ 2 j r k will be zero.The proposed approach reduced to the standard norm-based residual evaluation methods 2 .
Remark 3.15.The evaluated residual |r j k | is a stochastic variable.By using the expectation and variance of |r j k | for the computation of the threshold, an upper bound of FAR is obtained.Such a bound is very useful in practice, as it can provide confidential information for a rising fault alarm.Without considering the variance in the residual evaluation, the FAR can be very high and no bounds of FAR can be established.Since there is unknown inputs in the system, we can only derive the upper bounds of the expectation and variance.Its higher order moments are difficult to obtain, which is lack of physical means.

Discussion on FD of Nonstationary MJLS
Without the assumption A2 , the MJLS could be in nonstationary state, that is, P k / p ∞ .In this case the expectation of r k is difficult to obtain, and 2.9 cannot be established.Furthermore, the residual evaluation approach presented in last section cannot be applied.Hence we propose also an FD system for nonstationary MJLS.Firstly, the residual generator is designed to satisfy 2.8 , and the solution is given in the following theorem.
where the nonzero elements of N pq are

4.3
where The optimal L i is then given by G −1 i11 Y i .
Proof.By applying Lemmas 3.1 and 3.3 and setting Y i G T i11 L i , the LMI 4.2 can be easily obtained for system 3.1 .Notice that implies that G i11 is nonsingular.Therefore, the feasibility of 4.2 always ensure the existence of optimal L i and W i .
Then 2.17 is selected as the evaluation functions, and the threshold is suggested as Hence the FAR is bounded by 1/β 2 with the threshold 4.7 .The result is summarized as the following theorem.
Remark 4.3.The evaluation method suggested in Theorem 4.2 can also be used for the MJLS in stationary state.The variance of r k E , which is difficult to obtain, is not involved in the computation of the threshold.When the variance of is very small, this evaluation method can be fairly conservative.

Numerical Example
To illustrate the proposed method, the following two-mode discrete-time MJLS is considered:

5.5
During the simulation, assume that the unknown inputs are discrete-time random noises with power 0.5, the fault appears at the 500th discrete time as a step function of amplitude 15.The residual signals are evaluated as in 2.20 and the thresholds are computed based on peak-norm and l 2 -norm as in 2.20 .We take the second residual signal of the system as an example.The thresholds are computed according to Theorem 4.1 with β 3, that is, FAR < 11.1%.The results are shown in Tables 1 and 2, respectively.The simulation results are given in the corresponding figures, where J e is the threshold calculated only based on the expectation of residual signals.

Conclusion
In this paper a complete design approach of FD system for stationary and nonstationary MJLS was proposed.The main focus was how to deal with the stochastic properties of MJLS.First the residual generator was designed to stochastically match an optimal reference residual model in order to achieve the best tradeoff between robustness against system unknown inputs and sensitivity to faults, where the reference residual model was selected based on the stationary expected behavior of the MJLS.Then novel residual evaluation methods based on peak-norm and l 2 -norm were presented for MJLS, which can guarantee an expected FAR and meanwhile reduce missing detection of faults.In those evaluation methods, not only the expectation of evaluated residuals but also their variances were taken into account for the computation of thresholds.The proposed FD system can provide confidential information of occurring faults, which allows a practical application in real physical systems.
the matrices associated with θ k i ∈ ψ by and θ k are defined as in 2.1 , d k ∈ R m is the l 2 -norm bounded input sequence.Given a constant γ > 0, x 0 0 and any θ 0 ∈ Ψ, then sup

Theorem 4 . 1 .
Given the system 2.1 under assumptions (A1), (A3)-(A4), the optimals L θ k and W θ k in the residual generator 2.4 in the sense of minimizing 2.8 can be obtained by solving the following optimization problem for all i ∈ ψ:

F
Figures show that many false alarms arise in the first 500 time steps with J e , and the number of false alarms is significantly reduced by considering the variances in the threshold computation.The fault is detected by all four thresholds.
.7 where sup d∈l 2 r k E can be easily obtained by applying Lemma 3.1.According to Markov inequality 26 , we have