Fault Detection Filter Design and Optimization for Switched Systems with All Modes Unstable

This problem of intelligent switched fault detection filter design is investigated in this article. Firstly, the mode-dependent average dwell time (MDADT) method is applied to generate the time-dependent switching signal for switched systems with all subsystems unstable. Afterwards, the switched fault detection filter is proposed for the generation of residual signal, which consists of dynamics-based filter and learning-based filter. The MDADT method and multiple Lyapunov function (MLF) method are employed to guarantee the stability and prescribed attenuation performance. The parameters of dynamics-based filter are given by solving a series of linear matrix inequalities. To improve the transient performance, the deep reinforcement learning is introduced to design learning-based filter in the framework of actor-critic. The output of learning-based filter can be viewed as uncertainties of dynamics-based filter. The deep deterministic policy gradient algorithm and nonfragile control are adopted to guarantee the stability of algorithm and compensate the external disturbance. Finally, simulation results are given to illustrate the effectiveness of the method in the paper.


Introduction
Switched systems [1][2][3] are an important component of hybrid systems, which are composed of a series of subsystems and build the connection between nonlinear systems and linear systems. As an efficient way to deal with complex nonlinear systems, switched systems have attacked considerable attentions and were applied in the areas of both military and economics [4,5], such as industrial manufacturing, flight control, robotic control, process control, and so on. e problems of modeling, stability analysis [6], controller design [7], and filter design [8] have been investigated recently and fruitful encouraging results emerged [9]. To mention a few, the problem of switched fault tolerant controller design is investigated in [10], the eventtriggered controller design method for discrete-time switched systems is proposed in [11], and the stability and l 2gain analysis are given in [12]. e basic problem for switched systems lies in stability analysis [13,14]. e scholars have developed tools to analyze the stability in past decades, such as common Lyapunov function (CLF) method, MLF method, and persistence Lyapunov function method [15]. e CLF method is mainly applied for the switched systems with arbitrary switching, which means that the switched systems are stable if all the subsystems share a CLF. However, it is difficult to design a CLF for all subsystems. To obtain tighter bounds, the stability analysis can be developed by the aid of MLF, which is mainly applied to constrained switching. Moreover, in practical system, the switching signal always depends on state or time. Because it is difficult to obtain the state measurements, the switching signal is time-dependent in many situations. us, the average dwell time (ADT) method and MDADT method provide efficient solutions to deal with the stability analysis for switched systems with constrained switching. In the work of [16], the issue of stability analysis for switched positive linear systems is studied.
e ADT method and multiple linear copositive Lyapunov function are combined and sufficient stability criteria for stability analysis are given. However, it is straightforward that common parameters for all subsystems with different characteristic will lead to conservativeness, which motivates the researches on MDADT. e definition of MDADT is firstly proposed in [17]. e problem of stability analysis is studied in the framework of MDADT method and tighter bounds are obtained. In [18], the problem of stability and robust controller design for switched systems with external disturbances is studied. e MDADT switching and MLF method are introduced to ensure the stability. Moreover, in practice, the error of the switched system will lead to unstability. erefore, the researches of stability analysis for switched systems with unstable subsystems are still one of the most important topics in control areas. In [19], the piecewise Lyapunov functions and MDADT method are combined to deal with the problem of stability analysis for switched systems with unstable modes. e fast switching is applied to unstable modes and slow switching is applied to stable modes. In [20], the problem of stability for switched systems with input time delay is studied. e unstable subsystems and asynchronous switching caused by time delay are taken into consideration. e extended stability criteria are obtained by the aid of Lyapunov-Krasovskii function method and sufficient conditions for stability analysis are presented. e fault case of system will no doubt lead to undesirable response [21,22]. One issue in the industrial systems or aeronautic engineering is that the maintenance cannot be given immediately to ensure the reliability and safety [23].
us, the presence of undesirable fault and the possibility of the occurrence of faults has to be considered in the stability analysis and system design, which motivates the study on fault detection and fault tolerant control. As an efficient way to deal with the undesirable faults, fault detection technique has attracted more and more attention, which can be seen in the recent important results, such as [24][25][26]. e fault detection system can detect the fault in time and efficiently so that we can reconstruct the control diagram to adjust to the fault environment. Among these fault detection methods, the most valuable and applicable method is the model-based fault detection filter design method. It can be inferred in existing literature that the model-based method consists of a residual generator and residual evaluator, which is applied in the areas of networked control systems, aerospace engineering, and process control. In [27], the problem of H ∞ /H-fault detection for switched systems with unstable modes is investigated. e robustness to external disturbance and the sensitivity to fault are both taken into consideration. In [28], the problem of cooperative control for multiagent systems is investigated based on adaptive control and fuzzy control. e unknown control parameters and actuator fault are taken into consideration. e backstepping control is utilized to derive the controller. In the work of [29], the problem of fault detection subject to nonlinearities and disturbance is investigated according to event-triggered scheme. Sufficient conditions to guarantee the system is stable with the prescribed performance are obtained based on ADT method and MLF method. Based on the literature mentioned above, it can be inferred that the model-based fault detection filter is popularly applied in most of engineering problems because of the design simplicity and physical execution. However, it is difficult to achieve optimal compromise between robustness and transient performance. e model-based method can guarantee the stability and robustness of the closed-loop system. But the transient performance cannot be guaranteed. It is essential and significant to improve the transient performance of fault detection filters. Moreover, due to the lack of online-learning ability, this type of fault detection filters is relatively ineffective to have optimal performance in the real-world uncertain environment.
With the development of machine learning and computer science, the intelligent control has drawn considerable attention [30]. As a powerful nonlinear approximation approach, deep learning and deep reinforcement learning has promoted the considerable performance in realistic applications, such as controller parameters tuning, decision making, and so on. In particular, the DDPG algorithm has been illustrated to perform stably and efficiently on many high-dimensional action control tasks. In the work of [31], a noninteger PID controller based on DDPG algorithm is proposed for the tracking problem. To accomplish the control methodology, a kinematic controller and a dynamic controller are established independently, in which the kinematic controller is proposed based on the model of the vehicle and the dynamic controller is realized for the supplementary kinematic controller to achieve optimal performance. In [32], the assembly task is defined as a Markov decision process and a fuzzy DDPG algorithm is given to realize the task. To improve the performance and learning efficiency, a fuzzy reward system is developed for the assembly task. In addition, in the work of [33][34][35], the machine learning is applied in the design of controller and guidance law.
Inspired by the aforementioned discussion, an effort is conducted in the paper to design an intelligent fault detection filter for switched system with all modes unstable, which is composed of dynamics-based filter and learningbased filter. As well known, the model-based fault detection filter is designed in the existing literature, which can guarantee the stability of closed-loop system and the robustness to external disturbance. However, how to improve the transient performance of fault detection still remains an open problem. On the other hand, in most literature, the fault detection filter is proposed based on the assumption that all the subsystems are stable. But in many practical situations, the subsystems may be unstable, which motivates the study in this paper. e robust control theory is applied to establish the dynamics-based filter. e switched system with all modes unstable is considered and the fault detection filter is presented for generation of the residual signal. e 2 Computational Intelligence and Neuroscience MDADT method and MLF method are combined to ensure the stability and prescribed attenuation performance index of switched system. To achieve optimal performance, the learning-based filter is introduced based on DDPG algorithm in the framework of actor-critic, in which the filter parameters are optimized by online learning. e output of learningbased filter can be viewed as variation of the parameters of dynamics-based filter. us, the nonfragile control theory is introduced to guarantee the stability of switched systems. e main contributions of this article are stated as follows: (1) An observer-based filter is proposed to deal with the problem of fault detection for switched system with all modes unstable, in which the stability and attenuation performance index can be guaranteed by MDADTmethod and MLF method. Compared to the existing results, the characteristic of each subsystem is considered, and tighter bounds can be obtained, which provides more room for the improvement of flexibility.
(2) e learning-based fault detection filter is presented based on DDPG algorithm to achieve optimal performance, which overcomes the undesirable response caused by external disturbance and uncertainties. e action is defined by the variation of parameters of the fault detection filter. (3) e nonfragile control theory is applied to ensure the stability of closed-loop system, for the reason that the output of actor network is viewed as the variation of parameters of dynamicsbased filter. e rest of the paper is presented as follows: the model of switched system with all modes unstable is established in Section 2, in which the fault detection filter is proposed to generate the residual signal. In Section 2, the main results of fault detection are proposed, which is composed of dynamics-based filter and learning-based filter. e numerical example is given in Section 4 to validate the effectiveness of proposed method, which is followed by the conclusion in Section 5.

Problem Statement
e continuous-time switched system in this paper can be described as follows: , and F σ(t) are system matrices with appropriate dimensions.
To improve the transient performance of fault detection filter and achieve optimal performance, it is supposed that the fault detection filter in this paper is composed of two parts: the dynamics-based filter and the learning-based filter. e dynamics-based filter is designed according to robust control theory and the learning-based filter is proposed based on deep reinforcement learning, which can be described as follows: (2) L r,σ(t) is the parameters of dynamics-based fault detection filter, which is determined by robust control theory; L d,σ(t) is the compensation for external disturbance, which is generated by the aid of DDPG algorithm. To ensure the stability of DDPG algorithm, the compensation can be viewed as the variation of parameters. us, the stability of optimization algorithm is guaranteed by nonfragile control theory. e parameters obtained by DDPG algorithm can be written as M i and N i are the known matrices with appropriate dimensions; E i are unknown matrices satisfying E T i E i ≤ I.

Remark 1.
e designing process can be divided into two steps: (1) regarding the design of robust control theory, it is proposed to ensure the stability of closed-loop system and prescribed attenuation performance. (2) e deep reinforcement learning is utilized to improve the performance of fault detection, where the additional compensation of fault detection filter is viewed as the action of the agent. en the parameters of fault detection filter are optimized based on DDPG algorithm in the framework of actor-critic. Compared with the traditional methods, not only can the stability be guaranteed by robust control theory, but also the transient performance of fault detection can be improved based on DDPG algorithm.

Remark 2.
e problem of fault detection for switched systems with all modes unstable is studied in this paper. e unstable modes are taken into consideration and it is more applicable.
e MDADT method and MLF method are combined to analyze the stability and tighter bounds on dwell time can be obtained than the traditional ADT method. ere is more room for the switched systems to dwell long enough time to decrease the system energy.
erefore, the method proposed in this paper will improve the design flexibility of fault detection system. e dynamics-based fault detection filter is proposed to generate the residual signal, which can be described as follows: where x(t) is the state of fault detection filter; y(t) denotes the estimation of output signal y(t); r(t) is the residual signal and L σ(t) denote the parameter matrices of fault detection filter to be determined. Define the error of state measurement as e(t) � x(t) − x(t); the error of fault estimation r(t) � r(t) − f(t). us, we set the augmented state vector T and the augmented input signal as T , so we can obtain the closedloop switched system as follows: where en we can conclude that the design of robust fault detection filter can be converted by the problem of H ∞ filter design. erefore, for given prescribed attenuation performance index c w , the closed-loop switched system in equation (5) is globally uniformly exponentially stable (GUES) when ω(t) � 0; the following inequality holds for all nonzero ω(t) under zero-initial condition.
e residual signal is generated by fault detection filter; it is necessary to design a residual evaluator, which consists of an evaluation function and a threshold. e evaluation function and threshold can be written as η denotes the length of time window of evaluation function. Based on residual evaluator and threshold, the decision logic can be expressed as follows:

Robust Filter
Design. e definitions and lemmas are given as follows to make the proof convenient.
Definition 1 [17]. For given switching logic σ(t) and time en, τ ai is called the MDADT of fast switching and N 0i is called the chattering bounds; we set N 0i � 0 in this paper.
Lemma 1 (see [27]). For given matrices S, T and symmetric matrix Y, if there exist constant scalar κ > 0, such that Theorem 1. For given constant scalars 0 < μ i < 1, λ i > 0, if there exist positive definite matrices P i , such that then the switched system in equation (5) is GUES if MDADT satisfies the following equation: Proof. We set the Lyapunov function as follows: us, we have (17) Combining equation (14) and ω(t) � 0, we can conclude that It is supposed that the switching instants during time interval [0, t] are set to be t 1 , t 2 , . . .，t k with t k+1 � t; we can derive that Together with Definition 1, we have 4 Computational Intelligence and Neuroscience If the MDADT of switched system satisfies equation (15), we have Combining with Definition 2, one can conclude that the switched system in equation (5) is GUES when ω(t) � 0. □ Theorem 2. For given constants 0 < μ i < 1, λ i > 0, c > 0, if there exist positive definite matrices P i , such that en, the switched system with MDADT satisfying equation (15) is GUES with prescribed attenuation perfor- Proof. Define the Lyapunov function in equation (16); we can obtain the following equations under zero-initial condition.
Based on Schur complement, we have According to the statement above, we have Computational Intelligence and Neuroscience 5 Together with the condition V σ(t) (t) ≥ 0, one can conclude

Computational Intelligence and Neuroscience
Multiplying both sides of equation (26) Together with equation (21) and the condition in equation (28), we can obtain equation (29).
and it can be inferred that Integrating both sides of equation (30) from 0 to ∞, we have

Computational Intelligence and Neuroscience
We can conclude that the switched system in equation (5) is GUES with prescribed attenuation performance c d � c 2 μ −k/2 min . □ Theorem 3. For given constant scalars 0 < μ i < 1, λ i > 0, and c > 0, if there exist positive definite matrices P i , such that en, the switched system with MDADT satisfying equation (16) is GUES with prescribed attenuation performance c d ; the parameters of fault detection filter can be given by where Proof. It can be inferred that equation (14) can be rewritten as where us, by the aid of Schur complement, we can conclude that the switched system in equation (5) is GUES with prescribed performance c d ; the parameters of fault detection filters can be obtained by equation (33).

Remark 3.
e learning-based fault detection filter is viewed as the compensation of the robust filter. erefore, we can obtain the scheduling interval to ensure the stability and prescribed attenuation performance by the aid of nonfragile control theory in eorem 3. e upper bounds on action in DDPG algorithm can be obtained by the predefined scheduling interval.

Intelligent Filter Design.
e stability and prescribed attenuation performance are guaranteed by the theorems aforementioned. However, the transient performance needs to be improved. To solve the problem, deep reinforcement learning is applied in the framework of actor-critic. e optimization of parameters of fault detection filter can be viewed as an infinite Markov decision process, which is a series of continuous optimization processes.
us, the DDPG algorithm is developed in this paper to improve the performance of the filter. e frame of reinforcement learning consists of an agent and the environment. e state at kth time instant is defined as s k , an action a k is chosen by the agent, and then a reward function r k and the state of next step are obtained, where r k is developed to evaluate the performance of state-action pair generated by the agent. e fault detection system is viewed as the environment. We define the additional compensation L d,σ(t) as the action, which is utilized to maximize the sum of the expected discounted reward function over given future steps. e action and the sum of expected discounted reward function can be described in equations (36) and (37).
where c f ∈ [0, 1] is defined as the discount factor and K f represents the terminal step. e state of the agent is given in the following equation: e DDPG algorithm is proposed in the framework of deep Q learning and actor-critic. ere are two actor networks and two critic networks. e optimal policy is tried for realization based on policy gradient theory in continuous action spaces. e action-value is approximated by employing the critic network Q(s k , a k |ϖ Q ), whose weights are defined as ϖ Q . e current output of compensated parameters is generated based on the actor network a(s k |ϖ a ), whose weights are defined as ϖ a . e weights ϖ Q are updated according to the loss function, which is described in equation the following: where y k � r k (s k , a k ) + c p Q(s k+1 , ϖ(s k |ϖ a )|ϖ Q ). e weights of actor network are updated according to the policy gradient theory, which is given in equations (40) and (41). 8 Computational Intelligence and Neuroscience where L an represents the learning rate of actor network. Moreover, in the DDPG algorithm, two networks are adopted as actor target network and critic target network, which are defined as a ′ (s k |ϖ a′ ) and Q ′ (s k , a k |ϖ Q′ ). e weights of actor target network a ′ (s k |ϖ a′ ) are defined as ϖ a′ , which are updated according to the following equation: L at denotes the learning rate of actor target network. Similarly, the weights of critic target network Q ′ (s k , a k |ϖ Q′ ) are defined as ϖ Q′ , which are updated according to the following equation: L ct represents the learning rate of critic target network. In order to improve the robustness of the proposed algorithm, an exploration noise is introduced as a compensation of the output of actor network, which can be generated based on a k � π s k |ϖ a + N a . (44) erefore, based on the statement above, the pseudocode of intelligent fault detection filters design can be presented in Algorithm 1.

Remark 4.
e DDPG algorithm is developed in this paper to improve the transient performance of the fault detection filter. Compared with the traditional method, the stability, robustness, and dynamic performance can be guaranteed simultaneously, in which the robust control theory and nonfragile control theory are introduced to ensure the stability, and the compensation of controller can be viewed as the variation of predesigned controller. erefore, the stability of closed-loop system can be guaranteed by nonfragile control theory.

Numerical Example
In this section, simulation results are given to validate the effectiveness of the proposed method. e system matrices are given as follows: (1) Construct the fault detection filters for switched systems (2) Set the variation range of fault detection filters (3) Stability analysis and calculate the MDADT based on Eq. (15) (4) Design the fault detection filters based on eorem 3 (5) Randomly generate the initial weights of the actor network and critic network (6) Initialize the critic target network and actor target network (7) Initialize the Replay buffer and define episode � 0 (8) for episode � 1 to M do (9) Initialize a random noise process N a to improve the robustness (10) Initialize a random state vector s 1, then the initial observation can be obtained (11) for t � 1 to K do (12) Chose action a k � π(s k |ϖ a ) + N a based on the current state s k and noise in the environment (13) Calculate the reward function r k and state s k+1 (14) Store the transition pair (s k , a k , r k , s k+1 ) in the Replay buffer R (15) Sample a random minibatch of N transitions (s m , a m , r m , s m+1 ) from the Replay buffer (16) Set Update the critic network, actor network and target networks (18) end for (19)  It can be inferred that the subsystems A 1 and A 2 are unstable, respectively. e external disturbance is defined as follows: e other parameters of switched systems are listed as follows: erefore, the MDADT of switched systems are τ 1 � 0.5076, τ 2 � 0.4842. e prescribed attenuation performance index is set to be c � 0.8. en the parameter matrices of filters can be obtained in eorem 3.
To validate the effectiveness of the proposed method, we give two numerical examples. e stability criteria of the switched systems in eorem 2 are demonstrated by Example 1. Moreover, the effectiveness of the fault detection filter in eorem 3 and the DDPG algorithm is illustrated in Example 2.
Example 1. Firstly, we provide the proof that the proposed conditions can ensure the stability of switched systems. As a comparison, a randomly generated switching signal is    introduced to show that the switched systems cannot stay stable if all the subsystems cannot share a CLF. e switching logic satisfying equation (20) is given in Figure 1 and the randomly generated switching logic is given in Figure 2. e state response of the proposed switching logic is given in Figure 3 and the state response of randomly generated switching logic is showed in Figure 4. We can see that the switched systems cannot ensure the stability under randomly generated switching logic. However, the stability can be guaranteed according to eorem 1 in the paper.      Figure 13; we can see that the reward function can converge to a neighbor of the origin, which illustrate the effectiveness of the optimization algorithm.
In summary, we can see that the tighter bounds on dwell time and less conservative results are obtained. e stability of switched systems can be guaranteed by the proposed results in this paper. e fast switching strategy is applied, and it allows the subsystems of switched systems to remain unstable. e transient performance of fault detection filters can be improved by the aid of DDPG algorithm, and the stability, robustness, and optimal policy can be guaranteed simultaneously by the method proposed in this paper.

Conclusions
e problem of fault detection and online scheduling for switched systems with all modes unstable is studied. e observer-based fault detection filter is proposed to generate the residual signal, which consists of two parts: the dynamics-based fault detection filter and the learning-based fault detection filter. By employing MDADT method and MLF method, the stability of the switched systems is guaranteed. e solutions of fault detection filters are given in the form of LMIs. To achieve optimal control policy and improve the transient performance, the DDPG algorithm is utilized as learning-based fault detection filter, in which the output can be viewed as the variation of robust fault detection filter. erefore, the nonfragile control is provided to ensure the stability of the optimization algorithm. Finally, the simulation results are introduced to demonstrate the effectiveness and superiority of the proposed method.

Data Availability
e data used to support the findings of this study are included within the article.  14 Computational Intelligence and Neuroscience