Extremum Seeking Based Fault-Tolerant Cooperative Control for Multiagent Systems

We propose a novel fault-tolerant cooperative control strategy for multiagent systems. A set of unknown input observers for each agent are constructed for fault detection. Then a real-time adaptive extremum seeking algorithm is utilized for adaptive approximation of fault parameter. We prove that the consensus can be still reached by regulating the interconnection weights and changing the connection topology of the fault agent. A numerical simulation example is given to illustrate the feasibility and effectiveness of the proposed method.


Introduction
Recent years have seen a growing interest in the cooperative control of multiagent systems [1,2].Cooperative multiagent system refers to the concept that multiple agents work together to complete a task or achieve a target state according to the cooperative control law [3].With the rapid development of embedded systems, complex algorithms can be effectively implemented in multiagent systems.
In multiagent systems, a fault occurring in any agent may have an impact on other agents, which is different from the traditional faults occurring in isolated systems [4].Moreover, when faults occur to agents, the topology of the multiagent system may change [5].Therefore, fault detection should perform fast detecting to avoid affecting other agents, and fault-tolerant control should make the system has endurance to the failures while keeping the topology structure.
Fault detection of multiagent systems has to be completed before fault tolerance.In the last decade, scholars proposed different methods for fault detection, such as observer-based methods [6,7], parity equation [8], and the identificationbased method [9].Shames et al. derived sufficient conditions for the existence of unknown input observers for secondorder linear time invariant systems [6], which constituted the basis of the current study.Then, they extended these conditions to imprecise models [7].For parity space method, residual errors are obtained by collecting system input and output.Chan et al. developed a parity space-based estimator, which is sensitive to specific faults [8].The literature [9] overviewed the problem of identifying.The identificationbased method means that residuals for output variables are generated with adaptive nonparametric or parametric models.However, all these methods need a great amount of computations and long computing time when the system has large numbers of agents, which are not acceptable for practice.
Once a faulty agent is detected, fault-tolerant control is taken for handling faults.In this paper, fault-tolerant control is divided into two steps, namely, fault parameter approximation and adjusting some interconnection weights.Adaptive fault parameter approximation is developed on the basis of parameter estimation.Generally, fault parameters are estimated using the nonlinear neural network [10,11].We transform the fault parameter approximation problem into the optimization problem by using extremum seeking.Compared with the classical neural network method, the advantages of the proposed approach are that the approximation is real-time and online without any offline training.
Moreover, the extremum seeking based parameter approximation is significantly simplified.The design process of extremum seeking does not call for the understanding of the input and output characteristics of the system [12,13].
In step two, the faulty system is recovered by adjusting some weights of the cooperative protocol.There have been several studies in recovering faulty multiagent systems.Semsar-Kazerooni and Khorasani [14] and Azizi and Khorasani [15] used fault-tolerant control algorithms to recover an actuator fault detected by FDI.Furthermore, Azizi and Khorasani put forward a two-level architecture which contains partial recovery and cooperative recovery [15].Yang et al. proposed a cooperative protocol to adjust fault parameters for a target aggregation problem of nonlinear multiagent systems [5].However, in these studies, the cooperative faulttolerant control was used to adjust interconnection weights without isolating out the faulty agent, which leads to a lot of calculation when faulty agent has a number of neighbors.
The rest of the paper is organized as follows.Section 2 provides some preliminary knowledge and formulates the problems.Section 3 focuses on fault detection.In Section 4, an adaptive fault parameter approximation algorithm using extremum seeking is proposed.In Section 5, the cooperative fault-tolerant control of multiagent systems is discussed.In Section 6, an example of a multiagent team is given to demonstrate the effectiveness of the proposed scheme.In Section 7, conclusion is drawn.

Preliminary Knowledge
Agents and their link topology are mapped based on the graph theory [16].We consider a system constituted by  agents; (V, ) is an undirected graph with vertex set V and edge set , where  ∈ V represents agent .The edge (, ) ∈  denotes a connection between agent  and agent , and   is the weight of the interconnection.The set   = { ∈ V : {, } ∈ } represents all the neighboring agents that are interconnected with .Agent  is supposed to have a doubleintegrator dynamics: where   () ∈  and   () ∈  are the position and velocity of agent  and   is the controlled input based on the following formula: Formula (2) achieves the position and velocity consensus.The term   ( −   ) characterizes the time jump function of an actuator fault,   denotes faulty time of agent , if  ≥   ,   = 1, else   = 0.The variable   ∈  is the fault parameter of agent .The system dynamics in the presence of a fault are written as follows: where ∈  2 .We designed  =  2 (  means an identity matrix with the dimension  × ) to observe all the states of the multiagent system.The following text gave some details of  for the velocity consensus and position consensus problems: where  = diag(, , . . ., ) ∈  × ;  is the Laplacian matrix of the graph, where   = −  , if (, ) ∈ ,  ̸ =  (otherwise,   = 0), and   = − ∑  ̸ =   .By this definition, every row sum of the matrix  is zero, so the Laplacian matrix always has a zero eigenvalue, right eigenvector V = (1, 1, . . ., 1) T , and rank() ≤  − 1.

Fault Detection and Isolation
In this paper, a set of state observers are constructed for the second-order system using the unknown input observer (UIO) method.UIO refers to a robust fault diagnosis scheme for multiagent systems.Once an actuator failure is detected, residual errors are used to locate the faulty agent.In order to reduce the amount of calculation, only neighbors are observed for each agent.We rewrite (3) as where   − is   with the th column deleted,    is the th column of   ,  − () is  with the th component deleted, and   () is the th component of .Suppose graph (V, ) is interconnected and the topology of the system is fixed.
A full-order observer for system (5) is described by Choosing the matrixes , , , and  satisfies the following conditions: Then, there exists a UIO [7] for agent  as follows: where x () ∈  2 and   () ∈  2 are the estimated state and the observer's state for agent  and   ,   ,   , and   are unknown matrices of appropriate dimension, which must be designed such that x will asymptotically converge to .The unknown input observer is constructed to achieve the decoupling from   (), by designing matrixes   ,   ,   , and   .Matrix   is a stability matrix; that is, it has all its eigenvalues in the left-hand side of the complex plane.Thus, we can obtain the observer error and residual dynamics as where   () = () − x () is the observer error and   () is the corresponding residual, which is a fault indicator function that satisfies The detection and isolation condition for fault   () are given as follows: where    and    are isolation thresholds.If the above condition is satisfied, we can conclude that there is a fault affecting the system's th component.
The proposed approach in this section is feasible if a single additive fault exists.In order to isolate multiple faults, one can repeat the abovementioned fault detection procedure for each of the potential fault combinations.We can derive similar observers for all faults and then adopt the detection and isolation condition to isolate each of them.

Extremum Seeking for Approximating Fault Parameters
4.1.Single Faulty Agent Case.The fault detection scheme makes use of observers called unknown input observer, as described in the previous section.Then residuals and their thresholds are designed to generate false alarms, which is used for fast network fault location.Under the assumption of only one faulty agent in the network (suppose the th agent is faulty), the proposed extremum seeking framework is shown in Figure 1.
Theorem 1.Let  :   →  be a sufficiently smooth objective function, and suppose the changing rate of fault parameter estimation f is much faster than the changing rate of fault parameter   : where  has a global minimum ( f *  =   ).Then with the fault parameter estimation scheme shown in Figure 1, f will converge to the extremum point f *  .
Proof.Define the error between estimate state and real state by x() = () − x(): Then we obtain From the definition above we have (  ) = 0,   = 2, and there is only one faulty agent in the network:   = 0 if  ̸ = .Then formula ( 14) can be rewritten as Denote the estimation error of f as f =   −   , so we have f =   − f −  sin () .
The dynamic model of the system can be summarized as follows: where [ sin()] means the Laplace transforms of  sin().Then we have Then we have Then the signal which is processed with high-pass filter /( + ℎ) can be denoted as Mathematical Problems in Engineering So we have From f =   −   , we obtain We arrive at From formula (23), we find that the convergence rate is governed by the excitation signal gain , integration time , and the estimation error f .However, to implement the algorithm, some suitable parameters have to be designed; for example,  is designed large relative to , .
Remark 2. The principle of extremum seeking has been reported in the literature [12].Since Wang and Krstic proved the stability of extremum seeking in 2000 [17], scholars have shown increasing interest in extremum seeking.In addition, sin  (excitation signals) can be replaced by other excitation signals [13].The assumption that the changing rate of f is much faster than   can be fulfilled if the gain  and frequency  of excitation signal are chosen large enough.
Remark 3.There are many other self-adapting algorithms to estimate fault parameters.However, comparing with extremum seeking, most of the algorithms are more difficult to design.

Multiple Faulty Agents
Case.Theorem 1 gives the condition of parameter estimation when there is only a faulty agent in the network.When there are multiple faulty agents in the network, a multivariable extremum seeking parameter estimation scheme is proposed, which is concluded in Theorem 4.
where   ̸ =   and   +   ̸ =   ,  has a global minimum ( f = ), and f will converge to the extremum vector f * .
Proof.From formula (14), The first derivative and second derivative of cost function are The Hessian matrix is   = diag(2, 2, . . ., 2), and formula (25) can be rewritten as Denote the estimation error of f as f =  − , and we have Then we obtain We have From (31), we find that the convergence rate is governed by the integration time  and the estimation error f.

Cooperative Weight Accommodation for Fault Tolerance
In this section, we primarily focus on the design of cooperative fault-tolerant control laws for multiagent systems.The system contains  agents with fixed connection topology.
The connection matrix can be formulated as a Laplacian matrix, and the velocity of the th agent is influenced by connection weight   .When the unknown input observers detect a failure in the network, we can improve the influence of its nonfaulty neighbors on the faulty agent by adjusting the weight.

Single Faulty Agent Case.
Weights   and   are both positive constants.Based on (2), all agents asymptotically converge to the same velocity and reach the same position if there is no faulty agent.Agent  turns faulty at   and it was first isolated from the system.The new networked system without faulty agent can be written as a linear time invariant dynamical system as where 1) , and ,  ∈  2(−1) .All the eigenvalues of the matrix  − less than 0 guarantee the convergence to the target point.We defined the state of each agent as   = [  ,   ],   ∈  2 as the target state.We defined p =   −  0 , q =   −  0 and x = [ p1 , p2 , . . ., p−1 , q1 , q2 , . . ., q−1 ], and we have The networked system without faulty agent can be written as a linear time invariant dynamical system: The following text gives some details of  1 and  2 : where  ∈  −1 ,  = [ 1 ,  2 , . . .,  −1 ], and   = [0, 0, . . ., 0,  0 ,  0 , . . .,  0 ] T .
Lemma 5.The new system (34) achieves consensus asymptotically if and only if  − has exactly two zeros and all other eigenvalues have negative real parts, specifically, (36) Theorem 6.The faulty agent has an equilibrium state q 0 = 0 when where  min ≤  0 ≤  max .
Proof.The new multiagent system is still able to converge to the target state   = [  ,   ] after the faulty agent is isolated.For a faulty agent under   , the Lyapunov function is constructed as and the time derivative of  0 is Then we have where  min ≤   ≤  max .

Simulation
In this study, a cooperative multiagent team example is given to illustrate the feasibility and effectiveness of the proposed method.Seven agents communicate with each other to aggregate the consensus position.First, we build a set of unknown input observers for each agent.Then, we utilize extremum seeking to adaptively approximate the fault parameter.Finally, we use the fault tolerance method to recover the system.As shown in Figure 2, seven agents will converge to the consensus state in -label, where the position in -label is fixed.The start position of the seven agents is 10 * random (1), where the start velocity is zero.The initialization parameters are   = 1,  = 1, and  = 1, so the second maximum eigenvalue of matrix A for the position consensus problem is Re() = −0.5.At  2 = 16 s, agent 2 gets faulty, and the faulty parameter is f 2 = 2,5.Figure 3 shows the -position and velocity of seven agents, and all agents achieve target state consensus by cooperative control law.The final state of position is mean value of the initial positions, and the final state of velocity is zero for the initial velocity.When  >  2 = 16 s, there was a deviation between real state and estimate state.In extremum seeking, we choose  = 10,  = 0.001, and  = 0.1.The seeking process is shown in Figure 4.

Conclusion
This paper proposes a new framework of fault detection and fault-tolerant control of multiagent systems.This paper mainly studies fault parameter approximation using extremum seeking and cooperative fault-tolerant control using interconnection weights adjusting.Fault parameter approximation using extremum seeking simplifies the design process.Then, by adjusting some interconnection weights based on Lyapunov potential-energy function, we show that the target state is still reached.

2 Figure 1 :
Figure 1: Structure of fault parameter estimation with extremum seeking.

Theorem 4 .
Replace the excitation signals  sin  and sin  with excitation signals vector () and () in the structure of fault parameter estimation in Figure1:

Figure 2 :
Figure 2: Initial positions and desired positions of 7 agents.

Figure 3 :
Figure 3: The consensus process of seven agents.

Figure 4 :
Figure 4: The search process of extremum seeking.