Consensus Control of Time-Varying Delayed Multiagent Systems with High-Order Iterative Learning Control

We address the consensus control problem of time-varying delayed multiagent systems with directed communication topology. The model of each agent includes time-varying nonlinear dynamic and external disturbance, where the time-varying nonlinear term satisfies the global Lipschitz condition and the disturbance term satisfies norm-bounded condition. An improved control protocol, that is, a high-order iterative learning control scheme, is applied to cope with consensus tracking problem, where the desired trajectory is generated by a virtual leader agent. Through theoretical analysis, the improved control protocol guarantees that the tracking errors converge asymptotically to a sufficiently small interval under the given convergence conditions. Furthermore, the bounds of initial state difference and disturbances tend to zero; the bound of tracking errors also tends to zero. In the end, some cases are provided to illustrate the effectiveness of the theoretical analysis.


Introduction
In the past few decades, cooperative control of multiagent systems has been attracted outstanding attention.One of the main reasons is its wide application in various fields, such as military, aerospace, and industrial [1].The research directions include formation control, containment control, tracking control, swarm control, and flocking control [2][3][4][5][6].Generally speaking, the fundamental problem for cooperative control is consensus, where the objective is to design an appropriate control law so that the whole states of a group of agents eventually reach an agreement.
In the process of studying the cooperative problem of multiagent systems, diverse control strategies are proposed in many literatures.For example, the output regulation method and distributed containment control law for the containment control of linear multiagent systems were investigated in [7,8], while an advanced clustering with the frequency sweeping method was used for the general linear multiagent systems with communication and input delay in [9].Nevertheless, it should be emphasized that almost all practice systems involve nonlinear dynamics.In [10,11], the distributed adaptive control protocol and pinning control algorithm for the consensus of a class of second-order leaderfollowing nonlinear multiagent systems with directed topology were considered.In [12], a distributed cooperative method was applied to deal with the dynamic task planning of multiple satellites.Additionally, the neural network algorithm was also introduced to study the consensus problem of nonlinear multiagent systems [13,14].
In practice, time delays always exist due to the impact of physical factors of the sensor.Therefore, it is of great significance to study the time delay system.In [15], a distributed consensus control protocol with decaying gains for linear discrete-time multiagent systems with delays and noises was considered; and the cases of communication delay and input delay were studied in [16,17].For the problem of time-varying delays, Xiao and Wang [18] investigated the state control problem of multiagent systems with bounded time-varying communication delays and Chen et al. [19] presented a robust control law for a class of time-varying delayed multiagent systems with noisy environment.
As we know, iterative learning control is based upon the idea that the performance of a system that performs the same task repeatedly can be improved by learning from previous iterations [20].In early works, iterative learning control has been considered in various issues.In [21], an iterative learning control scheme was proposed to make the tracking error trajectory converge to the prespecified error trajectory.In [22], an adaptive iterative learning updating law was applied to solve the tracking problem of high precision motion systems.In [23], the packet dropout problem of nonlinear systems with iterative learning control was developed; and Wu et al. [24] solved the high precision satellite attitude tracking control by using iterative learning control.Additionally, a highorder iterative learning identification scheme was considered to extract the projectile's aerodynamic drag coefficient curve from radar measured velocity data in [25]; and the stability analysis of a high-order iterative learning control problem for discrete-time systems and nonlinear switched systems was also studied in [26,27], respectively.In recent years, the application of iterative learning control has been extended to multiagent systems.In [28], a D-type iterative learning control scheme was presented to deal with the tracking problem of multiagent systems.In [29], a distributed adaptive fuzzy iterative learning control algorithm for the coordination control problem of leader-following multiagent systems with unknown dynamics and nonrepeatable input disturbances was designed; and Meng and Jia [30] developed a finite-time consensus control protocol with iterative learning control for a class of multiagent systems.In addition, the iterative learning control was also considered to achieve formation control of multiagent systems [31,32].
By analyzing the above literature, it is not difficult to find that there exist few results about the consensus problem of time-varying delayed multiagent systems with iterative learning control.Moreover, the application of high-order iterative learning control for multiagent systems is also seldom described in the existing papers.These facts inspired our current study.In this work, we divert our attention to the consensus control of time-varying delayed multiagent systems with high-order iterative learning control.The main contributions of this work are summarized as follows: (i) a class of time-varying delayed multiagent systems with directed communication topology is considered in this paper.The dynamic of each agent contains time-varying nonlinear dynamic and external disturbance, where the time-varying nonlinear term satisfies the global Lipschitz condition and the disturbance term satisfies norm-bounded condition; (ii) different from [26,27], an improved high-order iterative learning control scheme is applied to guarantee the tracking error convergence in the iteration domain.Furthermore, we show that the bound of tracking errors also tends to zero when the bounds of initial state difference and disturbances tend to zero; and (iii) it is proven that the improved control protocol can effectively handle the consensus problem of multiagent systems with time-varying delays and external disturbances.
The rest of this paper is organized as follows.Some necessary preliminaries are introduced in Section 2. The problem formulation and main results about time-varying delayed multiagent systems with high-order iterative learning control are discussed in Sections 3 and 4, respectively.In Section 5, the effectiveness of the proposed control protocol is illustrated by simulations, and briefly, conclusions are outlined in Section 6.

Preliminaries
In this section, graph theory, definition, and lemma which will be utilized in this paper are introduced briefly.
2.1.Graph Theory.For a multiagent systems with n agents, the exchange information among agents can be modeled as an interaction graph with n nodes.Let G = V , E, A denote a directed graph with a node set V = v 1 , … , v n and a directed edge set E = i, j , i, j ∈ V , i ≠ j .And the adjacency matrix of the directed graph is defined by A = a ij ∈ ℝ n×n , where a ij > 0 if and only if i, j ∈ E and a ij = 0 otherwise.Moreover, it is assumed that a ii = 0.For agent i, the set of neighbors is denoted by In this work, an augmented graph is described as G which consists of n agents and one virtual leader agent.The communication between agents and the virtual agent is defined by the matrix B = diag b 1 , … , b n .If agent i can obtain the information of the virtual agent, then b i > 0 and b i = 0 otherwise.

Definition and Lemma
Definition 1 (see [33]).For a function y t : 0, T → ℝ n , the λ norm is defined by The following property for λ norm is held.

Construction of the Control Protocol for the Consensus Problem
In this section, the main works are carried out to formulate the consensus problem and construct the consensus control protocol.In addition, some assumptions are also provided.
Considering the multiagent systems which consist of n identical agents with time-varying delays and nonlinear dynamics, the dynamics of the ith agent at kth iteration are described by where z k i t ∈ ℝ 2 , u k i t ∈ ℝ, and y k i t ∈ ℝ 2 are the state, control input, and output of the system, respectively; differentiable in z and t, with g z ⋅ , ⋅ and g t ⋅ , ⋅ .In addition, it is assumed that z k i t = 0 if t < 0. The matrix form of ( 6) is expressed as The dynamics of virtual leader agent are given as which may show a given bounded desired trajectory y d t ∈ ℝ 2 ; there exist unique bounded input u d t ∈ ℝ and t ∈ 0, T such that when u t = u d t , the virtual agent has a unique bounded state z d t ∈ ℝ 2 and output y d t = g z d t , t .Similar to the expression of ( 7), the matrix form of ( 8) is expressed as where and ⊗ represents the Kronecker product.
Given the desired output trajectory of the virtual agent y d t , the goal is to find a control consensus u k t such that when k → ∞, the output y k t of all agents will track the desired output trajectory y d t as close as possible.
According to ( 7) and ( 9), the consensus tracking error of the agent i at the kth iteration is defined as and have where T , and e k t = y d t − y k t .
Before introducing a high-order iterative learning control scheme, we give the following assumptions.
We now present the following improved high-order iterative learning control scheme for the multiagent systems ( 7) and ( 9): where l = k − m + 1, ε l t = He l t , u 0 t is initial given control input, integer N ≥ 1 is the order of the iterative learning control scheme, and 0 < μ < 1 is a weighting parameter to prevent the large fluctuation of the input at the beginning of the iterative operation.The term μ may be allowed to vary with the iteration and let it fixed for simplicity in this paper.P m t is the time-varying matrix to regulate the rate of convergence, which needs to satisfy certain convergence condition (will be introduced below).Like the description of [25,27], Q m ⋅ , ⋅ , R m ⋅ , ⋅ , and S m ⋅ , ⋅ are learning matrices.In addition, it is assumed that Remark 1.If N = 1 for ( 14), the iterative learning control scheme may be considered as a PID-type control law similar to the convention PID controller; and if N ≥ 2, it can be seen as consisting of dual or multiple PID controllers.As more past learning errors are considered, it implies that the high-order iterative learning control scheme has better control effect than one-order.Generally speaking, the higher the orders of the control scheme, the better the iterative learning control performance is expected.However, the choice of N in practice is normally not more than 3.
Remark 2. Like the choice of parameters of the traditional PID algorithm, the choice of Q m ⋅ , ⋅ , R m ⋅ , ⋅ , and S m ⋅ , ⋅ is usually based on a trial-and-error method.
Considering that the purpose of Q m ⋅ , ⋅ is to increase the convergence speed of the system, R m ⋅ , ⋅ is to eliminate the steady-state error of the system, and S m ⋅ , ⋅ is to overcome the oscillation and improve the stability of the system, their weight can be tuned according to the control requirements.In addition, the control approaches such as a fuzzy adaptive method, a neural network algorithm, and particle swarm optimization algorithm can be also applied to obtain the optimal learning matrices.

Convergence Analysis
In this section, we will analyze the convergence of timevarying delayed multiagent systems ( 7) and ( 9) with a high-order iterative learning control scheme (14).The convergence conditions are shown in the following theorem.
For clarification of the remaining discussion, the variable t will be omitted, and the following notations will be introduced.That is, where α represents a function concerned.
The partial derivatives of α x t , t are expressed by Hence, it is easy to find that δα k ≤ κ α δz k , where κ α is the Lipschitz constant of the function α on the interval 0, T .We now state the main result of this paper.
and then the bounds of tracking errors z d t − z k i t and y d t − y k i t converge asymptotically to a sufficiently small interval as k → ∞ for i ∈ 1, … , n .Furthermore, the bounds of initial state difference and disturbances tend to zero; the bound of tracking errors also tends to zero.

International Journal of Aerospace Engineering
Writing the integral expression for z l and taking norms, we get where Obviously, there η 3 o λ < 1 for a sufficiently large λ and then have Similarly, taking λ norm on both sides of ( 25), we have where Substituting ( 31) into (32) yields where Considering the condition ( 19) again, we have β m < 1 for a sufficiently large λ, then β = ∑ N m=1 β m < 1.Hence, according to the Lemma 1, it is obtained that Furthermore, we also have From ( 31), (36), and (38), one can be observed that the tracking errors δz k , δu k , and δy k are all relevant to the initial state error δz k 0 and disturbance bound b w .Furthermore, if μ, δz k 0 , and b w tend to zero, the bound of tracking errors also tends to zero through the iterative operations of the iterative learning control algorithm.The proof completed.Remark 3.Under the convergence conditions ( 18) and (19), one finds that the convergence of iterative learning control is not influenced by the disturbances, the initial state difference, and even the choice of μ.However, the bound of the final tracking errors is directly influenced by those factors.In addition, the iterative learning control algorithm itself as using alone cannot eliminate disturbances or initial state difference, but it can realize a perfect tracking once those factors will not appear any more in the coming iterations.

Simulation Analysis
In this section, we will provide some cases to illustrate the effectiveness of the high-order iterative learning control scheme (14).Consider a time-varying delayed multiagent systems with five agents labeled as 1, 2, 3, 4, and 5.The virtual leader agent is labeled as 0. The communication graph with five agents and one virtual agent is shown in Figure 1.
The adjacency matrix is  The dynamics of agent i at the kth iteration are described as where The dynamics of a virtual leader agent are given as where z d t = x d t , v d t T , and (41) indicates that the desired output trajectory is obtained by a desired input u d t and state z d t .
According to (40), we have

42
In order to analyze the effectiveness of a high-order iterative learning control scheme (14), the following cases are discussed.It should be noted that the parameters m ∈ 1, 2, 3 are obtained by the trial-and-error method.
Case 1.In this case, N = 1, Velocity tracking trajectories Checking the convergence conditions (18) and (19) in Theorem 1, we get the following.
The simulation results for Case 1, Case 2, and Case 50th iteration are shown in Figures 2, 3, 4, and 5, respectively.
Figures 2, 3, and 4 show the tracking trajectories of time-varying delayed multiagent systems with a highorder iterative learning control scheme under different orders at the 50th iteration, which indicate that the consensus is achieved by using the proposed control protocol.However, compared with Figures 3 and 4, the fluctuation and control effect of Figure 2 are the worst.For Figure 4, it is not difficult to find that the control effect is better than Figures 2 and 3.The results of Figures 2, 3, and 4 imply that the higher the orders and the better the control effect.Figure 5 gives the control input curves of five agents for Case 3 and implies that the input of agents can converge asymptotically to the desired input value.Case 4. Comparison and analysis with other control protocols.

Position tracking trajectories
Velocity tracking trajectories   International Journal of Aerospace Engineering In order to illustrate the control effect, another control law is selected to compare with the proposed control scheme (14) (scheme 1) of this paper.Consider the high-order iterative learning control updating law (7) (scheme 2) in [27], which did not include specific provision for the first iteration term.Let the order N = 2, the parameter setting is same as the   27), (28) of Theorem 9 in [27], then have P 1 t + P 2 t = 0 5I 5 + 0 5I 5 = I 5 , ρ 1 = P 1 t − R 1 y t , t Hg z B max = 0 4257 < 1, ρ 2 = P 2 t − R 2 y t , t Hg z B max = 0 2416 < 1, and ρ 1 + ρ 2 = 0 6673 < 1, which mean the conditions are satisfied.The simulation results are shown in Figures 6 and 7.  Figures 6 and 7 show the comparison results of position and velocity with two control schemes.It is implied that the consensus of multiagent systems with the two control laws can be achieved.However, the control effects with the consensus law of this paper are better than the control law of [27].
In addition, the maximum tracking errors with different time delays for N = 3 are also analyzed.The simulation results are shown in Figures 8 and 9.
Figures 8 and 9 give the maximum tracking errors of position and velocity with different time delays.Although the time delays are different, the consensus of multiagent systems with a high-order iterative learning control scheme is still achieved.That is to say, the convergence of the proposed control protocol is not affected by time delays.

Conclusions
In this paper, we have discussed the consensus control problem of time-varying delayed multiagent systems with directed communication topology.An improved high-order iterative learning control scheme is applied to solve the consensus tracking problem of multiagent systems.The desired tracking trajectory is produced by a virtual leader agent.Under the given convergence conditions, the consensus can be achieved by using the proposed control protocol.Based on simulation results, the control effect can be improved by utilizing a high-order iterative learning control scheme.Also, it is implied that the higher the orders, the better the control effect.In addition, compared with another control law, the better control effect with the proposed control scheme of this paper can be obtained.Furthermore, through analyzing the maximum state tracking errors of a multiagent with different time delays, the delays do not play a significant role in the convergence of high-order iterative learning control.In our future work, the consensus problem of multiagent systems with control input delays will be considered.

3
International Journal of Aerospace Engineering Assumption 2. The functions g z ⋅ , ⋅ , B ⋅ , ⋅ , ⋅ , w k ⋅ , and z d ⋅ are uniformly bounded with bounds b g z , b B , b w , and b z d which are denoted by b and S m ⋅ , ⋅ H are bounded, and their bounds are defined by b Q , b R , and b S , respectively, for instance,

Theorem 1 . 18 and positive numbers β m satisfying 4 International
Let the multiagent systems(7) and the virtual agent dynamic (9) satisfy Assumptions 1 and 2 and suppose that the initial state difference z d 0 − z k 0 is bounded.If there exist 〠 N m=1 P m t = I n , Journal of Aerospace Engineering

Figure 8 :Figure 9 :
Figure 8: Maximum tracking errors of position with different time delays.