Distributed Optimization of Multiagent Systems in Directed Networks with Time-Varying Delay

This paper addresses a distributed consensus optimization problem of a first-order multiagent system with time-varying delay. A continuous-time distributed optimization algorithm is proposed. Different from most ways of solving distributed optimization problem, the Lyapunov-Razumikhin theorem is applied to the convergence analysis instead of the Lyapunov-Krasovskii functionals with LMI conditions. A sufficient condition for the control parameters is obtained to make all the agents converge to the optimal solution of the system. Finally, an example is given to validate the effectiveness of our theoretical result.


Introduction
In recent years, the distributed optimization problem of multiagent systems has been investigated by many researchers; researches on distributed optimization and control theorem have been developing rapidly and have been applied to various fields of industry and defense, like smart grid [1,2], sensor networks [3], social networks [4], and so on.The objective of distributed optimization problem is to solve an optimization problem cooperatively in a distributed way, where the objective function is formed by a sum of local objective functions, and each agent can only know one local objective function.The ultimate goal is to make the states of all the agents converge to optimal solution of the optimization problem via local coordination.Compared with the consensus problem of multiagent systems, which makes all agents achieve a common state [5][6][7][8], not only does the optimization problem make all agents achieve the same state, but also at the same time the achieved state minimizes the optimization problem.
The current literatures about distributed optimization problems are more focused on discrete-time algorithms (see [9][10][11][12] and references therein).In both papers [9,11], discretetime subgradient algorithms are proposed for unconstrained, separable, convex optimization problem and each agent communicates with the other agents over a time-varying network topology.A projected consensus subgradient algorithm is proposed for constrained optimization problem in [10], and, in [12], the authors devise two distributed primaldual subgradient algorithms over networks with dynamically changing topologies but satisfying a standard connectivity property.But, recently, some continuous-time methods have also been successfully used to solve distributed optimization problem.Based on the gradient algorithm and integral feedback, auxiliary-variables are introduced in continuoustime dynamical system [13][14][15].From the control system viewpoint, a continuous-time multiagent system dynamic is proposed with undirected communication topology [13]; the algorithm is further investigated over a strongly connected and weight balanced directed graph [16], and even a modified system is proposed in [14] with auxiliary-variables no longer needing to exchange information.In [17], the authors present a second-order multiagent system for distributed optimization network under bound constraints, and, in [18], a distributed protocol design for the high-order agent-network under a connected communication topology is proposed.In order to avoid using auxiliary-variables, a family of Zero-Gradient-Sum algorithms are proposed over fixed communication topology in [19].
On the other hand, it is common that time-delay exists in practical systems because of the finite speeds of information transmission and spreading as well as traffic 2 Journal of Control Science and Engineering congestions.Therefore, time-delay should be taken into account in algorithm design of multiagent systems.For timedelay systems modelled by delayed differential equations, an effective way to deal with their convergence and stability analysis is based on the Lyapunov-Krasovskii functionals or Lyapunov-Razumikhin functions.Most of the existing works concentrate on Lyapunov functions combining with Linear Matrix Inequality (LMI) techniques to deal with the consensus problem of multiagent systems with time-delay [20,21].The methods based on Lyapunov-Krasovskii functionals can be applied to a wide variety of problems and may provide necessary and sufficient conditions of convergence and stability, but it often leads to computational complexity and poor scalability.When the number of the agents is large, it would be difficult to verify the solvability of the LMI conditions.However, based on the Lyapunov-Razumikhin theorem, the authors propose a neighbor-based distributed controller [7,8] enabling the agents to achieve consensus along with interconnection delays, which can avoid verifying the LMI condition and reducing computational burden.In [15], distributed consensus optimization algorithms are proposed for continuous-time multiagent systems with timedelay, and some sufficiency conditions based on LMI are obtained.
Motivated by the above observations, the distributed consensus optimization problem of continuous-time multiagent systems with time-varying delay is considered.The interconnected graph is assumed to be directed, strongly connected, and weighted-balanced.The Lyapunov-Razumikhin function is used in the stability analysis.The convergence of the proposed algorithm is guaranteed with the model parameters satisfying some conditions.Meanwhile, the conditions can also give an estimate of the upper bound of the time-delay, which can avoid verifying and calculating the complicated LMI conditions.From the results, we can also see clearly the relationship among the parameters in the system.
The outline of this paper is organized as follows.Some basic knowledge on the algebraic graph theory and useful lemmas are presented in Section 2. The convergence results of the algorithm are established under the given communication condition on network topology by applying Lyapunov-Razumikhin Theorem in Section 3.An example is provided to illustrate the result in this paper in Section 4. Finally, the concluding remarks are given in Section 5.

Preliminaries and Problem Statement
2.1.Preliminaries.Consider a multiagent system consisting of  agents, if each agent is regarded as a node, the communication topology among these agents can be described by a weighted digraph G = (V, E, A) with the finite set of nodes V = {1, 2, . . ., } and edge set E ⊂ V × V.An edge starts from  and ends on , which means that agent  can send information to agent .The weighted adjacency matrix A = [  ] ∈  × is defined as   > 0 if (, ) ∈ E and   = 0 otherwise.If ∑  =1   = ∑  =1   for all  ∈ V, the digraph G is called weighted-balanced.A path is a sequence of connected edges in a graph.If there is a path between any two nodes of a digraph G, then digraph G is said to be strongly connected, otherwise disconnected.The degree matrix D = diag{ 1 ,  2 , . . .,   } ∈ R × of graph G is a diagonal matrix with the th diagonal element being The next lemmas related to the important properties of Laplace  and provide useful mathematical tools.
Lemma 1 (see [22]).Laplace matrix  has at least one zero eigenvalue with 1  = [1, 1, . . ., 1] ∈ R  as its eigenvector, and all the nonzero eigenvalues of  have positive real parts.Laplacian L has a simple zero eigenvalue if and only if G is strongly connected.

Problem Statement.
We consider a multiagent system consisting of  agents.The dynamics of the th agent,  ∈ V, is described by where   ∈ R  denotes the state of agent  and   ∈ R  is the control input.Consider the multiagent optimization problem, in which the goal is to minimize the sum of local cost functions associated with the individual agent.More specially, it can be expressed as Let x = col( 1 ,  2 , . . .,   ) ∈ R  .Next, we provide an alternative formulation of (2), that is, We can see that the problem (2) on R  is equivalent to the problem (3) on R  .In this paper, our goal is to design a distributed controller for each agent such that the states of all the agents converge to the optimal solution of the optimization problem (2) via local communication.
Before proceeding, we give the following assumption on the local cost function   based on convex analysis [24].Assumption 4. (a) For each  ∈ V,   is differentiable and its gradient is Lipschitz with constant   > 0 in R  : ∀,  ∈ R  . ( Remark 5.Under Assumption 4(b), we can note that  is strictly convex; then the problem (3) has an unique optimal solution.
Assumption 6.The digraph G is weighted-balanced and strongly connected.
From Lemma 1 and Assumption 6, there exists a matrix  ∈ R ×(−1) with such that the matrix    = , where the real parts of all the eigenvalues of  are positive, and  +   is positive definite.When considering the presence of time-varying communication delay among the information transmission, the continuous-time distributed optimization protocol is proposed for agent  ( ∈ V) as follows: (0) = 0, (7) where   () is an auxiliary state of agent  and () is a continuously differentiable function satisfying () ∈ [0, ] with  > 0 for all  > 0 and , ,  are the scalar tuning positive parameters; −∇  (  ()) is the gradient term to guide the agents for optimization; is the consensus term with time-delay to make all the agents converge to the same point; −  () is an integral term to correct the error caused by the consensus term.Let Then the closed-loop system of ( 1) and ( 7) can be expressed as a compact form: Let the right-side of closed-loop system (9) be equal to 0; then we can get the equilibrium point (x * , w * ), that is, According to the properties of Laplacian matrix and from (10), one can obtain Under Assumption 6, we have 1    = 0. Left multiplying the second equation of ( 9) by 1    ⊗  and using initial conditions   (0) = 0, we obtain ∑  =1 ẇ  () = 0; then Left multiplying the second equation of ( 11) by 1   ⊗   again results in Thus, the optimal condition ∇(x * ) = 0 is satisfied, which means x * = 1  ⊗  * ,  * ∈ R  is the optimal solution of the optimization problem (3).Using the transformation one can shift the equilibrium point into the origin; then the system (9) can be transformed into the following form: where Ψ(x()) = ∇(x()) − ∇(x * ).

Main Results
Before analyzing the consensus and optimization problem (9), we introduce the stability of time-delay systems.Consider the following time-delay system: where   () = ( + ), ∀ ∈ [−,  0 ] and (, 0) = 0.In the sequel, suppose that  0 = 0. Let ([−, 0], R  ) be a Banach space of continuous function defined on an interval [−, 0], taking values in R  with topology of uniform convergence, and with a norm The definition of the stability of the solution  = 0 is given as follows in terms of the solution of the delayed equation ( 16).
Then the main results can be obtained as follows.
According to the Lyapunov-Razumikhin Theorem, take () =  for some constant  > 1.In case that Then from (35) and above inequalities, we have where and we take  has the upper bound in (22); then V(()) is negative definite.Thus by the Lyapunov-Razumikhin theorem, we can conclude that () → 0; that is, () → 0  , () → 0  as  → ∞.
Remark 9.The continuous-time protocol considered in this paper is based on the algorithm proposed in [15], and under the same communication topology, but the conditions of convergence analysis needed by this paper are more relaxed.From ( 20) and ( 21), it is clearly shown that  * is independent of parameters  and  but dependent on  and communication topology, while  * is independent of constant   in this paper compared to [15].We can know when the number of the agents is large, it would be difficult to verify the LMI condition, but, in this paper, it only needs the model parameters to meet some boundary conditions, and when considering the dynamic system with time-varying delay, the Lyapunov function with Razumikhin technique is also an effective method compared to Lyapunov-Krasovskii method.
We can see that the trajectories   of each agent  converge to the global optimal solution  * = 0.6565 of the objective function () = ∑  =1   () and all the trajectories   converge to a constant, respectively, for  = 1, 2, . . ., 5. The optimal value of () is 45.2602.

Conclusion
In this paper, the consensus optimization problem of multiagents with communication delays was considered.By a continuous-time algorithm, consensus and optimization under some parameter bound conditions are ensured.Graph theory is used to describe the interconnection topologies.Lyapunov-Razumikhin theory were employed for stability analysis.The connectivity assumption of directed graph plays a key role in the analysis of algorithm convergence.Numerical examples were given to illustrate the theoretical results.