LQR Based Optimal Topology of Hybrid-Weighted Multiagent Systems

In this paper, the optimal topology structure is studied for hybrid-weighted leader-follower multiagent systems (MASs). The results are developed by taking advantage of linear quadratic regulator (LQR) theory. We show that the multiagent star composite structure is the optimal topology which can enable the MAS to achieve the bipartite consensus. In particular, we prove that the optimal topology corresponding to the multiagent system with the first-order static leader and the second-order dynamic leader is, respectively, a hybrid-weighted star composite structure and an unevenly hybrid-weighted star composite structure.The results of the paper indicate that, in addition to the necessary information communication between leader and followers, the information exchange among followers increases the control cost of the system.


Introduction
Real-life systems related to bionics, including the animal world, ants and geese formations, can often be studied under the framework of multiagent systems.Many other practical systems have also benefited from the study of multiagent systems, such as UAV formation, motion sensor control, and team handling robots .Consensus is one of the most extensive and in-depth studies in the research field of multiagent systems.Vicsek et al. pioneered the investigation of consensus problem [33].The consensus problem means a group of agents which eventually converge to a common state in the exchange of information and communication among neighbors.Since the communications between agents have both positive and negative weights, C. Altanfini put forward the definition and derived determination conditions for the bipartite consensus of multiagent systems [34].The bipartite consensus means that a set of agents finally converge to the same modulus with opposite state under the exchange of neighbor information.Based on [33], Hu et al. weakened the conditions into spanning trees for achieving the bipartite consensus of first-order multiagent systems [35,36].Along with the in-depth study of consensus problem, more and more scholars have devoted themselves to the investigation of optimization problems, in which the optimal cost control problem got special attention.Cao and Ren proposed an optimal control protocol for MASs by taking advantage of LQR theory without considering any leader.They proved that the optimal structure can be obtained in the corresponding directed graph [37].In [38], the optimal topology structure was proposed based on the LQR theory for the first-and second-order leader-follower multiagent system.In this case, consensus is still achieved.
Armed with the forementioned results, we discuss the optimal control structure for hybrid-weighted multiagent systems.Using the LQR theory, we show that the optimal topological structure associated with the MAS of static leaders is a hybrid-weighted star composite topology; and the corresponding optimal topology structure of dynamic leader MASs is the unevenly hybrid-weighted star composite topology.Each of these two structures is proved to be the optimal solution to the bipartite consensus for the corresponding multiagent system.
The rest of this paper is organized as follows.Section 2 contains the basic knowledge of graph theory and LQR theory.In Sections 3 and 4, the optimal control structure problem is established and solved, for the first-and secondorder leader-follower multiagent system with hybrid weights, respectively.For the general leader-follower multiagent system, the results on optimal topology structure are reported in Section 5.In Section 6, several simulation results are analyzed.Section 7 concludes this paper.

Preliminaries
Graph theory can abstract the information interaction and relative relation between agents, which plays an important role in the research of distributed coordination. = {V, E, A} describes a hybrid-weighted directed graph.The hybrid weight means that the edge weight can take positive or negative values, with positive value indicating a cooperative relationship between two nodes, while negative value indicating a competitive relationship.V is the set of vertices, and E is the edge set.A is an adjacency matrix.The corresponding concrete expression is Edge set E is an abstraction of the information exchange.(, ) ∈ E means that agent  receives information from agent . ∈ N  denotes a set with  agents; and   = 0 means that we do not consider the self-loop.We assume that     ≥ 0.   = 0 indicates that there is no connection between V  and V  .If V  and V  are cooperative, then   > 0; otherwise   < 0. The spanning tree of a connected graph with  nodes is a minimally connected subgraph of the original graph which contains all the  nodes in the original graph and keeps the least connected edge of the graph.
The neighbor set of agent  is denoted by C  , C  = { :   ̸ = 0}.Degree matrix D is a diagonal matrix with diagonal elements   = ∑ ∈C    .Laplacian matrix L = D − A. In a leader-follower multiagent system, leaders cannot receive neighbor information from follower nodes.The followers usually have at least one neighbor node.For a leaderfollower multiagent system, the star topology is a specific interconnection structure, in which all nodes are individually connected to a central hub.With the central node playing the single leader role (the central hub of the star composite structure plays the same role with respect to multiple leaders), the leader does not receive any information from other nodes while any other nodes can use the central node as a neighbor (the leader nodes cannot receive neighbor information but can send neighbor information to other nodes).In this paper, the leader is denoted by  * and the follower's neighbor set is denoted by   ,  ∈ N  .The information exchange associated with the first-order multiagent system involves only position information, which is described by (A).The corresponding Laplacian is  A .The information exchange associated with the second-order multiagent system includes both position and velocity information, which is described by an additional digraph (V) and a Laplacian matrix  V .In association with the second-order multiagent system, the directed graph (A) represents the position information and the directed graph (V) represents the velocity information.These two graphs have the same structure with different edge weights.The structural model of the system is called unevenly weighted interaction graph.
2.1.Infinite-Time Linear Quadratic Regulator Theory.Consider a linear system where Let  ∈ R × ,  ∈ R × , which are symmetric, nonnegative, and positive definite, respectively.The following cost function is defined by The cost function represents the total cost of system (1) from the initial state (0) to the equilibrium state.The task of the optimal control problem is to find the minimum cost function ((.),(0)) and the optimal control  * () = − * (), where  * is called the optimal feedback gain matrix [39].In other words, optimal control can bring the state to equilibrium without putting too much cost on it.The LQR optimal control lemma is given below.

LQR Based Optimal Topology of First-Order Hybrid-Weighted Multiagent Systems
3.1.Single Leader.The first-order leader-follower multiagent system is defined as where   () ∈ R and   () ∈ R represent the th follower's position information and the control input information, respectively.Let   * () represent the leader's position information and denote () = ( 1 (), ⋅ ⋅ ⋅ ,   ())  , () = ( 1 (), ⋅ ⋅ ⋅ ,   ())  .Then achieving consensus is to design neighboring information based feedback   ()( ∈ N  ) so that the final states of both the leader and followers converge to the same value.The corresponding expression is as follows lim t→∞       () −   * ()     = 0,  ∈ N  . ( We have where A  = [  ] ∈ R × .Define the Laplacian matrix  A as where ∈ N  , where   represents the (, )th element in the adjacency matrix A of (A) and   is the th element in .
Combining system (4) with protocol (9) yields Gauge transformation is a coordinate transformation performed in R +1 , which is represented by matrix  = diag{ 1 , ⋅ ⋅ ⋅ ,  +1 }.The set consisting of all gauge transformations in R +1 is represented by D = { = diag{ 1 , ⋅ ⋅ ⋅ ,  +1 },   ∈ ±1,  = 1, ⋅ ⋅ ⋅ ,  + 1}.We assume that (A) is structurally balanced, which means that all nodes can be divided into two parts, V 1 and ) [34].It is worth mentioning that if all the semicycles of (A) are positive, (A) is structurally balanced [40].Let   = 1 and   = −1 correspond to  ∈ V 1 and  ∈ V 2 , respectively.For a structural balance graph (A), the gauge transformation  can transform the adjacency matrix A of cooperative-competition MASs into a nonnegative matrix A.Consequently, the bipartite consensus problem of cooperative-competition systems is transformed into a standard consensus problem of cooperative systems.At the same time, the eigenvalues of  under this transformation coincide with those of the laplacian  of the original system.For the transformation where   = diag{ 1 , ⋅ ⋅ ⋅ ,   }, we see that  −1 = .For system (4) the following gauge transformation is considered and thus, π () =   Ẋ().One has Set the following cost function where   > 0,   > 0, which represent the weight of error and the weight of the control cost of follower , respectively.So, the leader-follower multiagent system optimal control problem is suitable for any initial state (0) ∈ R  .The goal is to find the minimum optimal control  * () of ((.),(0)).With ( 13), the optimal control cost is min Since  * () has been found, by (13), the Laplacian matrix structure can be revealed.Finding the optimal control cost is equivalent to finding the Laplace matrix structure corresponding to the system equation, which is then equivalent to finding the optimal topological structure.
Theorem 2. For (15), the optimal topology with respect to the control input ( 9) is a star structure, in which any followers are connected only to the leader with weight  *  = sgn( +1 )  √  /  .The star structure is the optimal topology solution achieving the bipartite consensus for MASs.
Proof.The control error of follower  is denoted by   () =   () −   * ().The control error vector is denoted by () = ( 1 (), ⋅ ⋅ ⋅ ,   ())  .The error system of ( 13) is Substituting () into ((), (0)), the optimal control problem ( 15) is transformed to the LQR problem min The above formula is derived from (16), where 16) is controllable.Lemma 1 implies that there exists a positive definite matrix  ∈ R × , satisfying ARE The optimal feedback gain matrix    A *    =  −1  stabilizes system (16).Premultiplying  −1 on both sides of ( 18) yields Then For all  ∈ N  , their neighbor nodes constitute only   * .That is, there is no exchange of information among followers.So only followers and leaders have information exchange: ( * ) consisting of  edges satisfies the above condition.Because there is no information exchange among followers, and there is only information exchange between leader and followers; we see that the optimal topology is a star structure.The condition of spanning tree guarantees that the topology structure of the first-order system (10) can achieve the bipartite consensus.Thus star topology is the optimal structure for system to achieve the bipartite consensus. is nonnegative.The diagonal elements of the coordinate transformation matrix   depend on   = ±1. +1 represents sgn( +1 ) ⋅ 1.  * represents that the exchange of information among topologies is a mixture of weights; i.e., there exists cooperation situation represented by positive weights, and there is competition situation represented by the negative weights.The bipartite consensus means that the individuals achieve the same modulus with opposite sign.For a first-order multiagent system, the structural equilibrium is a necessary and sufficient condition for achieving the bipartite consensus in an undirected connected graph or a directed graph with spanning trees [35,36].
As shown in Figure 1, the optimal structure for the leader ( ) communicating with followers ( ) is a star topology, and the weights corresponding to each edge are   = sgn( +1 )  √  /  .

Multileader.
The  leaders are denoted by +1, . . ., +.The Laplacian matrix of (A) is where Then the gauge transformation yields where Theorem 4. With respect to the control input (9), the optimal topology structure of leader-follower MASs ( > 1) is a star composite topology, which is the best form of information exchange between followers and leaders.The weight between leaders and followers is the solution of convex hull: min   {  (0) − √  /   *    (0)} 2 ;  *  1  = √  /, which is a star composite structure and is the optimal structure solution achieving the bipartite consensus of MASs, as shown in Figure 2.
where  and  are both -dimensional positive definite diagonal matrices; Similar to the proof of Theorem 2, the optimal control problem reads It follows from (22) and Lemma 1 that where b *  ∈ R 1× ( ∈ N  ) is the solution of the convex hull. min From ( 27), we see that followers only communicate with each pilot, and accordingly the corresponding optimal topology is a composite structure based on a star topology.Thus the composite topology is the optimal structure achieving bipartite consensus for the first-order leader-follower multiagent system ( > 1).
The picture in Figure 2 depicts a star-shaped composite structure consisting of two pilots ( ) and six followers ( ).There is no communication between these two leaders, and each follower only receives neighbor information from leaders.The weight satisfies   +   = √  /  and min   {  (0) − √  /  (  +   )  (0)} 2 .

LQR Based Optimal Topology of Second-
Order Hybrid-Weighted Multiagent Systems 4.1.Single Leader.The second-order leader-follower multiagent system is where In this section, let us consider the following agreement where   ,   represent the (, ) element of adjacency matrix A and W of the directed graph () and (), respectively, and   ,   are, respectively, the ℎ elements of b, d.With protocol (30), system (28) is written in a matrix form where By (11), the gauge transformation of (28) is From  =  −1 , () =   (), () =   (), one has It follows that For (31), () = − A  () −  W  () + b  * () + dV  * ().Therefore where By (37), the optimal control problem of ( 35) is min where   > 0,   > 0. Therefore, finding the optimal control  * () is equivalent to finding the optimal topology structures which correspond to their velocity and state graphs ((A * ), (A * )), respectively.Note that the structure is included, respectively, in Laplacian matrices    A *   and    W *   .
Theorem 5.For the optimal control problem (38) with the control input (30), the optimal topology is an unevenly weighted star structure, in which each follower  is only connected with the leader.The weight of the state graph is    +1 √  /  , and the weight in velocity graph is    +1 √ 2√  /  +   /  .The star structure is the optimal topology solution achieving the bipartite consensus of MASs.
Proof.The consensus error system of ( 35) is as follows The corresponding error power system is where Therefore, the optimal control problem ( 38) is transformed into the following LQR problem min subject to (41), where Since the above matrix is full row rank, system (38) is controllable.Similar to Theorem 2, the ARE equation corresponding to (42) is This equation has a unique positive definite solution.In addition, the optimal control is  * () = − −1     ( 2 ⊗   )().In (44), the structure forms of matrices , , ,  imply that  is a symmetric matrix.It can be written as where From (44), we have =  12  −1   12 .
(48) Premultiplying  −1 on both sides of (48), we have where In (50 ), which means that the optimal topology is a nonuniformly weighted star topology; i.e., follower  has only information exchange with the single leader; the weights on the state graph and velocity graph are    +1 √  /  ,    +1 √ 2√  /  +   /  , respectively.
As a consequence, the star topology with uneven hybrid weights can make system (28) achieve both the bipartite consensus and the optimal topology.As shown in Figures 3 and 4, the unevenly hybridweighted star structure is the optimal communication topology consisting of leader (yellow ball) and followers (red and blue ball).The star topology has the same position and velocity topology structure with different weights.The corresponding weight in the position and velocity topology is sgn( +1 )  √  /  and sgn( +1 )  √ 2√  /  +   /  , respectively.
where   () ∈ R, V  () ∈ R,   () ∈ R denote the state, velocity, and control of follower , respectively.  () ∈ R, V  () ∈ R represent the leaders' state and velocity information, respectively.In what follows, we denote The  leaders are indicated by  + 1, . . .,  + .The Laplacian matrices of (A) and (W) are where where   ,   denote the (, )ℎ element of A and W, respectively.Then   ,   are the (, )ℎ element of  A  and  W  , respectively.In view of system (51) and protocol (53), we have where Similar to the gauge transformations implemented in Theorem 4, we see that where The optimal control problem is considered subject to (57) min In a second-order leader-follower MAS with  > 1 leaders, we assume that the position state tends to   () and the velocity state tends to   ().Define the cost function where   > 0,   > 0.
Theorem 6.For the optimal control problem (58) with the control input (30), the optimal structure of leader-follower MASs ( > 1) is an unevenly hybrid-weighted star composite topology consisting of a star structure formed by followers and leaders.In addition, the structure is the optimal topology solution achieving the bipartite consensus of MASs, as shown in Figure 5.
Proof.For system (54), the consensus error vector is defined by Similar to Theorem 4, the corresponding error dynamics system is where Therefore, the optimal control problem (53) is converted to the following LQR problem min where  = diag{ 1 , ⋅ ⋅ ⋅ ,   },  = diag{ 1 , ⋅ ⋅ ⋅ ,   }.Similar to Theorem 5, we have We have the following matrices: From b * =    A *    , d * =    W *    , we see that the optimal topology of the leaders-followers MASs ( > 1) is the unevenly hybrid-weighted star topology, in which follower  only exchanges information with the leaders.Then, the unevenly hybrid-weighted star composite structure is the optimal topology for system (51) to achieve the bipartite consensus.
Remark 7.For a leader-follower MAS ( > 1), the leaders' states drive the followers to make their states tend to a fixed value.In the above arguments,   () and   () described the final converge value of followers' position and velocity, respectively [41].
As shown in Figures 5 and 6, for a second-order leadersfollowers multiagent system ( > 1), the optimal communication topology is a hybrid-weighted weight star composite structure.In this structure, there is no information exchange between leaders; and the connection weights between leaders and followers in position graph are different from those in velocity graph.

LQR Based Optimal Topology of General Linear Hybrid-Weighted Leader-Follower Multiagent Systems
The general linear leader-follower multiagent system is defined as follows Assuming that (, ) is controllable, we consider the following cost function where   > 0,   > 0 represent the weight of error and control cost of follower , respectively.Find the optimal control cost under (72) min Theorem 8.For system (65) with the control input (66), the star structure is the optimal topology solution to achieve the bipartite consensus.For this star structure, any follower ( ∈ N  ) is only connected to the leader.
Proof.We denote the control error of follower  with   () =   () −   * ().Let () = ( 1 (), ⋅ ⋅ ⋅ ,   ())  represent the control error vector.Then the error system of (72) is The optimal control problem (74) can be transformed into a LQR problem min where The optimal feedback gain matrix  *   A *   *  =  −1 × ( *    ⊗  *  )   satisfies system (75).Since  is a solution of (77), the cost function is only related to ().Hence, the optimal topology is a star structure which enables the system to achieve the bipartite consensus.Remark 9. Theorem 8 points out that hybrid weighted star (composite) structure is the optimal topology for the firstorder and second-order system to achieve the bipartite consensus.In this situation, followers only exchange necessary information with leaders and the information exchange between followers increases the cost of control.For example, in an aircraft queue, followers are only controlled by the pilot, and no communication is performed with followers.The control cost can be effectively reduced and the target requirement can be achieved.

Simulation Result
In this section, simulations are given to show that, for the first-and second-order multiagent system, the information exchange among followers is not conducive to reducing the cost of control.Moreover, it is verified that star topology is the optimal structure for minimizing the control cost.

First-Order Multiagent System
Simulation.For a firstorder multiagent system which is shown in Figures 8, 9, 10, and 11, respectively, the single leader is represented by the yellow node and the five followers are represented by nodes with other colors.The information communications between the leader and followers are hybrid-weighted.Three different topology structures are considered which are depicted by Figures 8, 9, and 10, respectively.The control cost under these topologies is listed in Table 1 with respect to different initial values.The data analysis in Table 1 shows that the control cost is minimum under star topology, and accordingly star topology is a feasible solution to achieve the optimal control.Take  =  =  5 , and 1 as the initial state of the leader.Let  0 = [−7.54,−7.73, 6.21, 2.39, 4.32]  be the initial state vector of followers.Then the convergence state of the firstorder MAS is shown in Figure 7.The -axis and the -axis in Figure 12 represent the weight variables  and  in Figure 11, respectively.The -axis in Figure 12 represents the cost.The  changes in the value of -axis indicate that the control cost is lowest when the values of  and  are both taken as 1.This is consistent with Theorem 2.

Second-Order Multiagent System
Simulation.For a second-order multiagent system, there are two graphs associated with state and velocity, respectively, in which the leader is represented by a yellow node and the five followers are represented with nodes of other colors.These two graphs have the same topology structure, while the edges are different.For the second-order system, the graphs associated with states are depicted by Figures 8, 9, and 10.The topology structure of velocity graph is the same as state graph, while its link weights are √ 3 times the weights of state graph.Take  =  =  5 , and set 0 as the initial value of state and velocity.Table 2 shows the control cost for three different topology structures when followers take different state and velocity initial values.The data analysis in Table 2   shown in Figures 13 and 14.For Figure 17, -axis represents control cost.The link weights  and  in Figures 15 and 16 correspond to the values of -axis and -axis in Figure 17, respectively.By analyzing the  variable in Figure 17, it can be seen that, at point (1, √ 3), the control cost is lowest, which conforms to Theorem 5.

Conclusion
In this paper, we focused on the LQR based optimal control structure for multiagent systems in which the communication between agents is represented in a hybrid weighted form.We showed that the optimal topological structure is characterized by the hybrid-weighted composite star structure (the star structure is regarded as a simple composite structure).More specifically, the paper proposed a quadratic cost function independent of the topological structure first.Then, gauge transformation was implemented to the  hybrid-weighted multiagent system.The related Riccati equation is employed for the system to obtain the optimal topology.For the first-and second-order leader-follower multiagent system, it was shown that the optimal solution always corresponds to a star composite structure, in which there is no information exchange between followers, and only leaders' communication information is received by followers.Future research will focus on the optimal control of distributed multiagent systems, such as the optimal topology of static and dynamic systems, and the convergence rate.

Remark 3 .
The optimal topology of the leader-follower MAS is a star structure.The weight of information exchange is  *  =   √  /   +1 .The above arguments show that any element of  A *

Figure 7 :
Figure 7: The first-order state response of MASs.

Figure 13 :
Figure 13: The second-order state response of MASs.

Figure 14 :
Figure 14: The second-order velocity response of MASs.

Figure 17
Figure 17 and   refers to the connection between the leader and the th follower.