Time-Varying Consensus Control Method of Hybrid Hierarchical Distributed Formation

Aiming at the problem of formation control, a hybrid hierarchical distributed formation structure based on the virtual leader, centralized, and distributed structure is proposed and named active agent AA. Based on the established formation structure, a time-varying formation consensus control model and the changed model with no collision condition, which combined the formation ﬂ ight problem with the collision avoidance problem are constructed, simpli ﬁ ed the structure of the controller, then theoretically analyzed the consensus conditions of non-AA agents, and AA agent, respectively, obtained the necessary conditions of consistent convergence, and ﬁ nally, simulated a formation with one virtual leader and four followers; the simulation results show that the control law designed in this study achieved the velocity consensus of non-AA agents and the location consensus of the AA agent; it also shows that the hybrid hierarchical distributed formation structure proposed in this study is reasonable, e ﬀ ective, and achievable.


Introduction
Swarm coordinated operation can effectively increase the intelligence degree of combat, give full play to the advantages of the system, and realize the doubling and crushing of noncluster firepower, to improve the combat effect of saturation attack outside the defense area, which is a subversion of traditional combat. In recent years, several local conflicts have highlighted the advantages of swarm warfare and become a new mode of warfare. Among the swarm key technologies, formation control is the core to accomplish the coordinated tasks, including the achievement of a predefined formation, formation tracking control, and formation obstacle avoidance. To generate an appropriate formation according to the task is the premise for the swarm coordinated. Formation tracking and control is the basis of formation of movement towards the target, and formation obstacle avoidance is the key to effectively implement combat tasks in complex environment. With the research deepening of formation control, many formation control methods have emerged [1], such as behavior based on method, leader-following method, virtual leader method, and artificial potential field method.
Behavior-based formation control [2,3] and formation members can be divided into collision avoidance, obstacle avoidance, target information acquisition, and formation maintenance according to the information obtained from sensors and support networks. Its biggest feature is to determine which response behavior each member of the formation should take with the help of the average weight of behavior response control. The problem of this method is that the formation is not rigid enough. The leader-following method is based on the predesigned formation structure [4][5][6], and the follower tracks the leader's velocity and position to achieve the purpose of formation control. This method is the most common and basic method used in formation control at present, and it is easy to combine with other methods, but the main problem of this method is that when the leader is damaged, the whole formation will be paralyzed; virtual structure method [7,8] uses the method of virtual leader to coordinate other members to achieve the purpose of formation control and takes multiple real individuals as the direct followers of the virtual leader in the formation. This method can well avoid the paralysis of the whole formation after the leader is damaged. However, the problem of the virtual leader method is that the position of the virtual leader needs to be synthesized at the cost of high communication quality computing ability. The artificial potential field method is a kind of virtual force method [9][10][11][12][13], which has the advantage that the formation can maintain a stable formation in flight by designing the expected distance between formation members. It has good obstacle avoidance effect and does not need global coordinates, but it has high requirements for information interaction mode, and when the formation changes, it needs to redesign the potential field function parameters, which is not flexible enough.
In recent years, the consensus of linear time invariant cluster systems or swarms of UAVs systems has attracted more and more attention in [14][15][16][17][18][19][20][21][22][23]. Consensus problem refers to the agent continuously modifies its own state according to the information of neighbor nodes through information interaction with adjacent nodes and finally makes all individual states in the system consistent. In reference [15], it is pointed out that the consensus problem can be used to deal with the control of formation, and the leaderfollowing method, behavior-based method, and virtual leader method can be regarded as the special cases of consensus problem. Consensus algorithm can overcome the shortcomings of the above methods. Moreover, through a consensus approach, the damage or destruction of individuals has little effect on overall formation, which makes the consensus algorithm robust, adaptable, and expandable. Therefore, in the background of the rapid development of swarm key technologies, it has great significance to study the consensus control method of formation. Recently, the research direction of consensus formation control has been continuously refined, and fruitful results have been achieved from many aspects, such as theoretical analysis, simulation test, and flight verification. References [16,17] and others have studied and obtained the necessary and sufficient conditions for the UAV group to achieve uniform convergence under the dynamic network topology: if the network topology in any time period has a spanning tree structure, the UAV group can achieve uniform convergence. The writer in [18] divided the consensus control strategy of formation into four categories according to geometric constraints, that is using absolute position definition formation, relative position definition formation, relative distance definition formation, and relative orientation definition formation, and, based on the established nonlinear agent model, analyzed the consensus control under different constraints, such as under the condition of unmeasurable velocity information, unmeasurable state information, constraint of convergence error, and unknown time-varying communication delays. According to different swarms of UAVs models, the consensus problem of swarms of UAVs is divided into first-order swarms of UAVs system, secondorder swarms of UAVs system, and high-order agent system, and the follower control model is described, and the flight control law of formation is designed by using consensus theory in reference [19,20]. Furthermore, Olfati-Saber proposed a special potential energy function, which can avoid individuals collision, and through a certain communication structure design, each individual can obtain the leader's information in [21,22]. Su [23] and others later made further improvements on the basis of Olfati-Saber, making the problem more general. The main idea is taking only a small number of individuals received the leader's information and proved the consensus of the system. By analyzing above, it is evident that a lack of literature contributions that simultaneously allow to take into account the consensus method combines the leaderfollowing method, the virtual leader method (the virtual leader at the front of the formation), and the artificial potential field method that ensure the obstacle avoidance of swarm. Therefore, in order to overcome the encountered limitations of the recalled literature contributions, in this paper, we propose a hybrid hierarchical distributed formation structure and a formation control strategy for swarms of UAV based on the combination of leaderfollowing method, virtual structure method, artificial potential field method, and consensus control. Compared with the existing formation control, this work has the following three innovation points: first, proposed a hybrid hierarchical distributed formation structure based on the virtual leader, centralized, and distributed structure characteristics and named an active agent AA. This formation structure highlights the advantages of virtual leader, centralized and distributed structure, respectively, and made the control hierarchical, easy to control, and has good expansibility, strong flexibility, and small amount of calculation. Secondly, based on the formation structure, a consensus time-varying control strategy combining artificial potential field and virtual leader is proposed, which combined the artificial potential field method, virtual leader method, and consensus formation control method. This control algorithm can not only avoid the shortcomings of each respective method but also realize the formation collision avoidance, combine the formation problem with the collision avoidance problem, and simplify the controller structure; thirdly, derived the establishment conditions of changed model and theoretically analyzed the consensus of non-AA agents and AA agent, respectively, obtained the necessary and sufficient conditions of consistent convergence, and realized the formation outer loop control of the velocity and position.
With respect to the current state of the art in the field of control techniques for swarms of UAVs, the time-varying consensus control of hybrid hierarchical distributed characterized by the following features.
(i) The location of the virtual leader does not require complex synthetic calculations (ii) Definition an AA agent as the subgroup leader which can obtain both information from virtual leader and adjacent agents (iii) The proposed control algorithm not only can control in hierarchical but also can avoid collision between formations The paper is organized as follows: Section 2 introduces the hybrid hierarchical distributed formation structure. Section 3 presents the basic knowledge of graph theory and designs a time varying consensus controller. Section 4 2 International Journal of Aerospace Engineering provides the establishment conditions of control law and the analysis of consensus of non-AA agent and AA agent. Section 5 presents a simulation with one virtual leader and four followers' formation. Finally, Section 6 presents the conclusions and future work. Note: the agent mentioned below refers to small UVA in a swarm.

The Design of Hybrid Hierarchical Distributed Formation Structure
In formation control, the most important thing is to design a desired formation structure by using geometric constraints [24]. The UAV used in this study, due to its small volume and light weight, is very vulnerable to the influence of air flow, resulting in lateral deviation of flight direction. In addition, during the flight, it is necessary to do maneuver flight to avoid obstacles and sudden changes in the threat environment, but it is easy for members of the formation to collide with each other and cause losses of their own. Therefore, it is very important to design the formation structure reasonably.
There are three factors that should be considered in the design of formation structure [25,26]. First, the influence of vortex. In flight, the wing and tail are subject to air resistance to produce vortex. Although vortex can be conducive to the flight and save energy, it also affects its own stability and the stability of other formation members. Secondly, when the target changes suddenly, the formation can quickly maneuver and adjust. When the enemy disturbs the formation with various weapons, it can quickly avoid to ensure the safety of the formation. Thirdly, when the formation executes different tasks, the distance between formation members and formation structure should be changed according to needs. At present, the formation structure mainly includes horizontal formation, vertical formation, wedge, and diamond [27][28][29]. Considering the small volume of agent used in this study, the aerodynamic impact between agents is large during low altitude flight, and the aerodynamic interference will directly affect and change the forces on aircraft at different positions of the formation. If the formation is appropriate, the aircraft resistance can be reduced, and the resistance can transform the resistance into the power conducive to formation flight. Inspired by bird flight and the kinetic advantage of diamond structures, four types of formations and communication structures were designed as shown in Figures 1 and 2. Through analyzing the structural characteristics of the above four formations, this study proposes to use the hybrid hierarchical distributed formation structure shown in Figure 2. The hybrid hierarchical distributed formation structure is constructed on the basis of making full use of the advantages of virtual leader and centralized and distributed alliance structure. The virtual leader and each alliance adopt a centralized structure, while the agents in the alliance adopt a distributed structure. The formation named an active agent (AA) refers to agents capable of obtaining both information from virtual leader and adjacent agents, as shown in Figure 2 (1,5,9). Hierarchical refers to virtual leader; AA agent and non-AA agent are divided into three layers; distributed refers to distributed structure among all agents in the alliance.
The outstanding characteristics of this structure are manifested in the following three aspects: firstly, in this structure, the centralized control structure is adopted between the virtual leader and AA agents, which is characterized that the information of the agent itself interacts with all other agents in the formation, that is, each UAV in the formation needs to know the information of the other agents in the formation. However, in order to reduce the calculation amount in the formation structure, the virtual leader only connects with the AA agent, making the calculation amount greatly reduced and has a good control effect. Secondly, the position of virtual leader is a geometric center formed by multiple agents in the general formation, which also requires a large amount of calculation, especially when the formation increased, the computation amount will multiply.
In this formation, the position of the virtual leader is set at a fixed distance in front of the formation. This setting method can not only avoid the disadvantage that the formation cannot be maintained when the leader failed or damaged in the leader follower method but also reduce the computational amount formed by the central position of the virtual geometry. Thirdly, the alliance adopts distributed structure, distributed structure is each intelligence only needs to interact itself information with their adjacent intelligence, this control mode is relatively poor relative to the centralized control effect, but the interaction between intelligence is less, no central node, and the system implementation is relatively  International Journal of Aerospace Engineering simple and strong scalability. In addition, although only the AA agents can receive the information from the virtual leader, but as long as the initial information is established, the status of the agents in the alliance is equal; when the AA intelligence is damaged, other alliance members act as AA agents that keep the formation continue to fly. Therefore, this structure gives full play to the advantages of the virtual leader structure, centralized, and distributed, respectively, not only reduced the computing amount but also enhances the scalability and robustness of the system.

Graph Theory and Design of Time Varying
Consensus Controller 3.1. Basic Knowledge of Graph Theory. This part mainly defines the basic knowledge required by graph theory, such as directed graph, node, neighbor, interdistance, formation safety distance, adjacency matrix, and edge set. Directed graph [30]: consider a graph G = ðε, E, AÞ with a set of N nodes ε = fε 1 ⋯ ε N g; a set of edges E ⊂ ε × ε that inter connects the nodes in the graph and its associated adjacency matrix A = ½a ij , ε × ε whose entries depend on the communication between agents in the graphs it is explained later.
Neighbor: for any i, j ∈ ε, the ordered pair j, i ∈ E, if and only if j is a neighbor of i, or j ∈ N i where N i is called the neighbor set of i.
Interagent distance d ij ðtÞ: At time t, the distance between UAV agents of formation is defined as d ij ðtÞ = kε j − ε i k, i ≠ j.
Safety distance d si ðtÞ: when d ij ðtÞ < d si ðtÞ between agents ε i and other members ε j of the formation or obstacles, the agent must take corresponding collision avoidance mea-sures; when d ij ðtÞ = d si ðtÞ, ε i agents were ready to take corresponding obstacle avoidance measures.
Adjacency matrix: a matrix representing the adjacency between vertices. One-dimensional array is used to store the data of all vertices in the graph. Two-dimensional array is used to store the data of the relationship(edge) between vertices. This two-dimensional array is called adjacency matrix.
In multiagent system, the active agent AA is represented as WðtÞ, which defined the following matrix: L a ðtÞ = LðtÞ + BðtÞ.  [31]. In this section, only the outer loop is mainly studied and designed. Assuming that the number of intelligence in the group is N, the second-order control model can be expressed as: Model of virtual leader: x i ðtÞv i ðtÞ, respectively, represents the position and speed information of the i agent. x l ðtÞv l ðtÞ represents the position and speed information of the virtual leader, and the u l ðtÞ u i ðtÞ represents the control input of the i agent and the virtual leader.

Controller Design.
During flight, due to the characteristics of agents and the limitation of perception range, the adjacent relationship in the group may be time-varying. Therefore, the following time-related sets are defined:   International Journal of Aerospace Engineering Definition 1. When i ∈ ε, t ∈ ½t 0 ,+∞Þ agent sets are defined as: where 0 < r a < r nb is crucial for the choice of dwell time to be introduced and r a is the critical collision radius. r s is the maximum distance that agents can access each other. r nb is some positive number smaller than r s . In time series, each time interval is recorded as T = ft 0 , For all agents in the cluster, within the time interval t ∈ ½ t k+1 − t k Þ, the control law is defined as φ a ðd ij Þ represents the gravitational and repulsive potential energy function between i, j agents, which is used to generate gravitational and repulsive forces between two adjacent agents. φ l ðd il Þ represents the gravitational and repulsive potential energy function between the virtual leader l and the i agent. Its function is to attract the active agent and make it converge to the virtual leader in position and speed. f a ðd ij Þn ji , f l ðd il Þn li represents the control force exerted by the agent j and the virtual leader l on the agent i. Inspired by references [32][33][34], and defined as follows: x where r 0 < r 1 < r 2 , and when x = r 1 , there is a unique minimum value. Collision function: Theorem 2. Suppose kx i ðt 0 Þ − x j ðt 0 Þk σ > d si , ∀i, j ∈ ετ d < min fr s − r nb , r nb − r a g ffiffiffiffi ffi σ/ p ð2 ffiffiffiffiffiffiffiffiffiffiffiffi 2Jðt 0 Þ p , ∀t ∈ ½t k , t k+1 Þ,when ∀i ∈ ε,∀t ∈ ½t k , t k+1 Þ, that is, under the condition that there is no collision between agents, the control law proposed in formula (7) can be rewritten as: The later work of this paper is to theoretically deduce the conclusion of formula (13) above and prove that the formation consensus control algorithm based on virtual structure and artificial potential field method is established and convergent under given conditions.

The Establishment Conditions of Control
Law and the Analysis of Convergence 4.1. The Establishment Conditions of Control Law. On the basis of above definition, the energy between agents, the energy of virtual leader and active agents, is expressed as: In addition, the following functions are defined: Proof. Since the velocity v is continuous, there exists δ > 0 such that (7) holds for t ∈ ½t 0 , t 0 + δÞ. By the fact that f a ðd ij Þ = 0 for d ij ≥ r a , the control law (7) can be put into (13). Now, suppose t * ∈ ½t 0 , t 0 + t 1 Þ, t 1 > δ. By contradiction, suppose this is not true. Then, there exist some agent j ∈ N i ðt 0 Þ and sometime instant t * ∈ ½t 0 , t 0 + t 1 Þ, and assume d ij ðt * Þ = r s , S i ðtÞ = fj ∈ ε : kx i ðtÞ − x j ðtÞk < r s g, and N i ðtÞ = fj ∈ ε : kx i ðtÞ − x j ðtÞk < r nb g .

The
Analysis of Consensus of Non-AA Agent. Assumption: we make a connectivity in the group; any non-AA agent has a direct or indirect link with some AA at all times. And suppose that for all t ≥ t 0 , there is a path connecting any agent in ε/W to some agent in W in the group induced graph GðtÞ. Under this assumption, the symmetric matrix L a ðtÞ is defined as part two and is positive for any t ≥ t 0 ; its eigenvalues are expressed as λ m ,λ m ≔ min t≥t0 fλ min ðL a ðtÞg, and λ min ðL a ðtÞÞ is the minimum eigenvalue of the matrix L a ðtÞ. Theorem 3. If the above assumptions hold, when kx i ðt 0 Þ − x j ðt 0 Þk σ > d si , ∀i, j ∈ ε, there are no collisions between the agents. According to the control law, lim t⟶+∞ kv i ðtÞ − v l ðtÞk σ = 0, ∀i ∈ V, and for all ∀i ∈ ε, ∇ xi ðV a + V l Þ, namely, the virtual force applied on agent i, converges to zero. When According to formula (17), V l ðtÞ ≤ JðtÞ, and in instant ½t 0 , t 1 Þ, J is diminishing function, V l ðtÞ ≤ JðtÞ ≤ Jðt 0 Þ.
Because lim t⟶+∞ HðtÞ = 0, That is, when This proofed that the virtual force applied on agent i converges to zero, and the speed of the agent i converges to the virtual leader. It can be seen from Theorem 3 that under the formation without collision, when the non-AA agent can connect with at least one AA agent at any time, the control law can realize the formation speed to finally converge to the virtual leader speed.

The Analysis of Consensus of AA Agent
Theorem 4. If the above Theorem 3 holds, let us set the AA agent set as W = fBg. According to the control law, when t ⟶ +∞, lim t⟶+∞ kx B ðtÞ − x l ðtÞk σ = 0. That is, when there is only one AA in the group, the location of agents eventually tends to virtual leader.
Proof. If lim t⟶+∞ kx B ðtÞ − x l ðtÞk σ ≠ 0 exist, κ 1 > 0. In an infinite time, kx B ðtÞ − x l ðtÞk σ > κ 1 . According to formula (9), suppose there is κ 2 > 0, κ 3 > 0, make jdφ l ðd B Þ/dd Al j d Bl =d Bl ðtÞ > κ 1 , and kn lB ðtÞk = kxBðtÞ − xlðtÞk/ It has been previously shown in formula that lim t⟶+∞ ð−∇ x V a − ∇ x V l Þ = 0, because only AA agents are considered in this group, so lim t⟶+∞ ð−∇ x V a Þ = 0, that is From Theorem 4, we can know that if the group has only one fixed AA, each AA agents is connected with virtual leaser at any time, the designed control law (7) can drive the group to track and to migrate, and the AA converges asymptotically to the virtual leader.

Simulation Verification and Analysis
5.1. The Topological Structure. In flight, the altitude of the agent is set as a constant, and the proposed control method (7) is used to control the x-axis direction and y-axis direction. In this part, for convenience, four unmanned aerial vehicles are used to form a formation, and the topological structure is shown in Figure 2. The two-way edge between nodes indicates that nodes are neighbors and can conduct two-way information interaction. The one-way arrow from node 0 to node 1 indicates that the virtual leader is a neighbor of follower 1 and provides information to follower 1, and the follower is the active agent AA.
As can be seen in Figure 4, after a period of time, the randomly distributed agents can fly with the predetermined formation, and it can also be seen that the active agent 1 eventually converges to the virtual leader.

The Consensus of Position and Velocity.
To better illustrate the consistency of the formation position and velocity, use variable step length ode45 Runge-Kutta algorithm, simulation time set is 50 s, and step length is set to 0.01. The simulation results are shown in Figures 5 and 6.
In Figures 4-6, the simulation results show that the control law designed in this study achieved the location consistency of the AA agent and velocity consensus of interagent.

International Journal of Aerospace Engineering
This method makes the control algorithm has simple form and has obvious effect. It also shows that the formation structure based on the virtual leader-centralizeddistribution structure which proposed in this study is reasonable and effective; it is a new hybrid hierarchical distributed innovation formation structure that combined the virtual leader structure, centralized structure, and distributed structure.

The Formation Obstacle
Avoidance. It is assumed that when no obstacle is detected during formation flight, the above designed controller can fly together to avoid the collision between the UAV agents; when detected the obstacle during flight, for example, set obstacles during flight, φ a ðxÞ is used as the energy function between the UAV agents i and the obstacle j. The obstacle avoidance simulation effect is shown in Figure 7.
The Figure 7 indicated the UAV agents encounter obstacles in flight. Using such a controller can effectively avoid obstacles and finally fly in consensus.

Conclusion
This work presents a formation structure based on the virtual leader-centralized-distribution structure, propose a time-varying formation control model and the changed model which can fully combine the advantages of virtual leader method, artificial potential field method, and consensus control method, and realize the consistent flight of the formation and the obstacle avoidance. On one hand, this formation structure and control strategy fill in a gap in the current literature, where the control structure is layered, the leader is virtual with not need to geometric its position, and the control method can realize the obstacle avoidance on the basis of consensus control flight. On the other hand the simulation highlights the effectiveness of the proposed control strategy in solving the main problems related to swarm UVAs control, the simulation results show that the control strategy presented in this study can achieve the location consensus of the AA agent and the velocity consensus of inter-agent, and the formation structure based on the virtual leader-centralized-distribution structure which proposed in this study is reasonable and effective.
In future work, we should make every agent in the group be an AA agent. As can be seen from the previous analysis, the AA agent is defined as a subgroup leader, but when the leader damaged, other followers have no access to the virtual leader's information. In this case, how to design the switching rules and the control strategy model is the next work plan.

Data Availability
For some reason, the data obtained in this paper cannot be shared.