A Novel Clustering Method Based on Quasi-Consensus Motions of Dynamical Multiagent Systems

This paper presents a novel approach for clustering, which is based on quasi-consensus of dynamical linear high-order multiagent systems.The graph topology is associated with a selected multiagent system, with each agent corresponding to one vertex. In order to reveal the cluster structure, the agents belonging to a similar cluster are expected to aggregate together. To establish the theoretical foundation, a necessary and sufficient condition is given to check the achievement of group consensus. Two numerical instances are furnished to illustrate the results of our approach.


Introduction
With a rapid increase in the scale of massive data and information, the probing of potential knowledge in big data, such as data structures and certain unknown correlations, has been attracting more and more scholars.As an essential technique of data mining, clustering has been widely applied in various practical fields, such as linguistics [1], bone microarchitecture analysis [2], community detection [3], wind power prediction [4], image segmentation [5,6], and even genome expression study [7].
Clustering is about assigning certain set of data points into different clusters, for the purpose of highlighting the similitude of data points being organized in the same cluster, while simultaneously reflecting the distinctions between different clusters [8].In the past decades, diverse clustering algorithms and criteria have been developed adequately, for example, normalized cut [6] and min-max cut [9] algorithms, the graph partitioning algorithms [10], the algorithms based on optimization of modularity functions [11], and the spectral clustering algorithms [12].
In addition to the abovementioned literature, some clustering algorithms have also been presented which are built on the study of certain dynamical behaviors of networks, for example, the random walk algorithms and the network synchronization algorithms.Random walk implies shifting along a random route, with each step to a new nearest vertex.By random walk algorithms [13], the clustering structure of a given network topology could be probed with the aid of generated random walk flows.Conversely, the network synchronization algorithm stands from another perspective [14], which considers coordinating different signals among vertices.Typically certain Kuramoto vibrator is employed, such that synchronization of vibrations could be achieved within the same group.
In this research article, a novel method for clustering is presented, which is rooted upon observation to the motions of dynamical multiagent systems.Concretely speaking, for the purpose of discerning the different affiliations of vertices in a given network, the selected dynamical multiagent system should be linked with the concerned network topology, and there should be a one-to-one correspondence between each pair of dynamical agent and vertex.The agents keep on moving inside a common space, either physical or abstract.Those corresponding to the vertices that are affiliated with any specific cluster should aggregate as time elapses.Finally, the clustering structure would be uncovered automatically, according to the formation of agent positions.

Complexity
Analogical topics as discussed above are usually referred to as group consensus in the field of control theory.Yu and Wang earlier were concerned with this phenomenon in [15].Hu et al. discussed group consensus for two classes of dynamical multiagent systems, respectively, which are systems with hybrid protocols (there is discontinuous transfer of information between nearest groups [16]) and systems with distinct groups comprising different types of agents [17].To achieve group synchronization of the coupled vibrators, Su et al. considered both the adaptive pinning control strategy [18] and the case with multiple leaders [19].Xie et al. [20] addressed the group consensus problem of first-order systems.Other recent relevant studies include [21][22][23].
The novelty of the current work compared with the aforementioned results in the literature lies in several perspectives.As far as we know, in the existing researches, the expected clustering formation is prescribed in advance, paying primary attention to synthesis of specific information transfer protocol for each group, so that the agents assigned to the group could ultimately aggregate.In contrast, in the framework of the current paper, the cluster distribution should completely be determined by the topology of network itself, without the need of any prerequisite knowledge aforehand.In this regard, the advantage of the current work is explicit, especially in its potential practicability.
Besides introducing a novel method for clustering, a major contribution of this paper is proposing a criterion for checking whether or not a high-order LTI (Linear Time-Invariant) multiagent system can reach group consensus, which generalizes the existing well-known necessary and sufficient condition for consensus achievement [24][25][26][27][28].In fact, the existing condition on consensus is a particular instance of the condition for group consensus presented here.
In addition to the above, the current paper provides an exemplification of the usage of unstable dynamical systems, whereas, in contrast, unstable systems are conventionally regarded as being insignificant in control theory.
Theoretical studies on dynamical multiagent systems have already been extensive in the area of control theory, especially on the consensus problem.However, the application instances corresponding to these theories that can well support them are still scarce.Our exploration attempts to introduce a practical scenario from the field of data analysis, under a motivation to facilitate applying, verifying, or enriching certain relevant researches.
The remaining part of the current paper holds the following organization.The preliminaries and model formulation are introduced in Section 2. In Section 3, the clustering method is expounded in detail, based on theoretical analysis on condition for group consensus.Section 4 exemplifies the technique by two simulation cases.Finally, this paper is concluded in Section 5.

Model Formulation and Preliminaries
The topology of the network for clustering can be expressed by  = (,), with  being the vertex set and  the weighted edge set.The  is assumed to be a graph of th order, being undirected and connected.The ultimate objective of clustering is to assign the  elements of  into divergent affiliations, in accordance with the topology of .
For such a purpose, a procedure rooted in observing the motions of dynamical multiagent systems is put forward, to figure out the clustering formation automatically.The procedure is composed of the following two fundamental steps.
Step 1. Define a dynamical multiagent system as follows and attach with it the network topology concerned, with each agent associated with one particular vertex.
where  푖 () ∈ Θ ( = 1, 2, . . ., ) expresses the state of agent  which is moving in a normed state space Θ,  푖 푗 ∈  + is the weight indicating the strength of communication link between vertices  and , and the functions ℎ(⋅) and (⋅) express the self-dynamics and the dynamical interagent information transfer protocol, respectively.
As time elapses, group consensus or quasi-consensus would manifest, as long as the overall setup of both the structure and parameters of dynamical multiagent system (1) is feasibly configured.Finally, a clustering result can be concluded via observing the setup formed by the agent states within the state space.
The definition below formulates the concept of group consensus discussed in this paper.Definition 1.For the dynamical multiagent system (1), if then agents  and  reach an agreement.For a vertex set  푙 ⊂ , if ∀,  ∈  푙 , the agents  and  reach an agreement, and then system (1) reaches a consensus in  푙 .If consensus is reached in each  1 ,  2 , . . .,  푙 , . . .,  훽 , respectively, with then the entire dynamical system reaches a group consensus.

Remark 2.
The assumption that the graph is both undirected and connected is merely due to practical requirements of clustering and is not due to any technical limitations.This can be understood through later discussions.

Group Consensus of Dynamical Linear Time-Invariant Systems
In this section, the detail of the clustering process is elaborated by employing high-order LTI multiagent systems, which are described as where 푇 ∈  푛 ( = 1, 2, . . ., ) expresses the state vector of agent i and matrices ,  ∈  푛×푛 express the LTI self-dynamics of the agents and the information transfer protocol between nearest agents, respectively.The technique for successfully acquiring a rational clustering result depends on appropriately selecting values of the pair of matrices  and .
To this end, some previous results on consensus are reviewed first.
Lemma 3 implies that if a certain subset of agents reaches a consensus, then the overall motion of this subset will ultimately be dominated by the matrix A. As long as A is Hurwitz, the motions of distinct consentaneous subsets would independently converge to the origin of state space, even without any exchange of information.Consequently, in order to differentiate agents between distinct affiliations, matrix A should not be Hurwitz.Actually, it is preferable for A to be unstable to avoid the possibility of critical stability, such that distinct clusters would mutually diverge.Theorem 7. Consider the dynamical system (4).Suppose that the spectrum of Laplacian matrix of the directed graph with spanning tree is with the series of matrices being not Hurwitz and being Hurwitz.The pair of agents  and  + 1 (or  and 1, if  = ) reaches agreement iff the th row  푇 푖 of the product Ψ =  possesses the configuration: ⋅ ⋅ ⋅ 0 where  ∈  푚×푚 represents any feasible solution of the matrix equation ∈  푚×푚 represents the nonsingular matrix that transforms the Laplacian matrix into the similar Jordan canonical form and " * " denotes any arbitrary value.
Proof.If the stack vector of the states of agents is defined as then the system dynamics can be described by Let x = ( −1 ⊗  푛 ); then (12) can be transformed into which is equivalent to where "Ω" denotes an indefinite matrix that may be either  or zero.Define the auxiliary vectors and the stacked form as It is evident that  = ( ⊗  푛 ) and Substituting (9) into the above equation yields It follows that According to (14), Due to the structure of  and the fact that  1 = 0, the limits of the first n entries of vector ( ⊗  푛 )x are zero and the limits of the entries indexed by  + 1 to ( − 1) are indefinite, while the limits of the remaining entries are all zero.In other words, lim ⋅ ⋅ ⋅ 0 Equation ( 19) can be rewritten as According to the structure of Φ, the fact that agents  and  + 1 ( = 1, 2, . . .,  − 1) reach an agreement implies that the entries in the left side of ( 22) indexed from ( − 1) + 1 to  tend to approach zero.That is to say, lim ⋅ ⋅ ⋅ 0 ⋅ ⋅ ⋅ * (푚) ], then ( 23) is true because of the structure of (21).Conversely, if lim 푡→∞ ( 푇 푖 ⊗  푛 )( ⊗  푛 )x = 0, by noticing the fact that agreements are independent of the initial states and concerning the structure of ( 21), one can conclude that ⋅ ⋅ ⋅ 0 Remark 8. Theorem 7 can verify the agreement between any two agents, since the index assignment is trivial for a network.With this theorem, an overall portrait of the group consensus can be easily derived.
Remark 9.For the sake of the clustering application, only the undirected graph should be concerned.However, Theorem 7 is more generalized since it addresses the systems with graphs that could have directed arcs.
Remark 10.Matrix  can be figured out by solving the matrix equation ( 9) as with  † expressing a Moore-Penrose inverse [30] of the singular Laplacian matrix .Besides, matrix  is constituted by the generalized eigenvectors of .Therefore, it is simple to compute the matrix product Ψ = .
Remark 11.Both the matrices  and  are derived from the Laplacian matrix.Thus, if only the value of  is definite, the clustering result merely depends on the topology of graph.
Remark 12. Theorem 7 is a bridge connecting the concept of consensus with the diverse nonconsensus cases.The smaller the value of , the higher the degree of freedom that matrix Ψ would possess, for the sake of ensuring agreement.Here are two extremes: if  = 2, then the overall system reaches a consensus, whatever the value of Ψ is; oppositely, if all the matrices  −  푖  ( ∈ {1, 2, . . ., }) are not Hurwitz, then there should be no agreement at all.Actually, the existing well-known criterion for checking the consensus [24][25][26][27][28] (summarized as Lemma 6 here) can be regarded as a particular case or corollary of Theorem 7 when  = 2.

Simulations on Clustering
This section will exemplify the presented technique of clustering by two simple and typical simulation instances.
Example 1.Consider a graph in Figure 1.
The weighted adjacency matrix is with the associated Laplacian matrix and spectrum {0, 0.2, 1.095, 3, 3.105, 3.2}.Set the corresponding multiagent system as In this instance,  − Consequently, Based on Theorem 7, it is explicitly known that agents 1∼ 3 reach a consensus; meanwhile agents 4∼6 reach another consensus.As a result, a clustering formation is clearly exhibited.The motions of agents in the 2-dimensional state space are illustrated in Figure 2.
Example 2. In the second example, consider the graph illustrated in Figure 3.The adjacency matrix is and the Laplacian matrix is In this case,  − As a result According to Theorem 7, only two pairs of agents can achieve agreement, respectively, which are agents 1 and 2 and agents 5 and 6.The state trajectories obtained from the experiment are shown in Figure 4.
Although not all agents could achieve a precise group consensus in this case, from Figure 4, the six agents can still be differentiated as two distinct clusters, practically, because the speed of divergence between the two clusters is much faster than the speed of divergence of the agents inside any cluster, in a certain time span.The variation of difference between the states of agents 2 and 3 is illustrated in Figure 5, which is  22 () −  32 ().From Figure 5, one can clearly sense that the vicissitude of the difference between agent 3 and the other agents in the same cluster is sophisticated, including two phases: before  = 3, there is a tendency of agreement; however, afterwards, the tendency turns into a departure.This can be intuitively explained via a comparison with Example 1.
Let us inspect the dynamical equation of agent 3; that is, The vector field is composed of three components as shown above: the autonomous force, the interactive force from the nearest neighbors of the same cluster, and the interactive force from the other cluster.During the first phase of Example 2, the effect of the third component is relatively weak, because of the small value of  43 ; thus the states of agents belonging to the same cluster, that is, 1, 2, and 3, tend to achieve a consensus; however, as time elapses and the distance between the two clusters increases, the effect of the third component also keeps on increasing and ultimately it overwhelms the diminished second component and tears agent 3 away from its nearest neighbors 1 and 2. By contrast, in Example 1, a precise group consensus can be reached because the interactive force from the third component is balanced upon every agent.

Conclusion
A novel method for clustering is introduced in the current paper, based on group consensus or quasi-consensus of the motions of dynamical systems.For the sake of categorizing the vertex set into disparate clusters, a specific dynamical multiagent system is attached to the graph addressed, with each vertex associated with an agent.If the dynamical system is appropriately configured, then the agents affiliated with the similar cluster should gradually assemble together in their common state space; meanwhile the different clusters diverge away mutually.In addition to the method of clustering, a primary contribution is the presentation of a criterion for checking whether or not a group consensus could be reached by high-order linear multiagent systems, which is a necessary and sufficient condition generalizing the previous well-known condition of consensus.In fact, the scope of possible utility of the condition for group consensus is broader, beyond the clustering scenario being dealt with here.Two typical simulation instances are exhibited to exemplify the process of clustering method.As to the potential future extensions, there are several explicit major directions: (1) the method can be utilized to handle various real-world applications such as image segmentation, electrocardiosignal analysis, and classification of natural languages; (2) a further exploration toward the mechanism of group consensus or quasi-consensus could be conducted, and a more straightforward correlation between group consensus and the topological characteristics of networks might be obtained; (3) the patterns of diverse nonconsensus motions can be studied in depth.

Figure 1 :
Figure 1: Instance with group consensus.Default edge weight is 1.