Complexity
Volume 2017 (2017), Article ID 4978613, 8 pages
https://doi.org/10.1155/2017/4978613

Research Article

A Novel Clustering Method Based on Quasi-Consensus Motions of Dynamical Multiagent Systems

Ning Cai,1,2 Chen Diao,1,2 and M. Junaid Khan³

¹College of Electrical Engineering, Northwest Minzu University, Lanzhou, China
²Key Laboratory of National Language Intelligent Processing, Gansu Province, China
³National University of Sciences and Technology, Islamabad, Pakistan

Correspondence should be addressed to Ning Cai; caining91@tsinghua.org.cn

Received 18 February 2017; Accepted 26 March 2017; Published 13 September 2017

Academic Editor: Zeraoulia Elhadj

Copyright © 2017 Ning Cai et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

This paper presents a novel approach for clustering, which is based on quasi-consensus of dynamical linear high-order multiagent systems. The graph topology is associated with a selected multiagent system, with each agent corresponding to one vertex. In order to reveal the cluster structure, the agents belonging to a similar cluster are expected to aggregate together. To establish the theoretical foundation, a necessary and sufficient condition is given to check the achievement of group consensus. Two numerical instances are furnished to illustrate the results of our approach.

1. Introduction

With a rapid increase in the scale of massive data and information, the probing of potential knowledge in big data, such as data structures and certain unknown correlations, has been attracting more and more scholars. As an essential technique of data mining, clustering has been widely applied in various practical fields, such as linguistics [1], bone microarchitecture analysis [2], community detection [3], wind power prediction [4], image segmentation [5, 6], and even genome expression study [7].

Clustering is about assigning certain set of data points into different clusters, for the purpose of highlighting the similitude of data points being organized in the same cluster, while simultaneously reflecting the distinctions between different clusters [8]. In the past decades, diverse clustering algorithms and criteria have been developed adequately, for example, normalized cut [6] and min-max cut [9] algorithms, the graph partitioning algorithms [10], the algorithms based on optimization of modularity functions [11], and the spectral clustering algorithms [12].

In addition to the abovementioned literature, some clustering algorithms have also been presented which are built on the study of certain dynamical behaviors of networks, for example, the random walk algorithms and the network synchronization algorithms. Random walk implies shifting along a random route, with each step to a new nearest vertex. By random walk algorithms [13], the clustering structure of a given network topology could be probed with the aid of generated random walk flows. Conversely, the network synchronization algorithm stands from another perspective [14], which considers coordinating different signals among vertices. Typically certain Kuramoto vibrator is employed, such that synchronization of vibrations could be achieved within the same group.

In this research article, a novel method for clustering is presented, which is rooted upon observation to the motions of dynamical multiagent systems. Concretely speaking, for the purpose of discerning the different affiliations of vertices in a given network, the selected dynamical multiagent system should be linked with the concerned network topology, and there should be a one-to-one correspondence between each pair of dynamical agent and vertex. The agents keep on moving inside a common space, either physical or abstract. Those corresponding to the vertices that are affiliated with any specific cluster should aggregate as time elapses. Finally, the clustering structure would be uncovered automatically, according to the formation of agent positions.

Analogical topics as discussed above are usually referred to as group consensus in the field of control theory. Yu and Wang earlier were concerned with this phenomenon in [15]. Hu et al. discussed group consensus for two classes of dynamical multiagent systems, respectively, which are systems with hybrid protocols (there is discontinuous transfer of information between nearest groups [16]) and systems with distinct groups comprising different types of agents [17]. To achieve group synchronization of the coupled vibrators, Su et al. considered both the adaptive pinning control strategy [18] and the case with multiple leaders [19]. Xie et al. [20] addressed the group consensus problem of first-order systems. Other recent relevant studies include [21–23].

The novelty of the current work compared with the aforementioned results in the literature lies in several perspectives. As far as we know, in the existing researches, the expected clustering formation is prescribed in advance, paying primary attention to synthesis of specific information transfer protocol for each group, so that the agents assigned to the group could ultimately aggregate. In contrast, in the framework of the current paper, the cluster distribution should completely be determined by the topology of network itself, without the need of any prerequisite knowledge aforehand. In this regard, the advantage of the current work is explicit, especially in its potential practicability.

Besides introducing a novel method for clustering, a major contribution of this paper is proposing a criterion for checking whether or not a high-order LTI (Linear Time-Invariant) multiagent system can reach group consensus, which generalizes the existing well-known necessary and sufficient condition for consensus achievement [24–28]. In fact, the existing condition on consensus is a particular instance of the condition for group consensus presented here.

In addition to the above, the current paper provides an exemplification of the usage of unstable dynamical systems, whereas, in contrast, unstable systems are conventionally regarded as being insignificant in control theory.

Theoretical studies on dynamical multiagent systems have already been extensive in the area of control theory, especially on the consensus problem. However, the application instances corresponding to these theories that can well support them are still scarce. Our exploration attempts to introduce a practical scenario from the field of data analysis, under a motivation to facilitate applying, verifying, or enriching certain relevant researches.

The remaining part of the current paper holds the following organization. The preliminaries and model formulation are introduced in Section 2. In Section 3, the clustering method is expounded in detail, based on theoretical analysis on condition for group consensus. Section 4 exemplifies the technique by two simulation cases. Finally, this paper is concluded in Section 5.

2. Model Formulation and Preliminaries

The topology of the network for clustering can be expressed by , with being the vertex set and the weighted edge set. The is assumed to be a graph of th order, being undirected and connected. The ultimate objective of clustering is to assign the elements of into divergent affiliations, in accordance with the topology of .

For such a purpose, a procedure rooted in observing the motions of dynamical multiagent systems is put forward, to figure out the clustering formation automatically. The procedure is composed of the following two fundamental steps.

Step 1. Define a dynamical multiagent system as follows and attach with it the network topology concerned, with each agent associated with one particular vertex.

where expresses the state of agent which is moving in a normed state space , is the weight indicating the strength of communication link between vertices and , and the functions   and   express the self-dynamics and the dynamical interagent information transfer protocol, respectively.

Step 2. Determine appropriate initial values and then let the dynamical system operate autonomously.

As time elapses, group consensus or quasi-consensus would manifest, as long as the overall setup of both the structure and parameters of dynamical multiagent system (1) is feasibly configured. Finally, a clustering result can be concluded via observing the setup formed by the agent states within the state space.

The definition below formulates the concept of group consensus discussed in this paper.

Definition 1. For the dynamical multiagent system (1), if

then agents and reach an agreement. For a vertex set , if , the agents and reach an agreement, and then system (1) reaches a consensus in . If consensus is reached in each , respectively, with

then the entire dynamical system reaches a group consensus.

Remark 2. The assumption that the graph is both undirected and connected is merely due to practical requirements of clustering and is not due to any technical limitations. This can be understood through later discussions.

3. Group Consensus of Dynamical Linear Time-Invariant Systems

In this section, the detail of the clustering process is elaborated by employing high-order LTI multiagent systems, which are described as

where expresses the state vector of agent i and matrices express the LTI self-dynamics of the agents and the information transfer protocol between nearest agents, respectively. The technique for successfully acquiring a rational clustering result depends on appropriately selecting values of the pair of matrices and .

To this end, some previous results on consensus are reviewed first.

Lemma 3 (see [28]). For the dynamical LTI high-order system (4), if it reaches consensus, then, as , the motion of any agent is regulated by the equation .

Lemma 4 (see [29]). The Laplacian matrix of a directed graph has exactly one zero eigenvalue iff includes a spanning tree, with the corresponding eigenvector . Besides, all the remaining eigenvalues have positive real parts.

Corollary 5. The Laplacian matrix of an undirected graph has exactly a single zero eigenvalue iff is connected, with the associated eigenvector . Besides, all the remaining eigenvalues .

Lemma 6 (see [28]). For the dynamical multiagent system (4) with as the eigenvalues of the Laplacian matrix of the directed graph , if is not Hurwitz then the system reaches consensus iff

(1)the graph topology includes a spanning tree;(2)all the matrices are Hurwitz.

Lemma 3 implies that if a certain subset of agents reaches a consensus, then the overall motion of this subset will ultimately be dominated by the matrix A. As long as A is Hurwitz, the motions of distinct consentaneous subsets would independently converge to the origin of state space, even without any exchange of information. Consequently, in order to differentiate agents between distinct affiliations, matrix A should not be Hurwitz. Actually, it is preferable for A to be unstable to avoid the possibility of critical stability, such that distinct clusters would mutually diverge.

Theorem 7. Consider the dynamical system (4). Suppose that the spectrum of Laplacian matrix of the directed graph with spanning tree is

with the series of matrices

being not Hurwitz and

being Hurwitz. The pair of agents and (or and , if ) reaches agreement iff the th row of the product possesses the configuration:

where represents any feasible solution of the matrix equation

represents the nonsingular matrix that transforms the Laplacian matrix into the similar Jordan canonical form

and “” denotes any arbitrary value.

Proof. If the stack vector of the states of agents is defined as

then the system dynamics can be described by

Let ; then (12) can be transformed into

which is equivalent to

where “” denotes an indefinite matrix that may be either or zero. Define the auxiliary vectors

and the stacked form as

It is evident that and

Substituting (9) into the above equation yields

It follows that

According to (14), ; thus because is Hurwitz. Also, holds, leading to , because is Hurwitz and . A similar analysis can be recursively conducted till is derived. As a result, it can be concluded that

Due to the structure of and the fact that , the limits of the first n entries of vector are zero and the limits of the entries indexed by to are indefinite, while the limits of the remaining entries are all zero. In other words,

Equation (19) can be rewritten as

According to the structure of , the fact that agents and reach an agreement implies that the entries in the left side of (22) indexed from to tend to approach zero. That is to say,

If , then (23) is true because of the structure of (21). Conversely, if , by noticing the fact that agreements are independent of the initial states and concerning the structure of (21), one can conclude that

Remark 8. Theorem 7 can verify the agreement between any two agents, since the index assignment is trivial for a network. With this theorem, an overall portrait of the group consensus can be easily derived.

Remark 9. For the sake of the clustering application, only the undirected graph should be concerned. However, Theorem 7 is more generalized since it addresses the systems with graphs that could have directed arcs.

Remark 10. Matrix can be figured out by solving the matrix equation (9) as

with expressing a Moore-Penrose inverse [30] of the singular Laplacian matrix . Besides, matrix is constituted by the generalized eigenvectors of . Therefore, it is simple to compute the matrix product .

Remark 11. Both the matrices and are derived from the Laplacian matrix. Thus, if only the value of is definite, the clustering result merely depends on the topology of graph.

Remark 12. Theorem 7 is a bridge connecting the concept of consensus with the diverse nonconsensus cases. The smaller the value of , the higher the degree of freedom that matrix would possess, for the sake of ensuring agreement. Here are two extremes: if , then the overall system reaches a consensus, whatever the value of is; oppositely, if all the matrices are not Hurwitz, then there should be no agreement at all. Actually, the existing well-known criterion for checking the consensus [24–28] (summarized as Lemma 6 here) can be regarded as a particular case or corollary of Theorem 7 when .

4. Simulations on Clustering

This section will exemplify the presented technique of clustering by two simple and typical simulation instances.

Example 1. Consider a graph in Figure 1.
The weighted adjacency matrix is

with the associated Laplacian matrix

and spectrum . Set the corresponding multiagent system as

In this instance, and are unstable and are Hurwitz. Therefore, . It yields that

Consequently,

Based on Theorem 7, it is explicitly known that agents 1~3 reach a consensus; meanwhile agents 4~6 reach another consensus. As a result, a clustering formation is clearly exhibited. The motions of agents in the 2-dimensional state space are illustrated in Figure 2.

Figure 1: Instance with group consensus. Default edge weight is 1.

Figure 2: Group consensus motions with . Thick dots denote starting positions.

Example 2. In the second example, consider the graph illustrated in Figure 3.
The adjacency matrix is

and the Laplacian matrix is

with the spectrum . Let the associated agents be LTI second-order systems with

In this case, , are unstable, whereas are Hurwitz. Still, . It can be easily derived that

As a result

According to Theorem 7, only two pairs of agents can achieve agreement, respectively, which are agents 1 and 2 and agents 5 and 6. The state trajectories obtained from the experiment are shown in Figure 4.
Although not all agents could achieve a precise group consensus in this case, from Figure 4, the six agents can still be differentiated as two distinct clusters, practically, because the speed of divergence between the two clusters is much faster than the speed of divergence of the agents inside any cluster, in a certain time span. The variation of difference between the states of agents 2 and 3 is illustrated in Figure 5, which is .

Figure 3: Second graph where default edge weight is 1.

Figure 4: Quasi-consentaneous state trajectories of Example 2 with . Thick dots denote starting positions.

Figure 5: Variation of difference between states of agents 2 and 3.

From Figure 5, one can clearly sense that the vicissitude of the difference between agent 3 and the other agents in the same cluster is sophisticated, including two phases: before , there is a tendency of agreement; however, afterwards, the tendency turns into a departure. This can be intuitively explained via a comparison with Example 1.

Let us inspect the dynamical equation of agent 3; that is,

The vector field is composed of three components as shown above: the autonomous force, the interactive force from the nearest neighbors of the same cluster, and the interactive force from the other cluster. During the first phase of Example 2, the effect of the third component is relatively weak, because of the small value of ; thus the states of agents belonging to the same cluster, that is, 1, 2, and 3, tend to achieve a consensus; however, as time elapses and the distance between the two clusters increases, the effect of the third component also keeps on increasing and ultimately it overwhelms the diminished second component and tears agent 3 away from its nearest neighbors 1 and 2. By contrast, in Example 1, a precise group consensus can be reached because the interactive force from the third component is balanced upon every agent.

5. Conclusion

A novel method for clustering is introduced in the current paper, based on group consensus or quasi-consensus of the motions of dynamical systems. For the sake of categorizing the vertex set into disparate clusters, a specific dynamical multiagent system is attached to the graph addressed, with each vertex associated with an agent. If the dynamical system is appropriately configured, then the agents affiliated with the similar cluster should gradually assemble together in their common state space; meanwhile the different clusters diverge away mutually. In addition to the method of clustering, a primary contribution is the presentation of a criterion for checking whether or not a group consensus could be reached by high-order linear multiagent systems, which is a necessary and sufficient condition generalizing the previous well-known condition of consensus. In fact, the scope of possible utility of the condition for group consensus is broader, beyond the clustering scenario being dealt with here. Two typical simulation instances are exhibited to exemplify the process of clustering method. As to the potential future extensions, there are several explicit major directions: () the method can be utilized to handle various real-world applications such as image segmentation, electrocardiosignal analysis, and classification of natural languages; () a further exploration toward the mechanism of group consensus or quasi-consensus could be conducted, and a more straightforward correlation between group consensus and the topological characteristics of networks might be obtained; () the patterns of diverse nonconsensus motions can be studied in depth.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work is supported by National Natural Science Foundation (NNSF) of China (Grants 61374054 and 61263002), by Fundamental Research Funds for the Central Universities (Grants 31920160003 and 31920170141), and by Program for Young Talents of State Ethnic Affairs Commission (SEAC) of China (Grant 2013-3-21).

References

A. Cangelosi and D. Parisi, Eds., Simulating the Evolution of Language, Springer, London, UK, 2002.

M. H. Edwards, D. E. Robinson, K. A. Ward et al., “Cluster analysis of bone microarchitecture from high resolution peripheral quantitative computed tomography demonstrates two separate phenotypes associated with high fracture risk in men and women,” Bone, vol. 88, pp. 131–137, 2016.

D. Liu, C. Wang, and Y. Jing, “Estimating the optimal number of communities by cluster analysis,” International Journal of Modern Physics B, vol. 30, no. 8, 1650037, 10 pages, 2016.

L. Dong, L. Wang, S. F. Khahro, S. Gao, and X. Liao, “Wind power day-ahead prediction with cluster analysis of NWP,” Renewable and Sustainable Energy Reviews, vol. 60, pp. 1206–1212, 2016.

A. Z. Arifin and A. Asano, “Image segmentation by histogram thresholding using hierarchical cluster analysis,” Pattern Recognition Letters, vol. 27, no. 13, pp. 1515–1521, 2006.

J. Shi and J. Malik, “Normalized cuts and image segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888–905, 2000.

M. B. Eisen, P. T. Spellman, P. O. Brown, and D. Botstein, “Cluster analysis and display of genome-wide expression patterns,” Proceedings of the National Academy of Sciences of the United States of America, vol. 95, no. 25, pp. 14863–14868, 1998.

L. Kaufman and P. Rousseeuw, Finding Groups in Data: An Introduction to Cluster Analysis, John Wiley & Sons, New York, NY, USA, 1990.

C. Ding, X. He, H. Zha, M. Gu, and H. Simon, “A min-max cut algorithm for graph partitioning and data clustering,” in Proceedings of the IEEE International Conference on Data Mining (ICDM '01), pp. 107–114, San Jose, Calif, USA, 2001.

D. G. Corneil and C. C. Gotlieb, “An efficient algorithm for graph isomorphism,” Journal of the Association for Computing Machinery, vol. 17, pp. 51–64, 1970.

M. E. J. Newman, “Modularity and community structure in networks,” Proceedings of the National Academy of Sciences of the United States of America, vol. 103, no. 23, pp. 8577–8582, 2006.

F. R. Chung, Spectral Graph Theory, CBMS Regional Conference Series in Mathematics, American Mathematical Society, 1997.

S. M. von Dongen, Graph clustering by flow simulation [Doctoral dissertation], University of Utrecht, 2000.

A. Arenas, A. Díaz-Guilera, and C. J. Pérez-Vicente, “Synchronization reveals topological scales in complex networks,” Physical Review Letters, vol. 96, no. 11, Article ID 114102, 2006.

J. Yu and L. Wang, “Group consensus in multi-agent systems with switching topologies and communication delays,” Systems & Control Letters, vol. 59, no. 6, pp. 340–348, 2010.

H.-X. Hu, L. Yu, W.-A. Zhang, and H. Song, “Group consensus in multi-agent systems with hybrid protocol,” Journal of the Franklin Institute. Engineering and Applied Mathematics, vol. 350, no. 3, pp. 575–597, 2013.

H.-X. Hu, W. Yu, Q. Xuan, C.-G. Zhang, and G. Xie, “Group consensus for heterogeneous multi-agent systems with parametric uncertainties,” Neurocomputing, vol. 142, pp. 383–392, 2014.

H. Su, Z. Rong, M. Z. Q. Chen, X. Wang, G. Chen, and H. Wang, “Decentralized adaptive pinning control for cluster synchronization of complex dynamical networks,” IEEE Transactions on Systems, Man, and Cybernetics B: Cybernetics, vol. 43, no. 1, pp. 394–399, 2013.

H. Su, M. Z. Chen, X. Wang, H. Wang, and N. V. Valeyev, “Adaptive cluster synchronisation of coupled harmonic oscillators with multiple leaders,” IET Control Theory & Applications, vol. 7, no. 5, pp. 765–772, 2013.

D. Xie, Q. Liu, L. Lv, and S. Li, “Necessary and sufficient condition for the group consensus of multi-agent systems,” Applied Mathematics and Computation, vol. 243, pp. 870–878, 2014.

H. S. Son, J. B. Park, and Y. H. Joo, “Segmentalized FCM-based tracking algorithm for zigzag maneuvering target,” International Journal of Control, Automation and Systems, vol. 13, no. 1, pp. 231–237, 2014.

N. Cai and L. Xue, “Clustering by group consensus of unstable dynamic linear high-order multi-agent systems,” in Proceedings of the 34th Chinese Control Conference (CCC '15), pp. 7212–7216, July 2015.

L. Guo, H. Pan, and X. Nian, “Adaptive pinning control of cluster synchronization in complex networks with Lurie-type nonlinear dynamics,” Neurocomputing, vol. 182, pp. 294–303, 2016.

F. Xiao and L. Wang, “Consensus problems for high dimensional multi-agent systems,” IET Control Theory & Applications, vol. 1, no. 3, pp. 830–837, 2007.

J. Wang, D. Cheng, and X. Hu, “Consensus of multi-agent linear dynamic systems,” Asian Journal of Control, vol. 10, no. 2, pp. 144–155, 2008.

J. Xi, M. He, H. Liu, and J. Zheng, “Admissible output consensualization control for singular multi-agent systems with time delays,” Journal of the Franklin Institute. Engineering and Applied Mathematics, vol. 353, no. 16, pp. 4074–4090, 2016.

Z. Li, Z. Duan, G. Chen, and L. Huang, “Consensus of multiagent systems and synchronization of complex networks: a unified viewpoint,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 57, no. 1, pp. 213–224, 2010.

N. Cai, J.-X. Xi, and Y.-S. Zhong, “Swarm stability of high-order linear time-invariant swarm systems,” IET Control Theory & Applications, vol. 5, no. 2, pp. 402–408, 2011.

W. Ren and R. W. Beard, “Consensus seeking in multiagent systems under dynamically changing interaction topologies,” IEEE Transactions on Automatic Control, vol. 50, no. 5, pp. 655–661, 2005.

R. A. Horn and C. R. Johnson, Matrix Analysis, Cambridge University Press, Cambridge, UK, 1985.