Detecting Communities in 2-Mode Networks via Fast Nonnegative Matrix Trifactorization

With the rapid development of the Internet and communication technologies, a large number of multitype relational networks widely emerge in real world applications. The bipartite network is one representative and important kind of complex networks. Detecting community structure in bipartite networks is crucial to obtain a better understanding of the network structures and functions. Traditional nonnegative matrix factorization methods usually focus on homogeneous networks, and they are subject to several problems such as slow convergence and large computation. It is challenging to effectively integrate the network information of multiple dimensions in order to discover the hidden community structure underlying heterogeneous interactions. In this work, we present a novel fast nonnegative matrix trifactorization (F-NMTF)method to cocluster the 2-mode nodes in bipartite networks. By constructing the affinitymatrices of 2-mode nodes asmanifold regularizations of NMTF, wemanage to incorporate the intratype and intratype information of 2-modenodes to reveal the latent community structure in bipartite networks.Moreover, we decompose theNMTFproblem into two subproblems,which are involvedwithmuch lessmatrixmultiplications and achieve faster convergence. Experimental results on synthetic and real bipartite networks show that the proposed method improves the slow convergence of NMTF and achieves high accuracy and stability on the results of community detection.


Introduction
Community structure is an important feature of real-world networks as it is crucial for us to study and understand the functional characteristics of the real complex systems.A community is usually thought of as a group of nodes with more interactions among its members than outside of the network.With the rapid growth of the Internet and computational technologies in the past decade, numerous community detection algorithms [1][2][3] have advanced swiftly from the simple clustering of homogeneous datasets (1mode network) to heterogeneous datasets (multiple mode networks).Unlike homogeneous networks that only contain 1-mode nodes and have explicit community structure, the community structures of heterogeneous networks are usually obscure and complicated owing to the coexistence of multiple-mode interactions.Therefore, it is challenging to effectively integrate the network information of multiple dimensions in order to discover the hidden community structure underlying heterogeneous interactions.
The 2-mode network [4] is a simple and important kind of heterogeneous networks, which is composed of 2 kinds of disjoint node sets (for simplicity, we call them as subject-node set   = { 1 , . . .,   } and object-node set   = { 1 , . . .,   }).So the links only connect two end nodes from different sets and no links between nodes of the same set (Figure 1).In many studies, 2-mode networks are called bipartite networks by most researchers.Bipartite network is also the most common complex networks in the real world.Many real-world networks may be naturally modeled as bipartite graphs, such as actor-movie, authorpaper, word-document, product-consumer, and context-tag networks.Thereby, it is necessary and valuable for us to study community detection in bipartite networks.
Due to the special link patterns and multirelational nodes of bipartite networks, it is more suitable for mining the special community structure via coclustering methods.Recent works [5][6][7] have shown that clustering multimode datasets simultaneously, that is, coclustering, can effectively improve the clustering performance in the sense that coclustering can make full use of the dual interdependence between heterogeneous nodes to discover certain hidden community structures.In contrast to traditional one-side clustering (on either columns or rows), coclustering treats the data matrix in a symmetric form that a partitioning of rows can induce a partitioning of columns, and vice versa.By clustering both rows and columns of a data matrix simultaneously, coclustering can effectively deal with the multimode networks and discover the structure of them.
Among different approaches of coclustering, nonnegative matrix trifactorization (NMTF) provides a superior model for co-clustering, in which the relations between row and column clusters are explicitly embodied.There are three major advantages with NMTF methods.First, as shown in (1), NMTF introduces one more factor matrix  to absorb the different scales of , , and  and provides increased degrees of freedom such that the low-rank matrix approximation remains accurate.Second, instead of being independent, the clustering tasks of multimode nodes in heterogeneous networks are often closely related.NMTF coclusters both the rows and columns of the original networks simultaneously by making efficient use of the duality (intratype and intertype) information between 2-mode nodes.Finally, in NMTFbased coclustering scenario, the three factor matrices jointly determine the appropriate latent community structure of bipartite networks, where  and , respectively, cluster the subject nodes and object nodes and  is the correlation matrix reflecting the relationship between subject clusters and object clusters.Through the NMTF model, we can seamlessly integrate multiple node information to discover the underlying community structure in bipartite networks, which is highly valuable in many real world applications.
Here we adopt nonnegative matrix trifactorization as a tool to find the communities because of its powerful interpretability and applicability for coclustering in heterogeneous networks.In this work, we present a novel fast NMTF solution for community detection in bipartite networks.To summarize, the main contributions of this work include the following.(1) We construct the affinity matrices of 2-mode nodes as manifold regularizations of NMTF, which efficiently incorporate the intertype and intratype information of subject and object spaces, to enhance the community detection in bipartite networks.(2) We present an optimization algorithm for fast nonnegative matrix trifactorization (F-NMTF), which decouple the original optimization problem into two smaller subproblems requiring much less matrix multiplications, and then cocluster the 2-mode relational nodes from the heterogeneous interactions.
The remainder of the paper is organized as follows.In Section 2, we define the NMTF model of community detection and demonstrate the theoretical foundations of our approach along with an illustrative example.In Section 3, we formulate the coclustering NMTF method for community detection in bipartite networks and present an optimal algorithm to achieve fast convergence.Then we test our algorithm on a variety of artificial and real bipartite networks and present the experimental results in Section 4. Finally, Section 5 concludes the paper.

Preliminary
2.1.Related Work.In the past years, NMF-based algorithms [8][9][10][11] for detecting network communities have gained great attention, because the data matrix factorization effectively reflects the community structure of the networks and promises a meaningful community interpretation that is independent of the network topology.In addition to a quantification of how strongly each node participates in each community, nonnegative matrix factorization (NMF) does not suffer from the drawbacks of modularity optimization methods [12], such as the resolution limit [13].Psorakis et al. [8] proposed a Bayesian NMF model to extract overlapping modules.This method can automatically determine the number of communities, but also may mislead the factorization to return a bad solution, when some errors come out with its estimate of the community number.Cao et al. [9] used nonnegative matrix factorization with I-divergence as the cost function and introduce two approaches which are respectively applied to the directed and undirected networks.In [10], Nguyen et al. developed a novel NMF model where vertexes are measured by their centrality in communities and detect overlapping communities, hubs, and outliers from the NMF framework altogether.Based on the importance of each node when forming links in each community, He et al. [11] use nonnegative matrix factorization to form a generative model and take it as an optimization problem to discover the structure of link communities.
However, most community detection methods still focus on homogeneous networks and might not work well in the bipartite networks.There are two main reasons.First, the special link patterns of bipartite networks greatly limit the effectiveness of these methods, which tend to cluster the heterogeneous nodes by constructing the node or edge similarities of them, as they did in the unipartite networks.But for bipartite networks, the similarities among one-mode nodes sometimes can only be defined by the nodes of the other mode.That made these methods cannot keep working well in bipartite networks.Second, bipartite networks contain multiple types of nodes that are related to each other.Tackling each type independently will lose these interactions.So it is necessary for us to utilize the 2-mode nodes to gain a full understanding of the bipartite networks.
Motivated by recent progress in coclustering and matrix factorization, several novel NMF-based algorithms have been proposed to detect the underlying community structure, including GNMF [14], DRCC [15], and BNMTF [16].These researches showed that the NMTF model is more applicable to discover the hidden structures in the bipartite networks.Compared to the classical clustering algorithms, nonnegative matrix trifactorization provides a good model for coclustering, in which the relations between row and column clusters are explicitly embodied.In view of these facts [17][18][19][20], our approach aims to cocluster both the rows and columns of the bipartite networks simultaneously by making full use of the background information of 2-mode nodes.It is believed that adding intratype information as additional constraints can definitely improve the clustering performance.
In this work, we extend the NMTF model by adding the intratype information of 2-mode nodes as manifold constraints and develop an optimization solution to improve the coclustering performance of bipartite networks.In addition, due to the fact that NMTF-based methods often require the community numbers of networks to be specified beforehand, several methods [10,21,22] have been developed to solve this problem.Due to the simplicity and practicability of the existed method in [10], here we choose it to get the community numbers.

An Illustrative Example.
Before the introduction of our NMTF-based method for community discovery, let us see an illustrative example on how NMTF works on the bipartite networks.Considering a bipartite network (  ,   , ), where   is the subject-node set,   is the object-node set, and  denotes the link set of the network , |  | = , |  | = .Given an asymmetric adjacent matrix  ∈ R × denoting the bipartite network , NMTF methods approximates  by looking for 3 low-rank factor matrices with the form where  ∈ R × and  ∈ R × , respectively, cluster the subject nodes and object nodes and jointly determine the appropriate latent community structure in bipartite networks,  ∈ R × . is associated with the number of the subject-node clusters, and  is the number of object-node clusters.In most cases, let  =  as the preseted community number.In this way, NMTF can simultaneously group the subject-node set and the object-node set into  clusters, where each community is the mixture of the 2-mode heterogeneous nodes.
Let   represent the weight of the edge connecting subject-node  and object-node .Generally, the original NMTF can be computed using the following multiplicative update rules [23]: The NMTF framework of community detection on bipartite networks is shown in Figure 2.Here the product   can be regarded as an approximate form of the network.Thus, the results of NMTF methods can be interpreted in which   indicates the membership internal-strength of the subjectnode  in the th community and   denotes the membership internal-strength of the object-node  in the th community.
Consider a toy network with |  | = 6, |  | = 8 nodes and || = 36 edges of varying weights (Figure 3(a)).Specifically, the NMTF coclusters the 2-mode node sets to yield a comprehensive network partition solution (Figure 3(b)), where  is the subject nodes' community indicator, and  reflects the community structure of object nodes.The larger square indicates the larger value of a corresponding element in the matrix.Specially, it is worthwhile to note that the matrix  represents how subject-clusters are related to objectclusters.Each column in  reflects which subject-clusters make contribution to each object-cluster.In this case, subjectcluster 1 and 3 contribute to object-cluster 1, while subjectclusters 2 and 3 contribute to object-cluster 2.
Hence, our NMTF framework can simultaneously cocluster both the subject set and object set.By incorporating the duality information of 2-mode nodes, we can effectively capture the heterogeneous community structure in bipartite networks.From the following illustrative example, we can see that the coclustering results of NMTF intuitively agree with the real community structure of the bipartite network and directly indicate the relationship between subject clusters and object clusters.

Fast NMTF Algorithm for Coclustering Community Detection
In this section, we propose a fast nonnegative matrix trifactorization (F-NMTF) method as the solution of community detection on bipartite networks.First, we respectively formulate the affinity matrices of 2-mode node sets in bipartite networks and incorporate them as two manifold regularizations of subject-set and object-set in the objective function.
Then we decompose the original optimization problem into two smaller subproblems and present an optimization algorithm to develop the iterative updating rules of three factor matrices.For convenience, we present in Notations section the important notations used in this paper.

Constructing the Affinity Matrices.
For the original NMTF framework (on (1)), it just considers the intertype information of 2-mode nodes.Such formulation assumes each subnetwork to be independent and fails to model the bipartite networks in a unified way.Recently, some researchers [15,24] have found that coclustering data on manifold is well applied to bipartite networks for regularizationbased clustering, because it can promote the performance of the intrinsic structure discovery in multimode networks.As a result, by constructing the two affinity matrices, the optional intratype information of 2-mode nodes is incorporated into NMTF as manifold regularizations.More importantly, we can exploit the manifold structures in both subject and object spaces to group like-minded users from different social perspectives, thus strengthening the community detection in bipartite networks.First, we construct the affinity matrix   as [15,25] whose entries correspond to subject nodes   = { 1 , . . .,   }.Generally, if nodes   and   share most connections to the nodes of the other mode, it means they are close to each other, and then their corresponding indicator vectors  ⋅ and  ⋅ should be close as well.It is formulated as follows: where ‖ ⋅ ‖ is the Frobenius norm,  ⋅ is the th row of the subject indicator matrix  and indicates the community membership of subject-node   .  (, ) is the affinity relationship between community indicator vectors  ⋅ and  ⋅ .
For simplicity, we define the affinity matrix   as follows: where N(  ) denotes the -nearest neighbor of   .In addition, other kinds of affinity can also be adopted, for example, heat kernel [26].And (3) can be further rewritten as where    = ∑    (, ) is the diagonal degree matrix and is the normalized graph Laplacian of the subject-node set   .
Likewise, we also define the affinity matrix   whose entries correspond to object nodes   = { 1 , . . .,   } as follows: where N(  ) denotes the -nearest neighbor of   .And (3) for object-node set can be further rewritten as where  ⋅ is the th column of the object indicator matrix  and indicates the community membership of object-node   .
= ∑    (, ) is the diagonal degree matrix, and is the normalized graph Laplacian of the object-node set   .
Here we construct two affinity matrices based on the intratype information of 2-mode nodes and introduce them as manifold constraints to explore the hidden community structures of bipartite networks.In the following, we impose these two constraints on NMTF to achieve additional flexibility and incorporated the intratype information to enhance the orthogonality and accuracy on matrix factorization.

Objective Function of F-NMTF.
Applying the two manifold regularizations in (1), the objective of our F-NMTF approach is transformed to minimize where  ≥ 0 and  ≥ 0 are regularization parameters to balance the reconstruction error of F-NMTF in the first term and manifold regularizations in the second and third terms.Adding regulations to NMTF is a common strategy, since it not only improves interpretability, but also enhances numerical stability of the estimation by making the NMTF optimization less underconstrained.
Since  and  are constrained to be cluster indicator matrices, it is difficult to solve (8) in general.Hence we simplify this problem by using the following proposition.
The two manifold regularizations terms in (8) can be rewritten as Then, applying Proposition 1 in (8), the objective of our F-NMTF approach is transformed to minimize where   and   are computed from   and   following the procedures described in Proposition 1.
This section presents a general framework of NMTF, which are developed to introduce the affinity matrices as the dual manifold regularization.Hence, our F-NMTF method successfully incorporates the intratype and intertype information of the bipartite network to coclustering multitype relational networks.

Optimization Iteration.
Due to the fact that NMTF algorithms always have high computation complexity, it is essential and valuable to introduce fast iterative rules for iteration optimization.In this subsection, as a step toward accelerating convergence of NMTF, we apply a fast iterative algorithm to alternatively optimize the objective, with computing one factor matrix while fixing the other variables.

Theorem 2. Given a general optimization problem:
when  and  are fixed, the optimum  is given by  =   , where  =    and the singular value decomposition (SVD) of  is given by  = Λ  .
According to Theorem 2, the optimization problem of F-NMTF can be decoupled into two subproblems with much smaller sizes, and the decoupled subproblems would involve much less matrix multiplications.In this way our approach can be computationally efficient and scale well to large-scale real world networks.Now we alternatively optimize the four variables of the objective function (11).First, fixing  and , by setting the derivative of ( 11) with respect to  as 0, we obtain Second, by fixing , , and , we can decouple (11) Similarly, we obtain  as where  ⋅ corresponds to th community of the subject-node set.Repeat this procedure until convergence.The convergence and correctness of this alternating iterative procedure have been proofed in [15,23].We skip it due to space.
After iterations, we can infer the community membership of heterogeneous nodes based on the F-NMTF results.For simplicity, the community indices of subject/object nodes are determined by taking the maximum of each column/row in /.The detailed procedure is illustrated as shown in Algorithm 1.
In our algorithm,  and  are sparse matrices, and the computation of them only involves vector norm enumeration without matrix multiplication; thus it is more computationally efficient.Moreover, instead of minimizing each matrix factor optimally with time-consuming multiplications of large matrices, F-NMTF decouples the original optimization problem into two smaller subproblems requiring much less matrix multiplications and coclusters the 2-mode relational nodes in bipartite networks.Compared to the other representative NMF solutions, it effectively optimizes the matrix factorization with faster convergence and lower computational complexity.

Experimental Results
In this section, the experiments use a series of computergenerated synthetic networks and some real networks to validate the algorithms' performance.For all the bipartite networks, we compare the experimental results with other 4 well-known algorithms of community detection: Kmeans [27], GNMF [14], DRCC [15], and BNMTF [16].All the experiments are performed on an Intel Core2 Duo 2.0 GHz PC with 1 GB RAM, running on Windows XP.
For NMF-based methods, including GNMF, DRCC, and our F-NMTF methods, the number of nearest neighbor  is set by the grid {1, 2, . . ., 10} as in [25], and the regularization parameters  and  are set to 0.1.In addition, we obtain the community numbers  from the method as suggested in [10], which has been shown to well predict the number of network communities.In our experiments, we repeat each method with 50 times on all the bipartite networks and compute the average results.
In the following tests, different measures are introduced to evaluate the partition quality of the classical algorithms for community detection in bipartite networks.Since the structures of synthetic networks are all known, we adopt two standard measures widely used for clustering: normalized mutual information (NMI) [28] and execution time to quantify the partition quality of the community detection methods.For the real networks with unknown structure, we use bipartite modularity [29] and execution time for the collective validation.In particular, bipartite modularity is extended from Newman-Girvan modularity [12] and regarded as an effective standard to measure the community structure of partition results.
(1) Synthetic Networks.Here we have applied the model bipartite networks [29] to generate 9 groups of benchmark datasets with known structures.Each group contains 10 networks that are generated with the same parameters.In this experiment, we randomly choose 5 networks of each group to obtain the average values for comparison.Here  in is fixed at the value of 0.9 while  out is varied by tuning  out / in from 0.1 to 0.9 with steps of 0.1. in and  out , respectively, denote the intracommunity and intercommunity link probability.Generally, the higher the intracommunity link probability of the network is, the stronger community structure can be detected.Conversely, as  out / in becomes bigger, the community structure of the model bipartite network should become more and more obscure.We set community number |  | = 80, subject-node number |  | = 600, and object-node number |  | = 1000.
Due to the fact that Kmeans costs much less time for computation and the fluctuation of  out / in has limited effect on it, we mainly display the execution time of NMF-based methods in Figure 4(a), where  out / in of the model bipartite networks ranges from 0.1 to 0.9.As  out / in increases, the community structure of the model bipartite networks become weaker, and all the algorithms suffer varying degrees of performance degradation, and their execution time rises rapidly.Because iterative methods are usually necessary for NMF solution, they need to update matrix factors by multiplying each entry in each iteration round until convergence.Meanwhile, we can see that F-NMTF converges faster than other NMF methods, and the gap of execution time between them becomes greater, because F-NMTF is decoupled into two subproblems with much smaller sizes and optimizes the factorization of each matrix.Even the community structure of the synthetic bipartite networks tends to be weaker and weaker; our algorithm also shows much better robustness than the other three algorithms.
Figure 4(b) shows the average NMI values of different algorithms against changes of  out / in .When  out / in ≤ 0.4, the NMI scores of all the algorithms performance exceed 0.8.Specifically, three NMF methods have the similar well performance under the strong community structures, and only Kmeans is slightly inferior to other methods.But when  out / in ≥ 0.5, the community cohesion of model bipartite networks degrades along with  out / in gradual increase, and the performance of F-NMTF still remains to have relatively higher NMI scores and keeps the stability and accuracy of community detection, rather than tending to quickly decrease like other algorithms.Our method is also superior in terms of stability as well as approximation accuracy, which means that it can achieve small recovery errors for various source networks.Specifically, the average NMI of F-NMTF is up to 5.83% better than that returned by GNMF, and 3.13% better than DRCC.In summary, the performances of F-NMTF are highly competitive to those of other methods on bipartite networks.
(2) Real Networks.Real networks are always more irregular and various than synthetic networks and have more complex community structures.Here we choose 5 popular real bipartite networks in different sizes: Southwomen [30], Scotland [31], Irvine forum [32], MovieLens [33], and Cond-mat [34].These networks' basic properties (about nodes, edges) are presented in Table 1.
Southwomen shows the participation of 18 women in 14 social events over a nine-month period.The dataset was collected in the Southern United States of America in the 1930s.There is an edge for every woman who participates in an event.Irvine forum contains user posts to forums.The users are students at the University of California, Irvine.An edge represents a forum message.MovieLens consists of 100000 user-movie ratings from https://www.movielens.org/.An edge between a user and a movie represents a rating of the movie by the user.Cond-mat contains authorship links between authors and publications in the arXiv condensed matter section from 1995 to 1999.An edge represents an authorship connecting an author and a paper.Scotland dataset contains the corporate interlocks in Scotland in the beginning of the twentieth century.It lists the (136) multiple directors of the 108 largest joint stock companies in Scotland in 1904-1905, including 64 nonfinancial firms, 8 banks, 14 insurance companies, and 22 investment and property companies.
The average execution times found by different algorithms are shown in Table 2(a).We can see that Kmeans costs much less time than NMF-based algorithms, as it does not need the matrix factorization iterations.For all the real bipartite networks, F-NMTF effectively accelerates the convergence speed of nonnegative matrix factorization and converges in less iterations and CPU seconds than other NMF methods.Because the network scales are quite different, the corresponding performances of F-NMTF are different, too.For the larger networks, F-NMTF has a greater competitive advantage than other methods.Our method is only slower than Kmeans, which, however, has much worse clustering performance.
Table 2(b) shows the bipartite modularity values found by different algorithms.The methods using manifold constraints, including GNMF, DDRC, and F-NMTF, generally achieve better clustering results, which are consistent with the widely accepted hypothesis that clustering of both intratype and intertype information can help clustering of bipartite networks.F-NMTF method attains the maximum modularity community structure for most test cases, which means that our method has better partition quality and achieves accuracy community structure on the real bipartite networks.More importantly, our method does not suffer from the problems of modularity optimization methods and clusters both the rows and columns of the networks simultaneously by making full use of the duality information between 2-mode nodes, which can greatly enhance the performance of clustering algorithms.Therefore, compared to other 4 methods, we can conclude that F-NMTF has competitive clustering performance in terms of both accuracy and partition quality against popular community detection methods and with much faster computational speed.

Conclusions
In this work, we proposed a novel fast nonnegative matrix trifactorization method for community detection in bipartite networks.Based on the idea of nonnegative matrix trifactorization, we introduce the intratype information of 2-mode nodes into NMTF via the dual manifold regularizations, thus helping to extract meaningful communities of the bipartite networks.Meanwhile, we decouple the NMTF problem into two subproblems with much smaller sizes, and the decoupled subproblems involve much less matrix multiplications, which make our approach of particular use for real world large-scale data.
Different from the traditional NMF-based methods, our work is an instructive attempt to cocluster multitype nodes in bipartite networks.In practice, our coclustering NMTF framework jointly takes intertype and intratype information of 2-mode nodes into considerations, thus making the partitioning results more reasonable and effective and detecting communities with high accuracy and quality.Experimental results on the synthetic and real-world datasets show that our algorithm is a competitive method to explore community structures in bipartite networks.

Notations
: Adjacent matrix of a bipartite network   : Subject-node set of a bipartite network   : Object-node set of a bipartite network

Figure 2 :
Figure 2: The solution framework of fast nonnegative matrix trifactorization.

Figure 4 :
Figure 4: The average execution time and NMI of the community detection methods on the model bipartite networks with different  out / in parameters ranging from 0.1 to 0.9.

Table 1 :
Basic properties of the real bipartite networks adopted in our experimentation.

Table 2 :
Community detection results: (a) execution time; (b) bipartite modularity.Each entry shows the average values of the detected community structure for the 5 community detection methods on the real bipartite network.