Estimating Network Flowing over Edges by Recursive Network Embedding

In this paper, we propose a novel semisupervised learning framework to learn the flows of edges over a graph. Given the flow values of the labeled edges, the task of this paper is to learn the unknown flow values of the remaining unlabeled edges. To this end, we introduce a value amount hold by each node and impose that the amount of values flowing from the conjunctive edges of each node to be consistent with the node’s own value. We propose to embed the nodes to a continuous vector space so that the embedding vector of each node can be reconstructed from its neighbors by a recursive neural network model, linear normalized long short-term memory. Moreover, we argue that the value of each node is also embedded in the embedding vectors of its neighbors, thus propose to approximate the node value from the output of the neighborhood recursive network. We build a unified learning framework by formulating a minimization problem. To construct the learning problem, we build three subproblems of minimization: (1) the embedding error of each node from the recursive network, (2) the loss of the construction for the amount of value of each node, and (3) the difference between the value amount of each node and the estimated value from the edge flows. We develop an iterative algorithm to learn the node embeddings, edge flows, and node values jointly. We perform experiments based on the datasets of some network data, including the transportation network and innovation. (e experimental results indicate that our algorithm is more effective than the state-of-the-arts.


Background.
Learning the flow direction and amount of the edges for a network has been a critical problem in the network analysis. e edges connect two nodes, and the flow is defined from one node to another connected by the corresponding edge [1][2][3]. is problem is called the edge flow estimation. In this problem, we already know the network structure, including the sets of the nodes, and the edges between the nodes. We also know the directions and amounts of the flows of some edges, but for the rest edges, the flows are still unknown. e target is to predict the flow direction and amounts of these edges. Some examples of the applications of the edge estimation are given as follows: (i) For example, in the transportation network, each intersection is a node, and each road is an edge, while the flows along the roads need to be estimated for the purpose of traffic control. According to the historical data of the traffic flows, we know the flows of some roads; however, other read flows need to be estimated. is use case raises the problem of learning edge flows from both the traffic network and existing flows of roads of the network [4,5]. (ii) Another example is the innovation network analysis, where each research article is a node, and each citation is an edge between two articles. For the task of innovation, it is important to know the flowing of knowledge from an article to other articles. Please note that one article cites another one that does not necessarily mean there is an amount of "knowledge" flowing from the cited article to the citing article. By reading the content of the article which cites the others, we can annotate the "knowledge flowing'' if one work is inspired or a following work of the other research works. However, such annotation by reading is time-consuming and subject to individual annotators, thus it is very necessary to develop an automatic system to estimate the knowledge flow from the article citation network [6][7][8][9][10][11][12][13][14].
Although the edge flow prediction/estimation is such a critical problem, the works on this direction are very few. Most recently, Ransom et al. [1] designed a new algorithm to predict the edge flow by balancing the amount of flows moving into and out of the nodes. Another condition of this algorithm is that the predicted flow of edges should be consistent to the known flows, for these edges whose flows are already known. In this paper, we proposed a novel edge flow estimation algorithm based on both semisupervised learning and network embedding.

Our Contributions.
In this paper, we build a novel method for the problem of edge flow prediction. is method is both for network embedding and edge flow prediction. e two problems are solved at the same time. We designed a new framework for the learning problem. In this framework, for the first time, we bridge the node embedding and edge flow estimation, by introducing a node value for each node. On the one side, the node value is used to balance the flowing-in and flowing-out amount of a node. It plays the role of measuring the balance of the amount of the node at any moment of the flowing process, i.e., with the incoming flow and outgoing flow changing, the amount of the value in this node remains as the node value. One the other side, the node is employed to regularize the learning of the embedding vectors, thus we impose that the node value can be estimated from the embedding vector by a linear function.
We propose a novel algorithm to learn the embedding vectors and edge flows simultaneously. We model the learning problem as a minimization problem where the embedding vectors reconstruction error of the LSTM embedding model, the node value estimation error from the embedding vectors, and the node value flow amount error are minimized. In the iterative algorithm, the node embeddings, node values, and the edge flow amounts are optimized alternately.
We evaluated the proposed method over benchmark datasets of networks, conducted experiments to reveal the properties of the proposed algorithms, and show its advantage over state-of-the-art methods.
Remark: the superiority of the proposed semisupervised edge flow learning method compared with the traditional semisupervised learning method is listed as follows: (1) e traditional semisupervised learning methods can only predict the labels of the nodes but are not able to predict the flows of the edges. For the applications discussed in this paper, the traditional semisupervised learning methods are not suitable. Our method is especially designed for these applications. (2) Traditional semisupervised learning methods can only use the edge information of a graph but ignore the flow information, which is critical for both the node and flow label prediction. However, our method can effectively use them to learn better node embeddings and flow amounts.

Paper Origination.
is paper is organized as follows. In Section 2, we introduce the joint learning framework, with an objective and optimization solution. In Section 3, we experimentally evaluate the performance of the proposed method and conduct studies over its properties. In Section 4, we conclude this paper with some future works.

Proposed Method
In this section, we introduce the proposed joint network embedding and edge flow learning framework. In this framework, we propose a deep learning-based network embedding method [15][16][17][18][19][20][21][22][23] and further use the embedding vectors to estimate the flow amount regarding each node [15]. e flow amount is used to regularize the edge flow learning process.

Problem Definition. Suppose we have input network, denoted as
} is the set of n nodes, and ε k � (i, j) ∈ E is an edge linking the i-th and j-th nodes. Here, we assume that i < j. For a group of edges, we already know their flows. is set of edges is denoted as ε k ∈ E L . For such an edge ε k , we define its flow as f k ∈ R. e direction of the flow is the sign of f k , and the amount of the flow is the absolute value of f k . For the other edges, the flows are unknown, and we want to predict them. For these flows, we define a set as E U . We define a vector as the flows of all the nodes, as f � [f 1 , . . . , f |E| ] T ∈ R |E| . us, the prediction of the flows of the nodes is transformed to the solving of f.

Problem
Modeling. We firstly embed each node i ∈ V to a vector. e dimension of the vector is denoted as d. Moreover, for each node, we define a node value, ϕ i . We also propose to calculate the node value from the node's embedding vector. After ϕ i is given, we use it to regularize the estimation of the flow of edges connected to the node. e flow chart of the proposed semisupervised edge flow learning method is given in Figure 1.

Recursive Node Embedding.
e network embedding converts the nodes to a set of vectors, denoted as x 1 , . . . , x n , where x i ∈ R d . To this end, we want to reconstruct the embedding vector of one node from its linked nodes. e linked nodes are presented as a sequence of nodes. Accordingly, the sequence of neighbouring nodes of the i-th node are given as We firstly sort the nodes in the neighbouring set, N i , and the sorted set's embedding vector set is 2 Shock and Vibration We apply aln-LSTM to this sequence of embedding vectors. In this model, for an input node's embedding vector, x t ∈ S i , its inputs are x t itself and the output of its previous where g is the cell function of ln-LSTM and θ is its parameter. en, we try to use the output of the cell function from the last node to approximate the embedding vector of the i-th node itself, us, the minimization of the error of the approximation is modelled as By solving this problem, the learning of the embedding vector and the parameter of the cell function are solved together. With the good quality embedding and cell function parameter, we should be able to approximate the embedding vectors for the neighbouring nodes's embedding vectors.

Node Value Estimation from Recursive Embedding.
In out learning framework, the embedding of each node approximated by the recursive model plays two roles, representing the node's neighbor structure and estimating the amount of value hold by the nodes. We define a node value for the i-th node, ϕ i ∈ R. e function of this value is to balance the incoming and outgoing flow of the node. Moreover, it can somehow measure the nature of the node. For example, a node in traffic network at the working time may have more incoming flow, because it is an office area.
is value can indicate this node's nature of being in office area. Moreover, we want to use the embedding vector of the node to calculate the value as follows: where φ(x) � σ(w T h) is the function of node value approximation. is function is actually a single-layer neural network. We proposed to minimize the square error of the approximation over all the nodes, e minimization is performed regarding the node value amount, embedding vectors, recursive model parameters, and the single-layer neural network parameter. In this way, we bridge the learning of the flow amount and embedding of nodes by using an LSTM recursive model.

Flow Prediction by Using Node Value.
Given an edge ε k ∈ E, we want to predict its flow f k . To this end, we use the node value as reference. Our assumption is that for any node, the value of the node is controlling the incoming and outgoing flow. To be more specific, for a node, the difference between its incoming flow and outgoing flow should be equal to the node value itself. For this purpose, we define two sets of edges for a node R + i � ε k | ε k ∈ E, ε k � (u, i) and . e amount of the incoming flow is k∈R + i f k − k∈R − i f k . Since we hope this amount is equal to the node value, We also denote the node value of all nodes in a vector ϕ � [ϕ 1 , . . . , ϕ n ] T ∈ R n . A matrix is also proposed as It is the node-flow mapping matrix. So, equation (8) is transformed to We minimize the squared approximation error regarding both the edge flow vector and the node value vector, e final objective and minimization problem is the combination of (5), (7), and (10) as follows: Shock and Vibration 3 Here, we also add the l 2 norm regularization terms to the parameters to prevent the over-fitting problem.

Problem Solution.
e solving of the minimization problem is conducted in an iterative algorithm. In each iteration, every parameter is updated sequentially. e others are not updated when one parameter is being updated. In the following subsections, we introduce the optimization of each parameter one by one, while the others are fixed.

Optimization of LSTM Parameters, θ.
When the other parameters are fixed, and only the LSTM model parameters are considered, we have the following minimization problem: which can be solved by the ADAM algorithm. To use the ADAM algorithm, we calculate the gradient function of the objective o 1 (θ) regarding θ as follows: where zf(x)/ zx is the derivative function of f regarding variable x.

Optimization of Embedding
Vectors, x i | n i�1 . When the other variables are fixed and only the embedding vectors are considered, we have the following minimization problem: To solve the vectors, we apply the sequential optimation method and optimize the embedding vectors of the nodes one by one. When one node embedding vector is optimized, others are fixed. When the i-th vector, x i , is considered, we have the following problem for minimization: Again, we use the ADAM algorithm to solve this problem similar to the optimization of θ.

Optimization of Edge
Flow Vector, f. When the edge flow vector is considered, we have the following minimization problem: is is a linear constrained quadratic programming problem; we employ the active-set algorithm to solve it.

Optimization of Node Value Vector, ϕ.
To solve the node value vector, we fix the other variables and have the following minimization problem: We set the derivative of o 4 regarding ϕ to zero and have the solution of ϕ as

Optimization of w.
To solve the parameter of function ϕ, we have the following minimization problem: We still employ the ADAM algorithm to solve this minimization problem.

Experiments
In this section, we give the experimental setting and results. e algorithm tested and developed is called edge flow prediction by network embedding (EFPNF).

Benchmark Dataset and Experimental
Setting. Four datasets of network are used in our setting. We summarize the datasets in Table 1. e 10-fold cross-validation is used to generate the training and test set. e Pearson correlation coefficient is used to measure the quality of the flow predicted by the algorithm [28]. Figure 2, the algorithm is compared to the others which are also used to predict the flows of edges. e compared algorithms are flow SSL algorithm proposed by Jia et al. [1] and the LineGraph algorithm. According to the figure, the proposed algorithm EFPNF has better performance for all four datasets than the compared algorithms. Especially for the Balerma dataset, the EFPNF is the only algorithm which has the correlation score larger than 0.7. We also can see that the knowledge network dataset is the hardest task, and the EFPNF still gives the best performance.

Convergence Analysis.
Since our algorithm is an iterative algorithm, we are also interested in the convergence of the algorithm. us, we plot the curve of correlation while the number of iterations is increasing (Figure 3). We can see from these curves that our algorithm's performance is boosted when the iteration number is increased from 5 to 20. Since our algorithm aims to minimize the objective function, more iteration numbers result in a smaller value of the objective. is verifies the effectiveness of the objective to  Shock and Vibration 5 achieve a better edge flow estimation performance. However, when the iteration number further increases from 20 to 100, the performance improvement is not significant, meaning our algorithm does not need a large number of iterations to give a good performance.

Conclusion
We developed an iterative algorithm to learn the node embeddings and missing flows of the edges for a network. is algorithm is based on the deep recurrent network for the embedding purpose. Moreover, it uses the embeddings to calculate the node values and further uses the node values to approximate the flows around the node. A unified learning framework is built for the learning of embedding network, node value, and flows of the edges. e learning process is guided by the minimization of the reconstruction errors of embedding vectors, node values, and incoming/ outgoing flows. Experimental results show the advantage of the proposed algorithm.

Data Availability
All the datasets used in this paper to produce the experimental results are publicly accessed online.  Shock and Vibration