Network Embedding-Aware Point-of-Interest Recommendation in Location-Based Social Networks

,


Introduction
With the popularity of mobile devices and the development of global positioning system, location-based social networks (LBSNs) have emerged in recent years (e.g., Gowalla (https:// en.wikipedia.org/wiki/Gowalla) and Foursquare (https:// foursquare.com/)), which enable people to share their experiences with friends and check in at interesting places (e.g., restaurants, tourist spots, and stores). POI recommendation as one of the important techniques to explore interesting unknown places has been well studied.
Compared with traditional recommendation tasks, POI recommendations are challenging for three reasons. First, POI recommendation methods often su er from the data scarcity and user/item cold-start problems, which means only a small number of users have su cient check-in data, and most of the users have very little or even no check-in data (as shown in Figure 1). In our experiments, the data scarcity is more than 99%. Second, in LBSNs, the check-in data are not explicit but implicit, which makes POI recommendation more di cult. In check-in data, only positive behaviors can be observed, from which we can only get the POIs that the users have visited, but cannot obtain the POIs that the users do not prefer. e missed check-ins are mixed of actually negative samples and undiscovered but potentially positive POIs. We cannot learn the user interests directly from this mixed data by utilizing a common approach, since they are unable to be differentiated from these two levels anymore. ird, users and POIs have different types of context information, e.g., social relationships of users and geographical information of POIs. It is important to exploit context information to improve recommendation accuracy. For example, in real life, we usually turn to our friends for POI recommendation. e social relationships among users help them locate the POIs that they are potentially interested in. In our real life, geographical information is another important type of context, which is not available in conventional recommendation task. As users are more likely to visit nearby POIs, the geographical information can help them locate the POIs that have neighborhood relationship with their preferred places.
How to overcome these challenges to further improve the recommendation performance has become one of the hot spots in recent studies. For example, Ye et al. [1] exploited the user preference and social and geographical influence for POI recommendation. In their work, a power law distribution is utilized to model the discovered spatial clustering phenomenon. Cheng et al. [2] captured the geographical influence by a multicenter Gaussian model and then fused it with matrix factorization (MF) and social influence for POI recommendation. Different from the work in [1,2], a kernel density estimation approach was proposed by Zhang and Chow [3] to model the users' visiting behaviors. To deal with the implicit check-in data, Lian et al. [4] first introduced the weighted matrix factorization (WMF) method as their basic framework, which fits the positive user activities by utilizing larger weights and unobserved user behaviors by utilizing smaller weights. en, an augmented factor model was proposed to incorporate the geographical influence. Instead of directly fitting zero check-ins, Li et al. [5] treated POI recommendation as an item ranking problem and learned the latent factors by optimizing an ordered weighted pair-wise classification criterion, which further considers the ranking task as a set of classification problems. Ying et al. [6] exploited the recommendation performance in another famous ranking criterion, i.e., the Bayesian personalized ranking (BPR) method [7], and proposed a pair-wise-based ranking method.
Although the above methods can work well in many recommendation scenarios, most of them only model the context information as regularization terms. e effectiveness of the deep information hidden in the network structure has not been fully exploited. On the other hand, deep learning-based embedding technique has shown its power in many recommendation tasks with its capability of extracting hierarchical representations from raw data [8]. For example, Wang et al. [9] applied the embedding model learned from the sequential behavior into next basket recommendation problem and proposed a two-layer hierarchical representation model. Zhao et al. [10] utilized the recurrent neural networks to learn the feature presentations of both users and products from an e-commerce website and proposed a feature-based matrix factorization method to leverage the learned features. However, most of these existing works mainly learn the embedding models of users and items from sequential behavior and content information, and little work has been done to leverage the representations deduced from social networks and location neighborhood relationships to deal with the cold-start POI recommendation. For example, Grover and Leskovec [11] proposed a novel method node2vec for learning continuous feature representations of nodes in networks. But their work did not investigate the recommendation problem, and the learned embeddings cannot be directly utilized to collaborative filtering method.
Based on the above observations, we propose a network embedding-based POI recommendation method, which consists of an embedding model and a collaborative filtering model. In the embedding model, a neural network, i.e., the node2vec model [11], is pretrained to learn the user and POI representations from the corresponding social network and a predefined location network, respectively. In the collaborative filtering model, to deal with the implicit check-in data, the ranking-based criterion is first introduced, which learns the user and POI factors by ranking the items correctly. en, the outputs of the node2vec are treated as the high-level representations of social and location context, and  2 Complexity they are further used as the priors of latent feature factors in the collaborative filtering model. In experiments, we evaluate our method in two real-world datasets [12], and the results indicate that our method is more effective than other related state-of-the-art recommendation methods. e main contributions of this paper are as follows: (i) We propose a network embedding-based POI recommendation method, which formulates the POI recommendation task as an item ranking problem, and considers the influence of social networks and geographical information by leveraging the capability of neural networks in extracting high-representative features from predefined networks (ii) We incorporate the learned embeddings as the priors of the latent feature factors in collaborative filtering model to enable our method can consider the network information and collaborative filtering information simultaneously (iii) We conduct extensive experiments on two realworld datasets to demonstrate the superiority of our proposed method We organize the remainder of this paper as follows: Section 2 reviews and discusses the related works to our study. Section 3 describes the POI recommendation problem and proposes a ranking-based recommendation framework with pretrained network embeddings. Section 4 shows the experimental results on two real-world datasets. Section 5 concludes this work and points out the future work.

Related Work
In this section, some works related to our studies are investigated, which include the context-based recommendation, implicit feedback-based recommendation, and network embedding-based recommendation.

POI Recommendation with Social and Geographical
Context. In LBSNs, social and geographical context plays an important role in helping users explore the unknown locations [12][13][14][15][16][17]. Motivated by the intuitions that users are more inclined to visit the places that their friends have suggested, and nearby locations tend to be visited by similar users, the social and geographical influence has been utilized to improve the POI recommendation. Cho et al. [18] studied the relation between human geographic movement, its temporal dynamics, and ties of the social network. ey found that people generally move periodically within a bound region but occasionally travel long distance due to the impact of the social relationships. Ye et al. [1] first modeled the spatial clustering phenomenon by a power law distribution.
en, a unified POI recommendation framework was proposed, which fuses user preference to a POI with geographical influence and social influence. Cheng et al. [2] first captured the geographical influence by utilizing the multicenter Gaussian model and then fused it with the social influence in a generalized matrix factorization framework.
Unlike previous proposed methods, which model the user check-in activities universally, Zhang et al. [19] proposed a personalized geographical location recommendation framework, where each user is modeled as a personal distribution. Li et al. [20] improved the POI recommendation by leveraging the information of friends in a two-step framework, where a set of potential locations that each user's friends have checked in were first learned, and then three types of check-ins were incorporated into the matrix factorization model. Liu et al. [21] proposed a general geographical probabilistic factor model to jointly learn both geographical preference and user interest preference, where a Poisson-based probabilistic generative process is utilized for the recommendation model. Wang et al. [22] constructed a graph to model users and their social relations, which models the user preference and social influence by utilizing different types of edges. To capture the asymmetry of geographical influence and the high variation of geographical influence across POIs, Wang et al. [23] exploited POIspecific geographical influence to improve POI recommendation and modeled the geographical influence between two POIs by using three factors: the geo-influence of POI, the geo-susceptibility of POI, and their physical distance. To incorporate the geographical context information, Ma et al. [24] proposed a novel autoencoder-based model to learn the complex user-POI relations. Guo et al. [25] explored the influence of geographical information from a location perspective and proposed a constrained ranking function by using a location regularization term. However, they did not consider the deep structure of social networks and POI relationships, and the influence of them is still largely unexplored.
Except the social and geographical information, other types of contexts have also been explored [8,[26][27][28][29], such as the temporal influence [30], the sequential influence, and the content influence [31,32]. For example, based on the temporal properties of user movement observed from a realworld LBSN dataset, Gao et al. [33] proposed a temporal aware recommendation framework. Qian et al. [34] focused on the joint effect of time and location for POI recommendation and proposed a spatiotemporal context-aware and translation-based recommender framework (STA) to model the third-order relationship among users, POIs, and spatiotemporal contexts for large-scale POI recommendation. In literature [28], they further studied three types of content information that is related to the user's check-in actions and then modeled them in a unified POI recommendation framework. In addition to social and geographical influence, Zhang et al. [35] also considered the sequential influence. In this work, they first mined sequential patterns from location sequences and represented the sequential patterns as a dynamic graph. en, they predicted the probability of a user visiting a location by additive Markov chain with this graph. Zhang and Chow [36] modeled three types of context, that is, geographical, social, and categorical correlations, for recommendation, and utilized the two-dimensional KDE for geographical modeling and the power-law distribution for social and categorical influence. Lian et al. [37] proposed a scalable implicit Complexity feedback-based content-aware collaborative filtering framework to incorporate semantic content and to steer clear of negative sampling. However, their work is developed for semantic information modeling, while ours is focused on exploring network embedding information. Yang et al. [38] proposed a neural network to jointly model the social networks and mobile trajectories, which considers four influential factors, namely, user visit preference, influence of friends, and short-term and long-term sequential contexts. But their method is developed for friend recommendation and next location recommendation tasks. Next location recommendation is different from the general POI/location recommendation problem, since it focuses on modeling the location's sequential information rather than user's historical preferences. From the above studies, we notice that these types of contexts are also important factors for POI recommendations and are often fused together with social and geographical context to make more accurate predictions.

Implicit Feedback-Based Recommendation.
In many recommendation scenarios, we can only get the user's positive behaviors through what they have purchased, visited, or clicked, but never know what they do not like. e feedback usually is not explicit, but implicit, which makes the recommendation task more challenging (the implicit feedback also is well known as the one-class collaborative filtering (OCCF) problem) [39,40]. Note that, although the OCCF problem we studied in this work has similar problem definition with positive unlabeled learning (PU learning) [41][42][43][44], most of the solutions proposed for PU learning cannot be directly applied to OCCF problem. e main reason is that most of them are focused on dealing with documents or reviews, where a lot of document features can be exploited to decide the true-negative samples initially. But in OCCF problem, we usually can only obtain the positive interactions among users and items, and it is hard to identify the true-negative items without any additional information. Most of the recommendation-oriented methods are trying to model the implicit feedback in a ranking or weighted way. For example, Pan et al. [45] modeled the implicit feedback by using the weighted low rank approximations; that is, they gave larger weights to the positive samples and smaller weights to the negative samples. But their work needs to directly model all the missed values as negative samples. Rendle et al. [7] proposed a pair-wise ranking-based criterion to directly optimize the loss error for ranking, and a generic ranking framework derived from a Bayesian analysis of the problem was presented. Lian et al. [4] extended the weighted matrix factorization method proposed in [45] for POI recommendation, where the latent feature factors of users and POIs are augmented by the activity area vector and the influence area vector, respectively. For the sake of improving efficiency and flexibility, Lian et al. [46] further extended the GeoMF [4] and proposed GeoMF++, which map the location influential areas into the same latent space as that formed by weighted regularized matrix factorization. Different from [4], user activity area is not used in GeoMF++ anymore, but can be recovered from user latent factors using negative sparse coding. Liu et al. [47] exploited the geographical characteristics from a location perspective, where the neighborhood is modeled at two levels, i.e., the instancelevel neighborhood that is defined by its nearest neighbors and the region-level neighborhood that is defined by the geographical region where the location exists. en, these two levels of geographical characteristics are incorporated to the weighted matrix factorization to learn the latent feature factors of users and POIs. Li et al. [5] investigated the POI recommendation by treating it as the ordered weighted pairwise classification problem and proposed a ranking-based geographical factorization model. Zhao et al. [48] first proposed the cogeographical influence to filter geographically noisy POIs and then incorporated it into a personalized pair-wise preference ranking matrix factorization model. Although ranking-based POI recommendations have been well studied, most of them do not consider incorporating these contexts by utilizing the embedding techniques and cannot be directly applied to the cold-start situations.

Embedding-Based Recommendation.
With the development of distributed representation techniques [49][50][51], embedding-based models have been proved in many recommendation and other related tasks [10,52,53]. For example, Liang et al. [54] proposed a content-aware collaborative music recommendation by first training a neural network on semantic tagging information and then used it as the prior in a collaborative filtering model. He et al. [55] proposed a scalable factorization model to incorporate visual signals into predictors of people's opinions, where the visual signals are learned from product images by using the pretrained neural networks. Wang et al. [56] further studied the problem of enhancing POI recommendation with visual contents, where they used the convolutional neural network to extract features from images and used it to guide the learning process of latent user and POI features. Deng et al. [57] employed the deep autoencoder technique to improve the quality of the initial vectors of MF for trust-aware social recommendations and to differentiate the community effect in users' trusted friendships. Except the above models that integrate deep learning with traditional recommendation models loosely, there are also tightly coupled models, that is, learn the parameters from these two components simultaneously. In this way, the learning process of latent factor model and feature extracting model mutually influence each other. For instance, Chen et al. [58] introduced an attentionbased neural network to collaborative filtering to address the implicit feedback problem in multimedia recommendation. Guo et al. [59] combined the factorization machine and deep learning for feature learning and proposed a novel neural network architecture. To deal with the inherent limitation of the bag-of-words model, Kim et al. [60] integrated the convolutional network into probabilistic matrix factorization model to capture the contextual information of documents and enhance the rating prediction accuracy. Guo et al. [61] incorporated a pretrained network embedding into the weighted matrix factorization method. However, their 4 Complexity method was proposed for social recommendation and the network embedding was integrated by a linear latent factor model, and ours is regarded as the prior knowledge of the latent feature factor.

Differences.
Our method has significant differences with these existing methods. First, although Yang et al. [38] adopted a network embedding method for the construction of social networks, it is proposed for friend recommendation and next location recommendation. Moreover, Yang et al. [38] did not consider the influence of geographical information. But in real world, nearby POIs are more likely to be visited by similar users, and the geographical relationship between POIs is also an important factor to the POI recommendation task. Hence, we further incorporate the POI-POI graph to enhance the representation of POI embeddings. Second, although there are existing works that formulate the POI recommendation as an item ranking task and try to optimize a BPR loss function, they did not consider the performance of leveraging network embeddings as user's and item's prior knowledge. e advantage of our method is that it can enhance POI recommendation by exploiting high-level embeddings from external network information and can make cold-start predictions for the users/items that have no training data in history by only using the learned distributional representations.

Our Approach
In this section, we first describe the POI recommendation task in LBSNs. en, the users' high-level representations and social-aware recommendation method are proposed. To exploit the influence of neighborhood relationships among locations, we then learn the POI representations from a predefined location network. Finally, by fusing the extracted embeddings of users and POIs with collaborative filtering model, the location-aware social POI recommendation method is achieved.

Task Definition.
Let us consider the POI recommendation problem in LBSNs, where users can express their preferences by checking in at different locations, and the closer the two locations are, the more likely they are to be visited by similar users. But different from traditional social networks, in LBSNs, only positive behaviors can be observed, that is, we can only know the places that the user has checked in, but never know the places that the user does not like. In other words, the check-in data are not explicit but implicit. In addition to these check-in activities, users can also express their preferences by making different friends, and the closer the relationship between two friends is, the more likely they are to be affected. is recommendation scenario is shown in Figure 2, which is associated with three central elements: the user check-in data (also known as user-POI check-in matrix), social network, and location network (i.e., POI-POI relation network). Figure 2 shows a toy example, where users express their visiting preferences by checking in at different places. But the check-in data are very sparse (some users are totally new to the system), only a small number of users' activities can be observed, and the remaining are unknown data (denoted as "?"). Besides the check-in matrix, users have also made some friends that have common interests with them to further explore the unknown places. e social relationship helps us make more accurate recommendations by leveraging the locations that their friends have visited. Moreover, in POI recommendation, neighboring locations tend to share common users, which means users are more inclined to visit nearby places.
is kind of context is also beneficial for our recommender system. Hence, how to make effective recommendations by utilizing the above contexts from the implicit check-in data is the main task of this work.

e Pair-Wise Item Ranking Criterion.
To deal with the implicit check-in data, we treat POI recommendation problem as the pair-wise item ranking task and introduce the Bayesian personalized ranking (BPR) criterion [7] as our basic recommendation framework. Instead of optimizing individual scores for a single item, BPR infers the latent feature factors by optimizing the partial ordering relationships between items, which is more suitable to the implicit feedback.
Suppose we have a check-in matrix R ⊆ U × I (as shown in Figure 3), where U denotes the user set and I denotes the POI set. e entry r ui � 1 in this matrix represents that user u has visited POI i in history, and r ui � ? represents that the status of u to i is unknown, which means i is unattractive or undiscovered to user u. We cannot directly treat all these unknown entries as negative samples, as they may be potential attractive POIs to users. To provide a personalized total ranking > u ⊂ I 2 of all POIs to the user, BPR method proposes a pair-wise data policy to reconstruct the user-POI check-in matrix, which assumes that compared with unvisited POIs, users will prefer the POIs that he/she has  Complexity visited. From Figure 3, we can find some examples of creating pair-wise preferences i > u j for user u from the checkin matrix, where the sign "+" represents that u prefers i over j and the sign "− " represents that user u prefers j over i (see the left part of Figure 3). For example, as user u 1 has checked in POI i 2 and unvisited POI i 4 , then according to our data policy the partial order between i 2 and i 4 can be created as i 2 > u i 4 (u prefer i 2 over i 4 ). But for the POIs that both have been checked in by u, we cannot deduce any preference for him/her (e.g., POI i 2 and i 3 for user u 1 ). e same is true for the POIs that both have been unvisited by the user (e.g., POI i 2 and i 3 for user u 2 ). en, we can formalize the created training data D R : U × I × I as where I + u represents the POI set that has been checked in by user u, I \ I + u denotes the unvisited POI set by user u, and (u, i, j) ∈ D R means user u prefers POI i over j. As the ranking with respect to u ( > u ) is antisymmetric, the negative samples are regarded implicitly.
To rank all the POIs correctly for every user, BPR maximizes the posterior probability of the model parameter from a Bayesian analysis of the problem. By using p(i > u j | Θ) as the likelihood function and p(Θ) as the prior function, we can arrive at the objective function of BPR: where Θ is the model parameter, σ(x) is the logistic sigmoid function, and λ Θ is the regularization parameter. r uij (Θ) captures the relationship among u, i, and j, which can be calculated by any real-valued score function of Θ. By decomposing the estimator r uij as r uij � w uij (r ui − r uj ), and delegating the task of modeling r uij to the matrix factorization (MF) method, the MF-based BPR method can be arrived at where V ∈ R d×n and U ∈ R d×m denote the latent factors of POIs and users, respectively. λ U and λ V denote the regularization parameters. r uv � U T u V v is the score function computed by the dot product of U and V. As in MF, we also place the zero-mean spherical Gaussian priors on the latent factors U and V: where N(x | μ, σ 2 ) denotes the Gaussian distribution with variance-covariance matrix σ 2 and mean μ.

Pretrained Social Network
Embedding. Inspired by the success of neural network-based embedding model in link predication and node classification, in this work we pretrain a network embedding model node2vec [11] (based on the skip-gram architecture) in an semisupervised task and use the learned embeddings as the representations of social networks (the project of node2vec is available at https://snap. stanford.edu/node2vec/). e training data we used come from the Gowalla dataset [12], which consists of 1.2 million check-ins (the details can be seen in Section 4.1). e social network we used is the friend relationships among 18,738 users, which can be represented as an unweighted and undirected graph. Let G � (U, E) be the given social network and f : U ⟶ R D s be the mapping function from nodes to the D s dimension feature representations, where U denotes the user set and E denotes the friend relationship set. For every source node u ∈ U, we define N S (u) ∈ U as a network neighborhood of node u generated through a neighborhood sampling strategy S, that is, a biased random walk method that can explore neighborhoods in a breadthfirst sampling as well as depth-first sampling fashion. To learn the high-level representations of every node, node2vec tries to maximize the log-probability of observing a network neighborhood N S (u) for a node u conditioned on its feature representation, given by f: To make the above optimization problem tractable, two standard assumptions are made in node2vec: (1) conditional independence, which assumes the observation likelihood of a neighborhood node is independent of the likelihood of observing any other neighborhood node given the feature representation of the source, and (2) symmetry in feature space, which means a source node and neighborhood node have a symmetric effect over each other in feature space.
POI POI POI POI POI User Figure 3: e reconstructing data policy from user-POI check-in matrix to pair-wise POI dataset. 6 Complexity With these two assumptions, the objective function of node2vec is arrived at where Z u � v∈U exp(f(u) · f(v)) denotes the per-node partition function. As it is expensive to compute for large networks, the negative sampling method [62] is used to approximate it. In their work, stochastic gradient ascent method is used to learn the model parameters.
In our work, the parameters of node2vec are set as follows: the dimension of the learned feature vectors is set as D s � 128, the length of walk per source is set as l � 80, the number of walks per source is set as r � 10, the context size is set as k � 10, and the optimization runs only a single epoch.
e best in-out and return parameters are achieved by using 10-fold cross-validation on tune dataset and are set as p � 1 and q � 2.

Social-Aware Recommendation.
e output of node2vec can be interpreted as the network representations of users. As users always express their preferences by making different friends, these representations denote how well a user is influenced by his friends. By introducing the network feature as the prior of the latent user factor in BPRMF model, the distribution of U can be modified as where S u ∈ R D s denotes the learned network embedding of user u (D s is the dimension of S u ) and X ∈ R d×D s is the weight matrix that transforms the learned social network representations from the neural networks into the collaborative filtering latent space (via XS u ).
To complete the inference, we also assume that the weight matrix X follows the zero-mean spherical Gaussian distribution: en, the posterior probability over the user, item, and weight latent factor vectors can be rewritten as e log-posterior probability of U, V, and X is derived as Keeping the parameters fixed, maximizing the logposterior over the latent feature vectors is equivalent to minimize the following objective function, that is, the objective function of our social-aware recommendation method: where ‖·‖ 2 F is the Frobenius norm, and λ U � 1/σ 2 U , λ V � 1/σ 2 V , and λ X � 1/σ 2 X are the regularization parameters of U, V, and X, respectively. Gradient descent-based algorithm is an obvious option to help find the local minimum of this objective function. And to alleviate the influence of huge training pairs (O(|U||I|)), the stochastic gradient descent method is utilized in our algorithm. e gradients of the corresponding latent factors are shown as follows: zSRank

Pretrained Location Network Embeddings.
To utilize the context of geographical information, we first deduce a POI-POI graph based on the intuition that nearby locations are more likely to be visited by similar users and then pretrain the node2vec model for learning the representations of POIs. e definition of the POI-POI graph is shown as follows. Definition 1 (POI-POI graph). e POI-POI graph is denoted as G V � (V, E V ), which captures the neighborhood relationships of POIs, where V is the POI set and E V is the edge set of POIs. Given a distance threshold ΔD (set as 0.2 km), for any two POIs v i ∈ V and v j ∈ V, if the distance (here, the distance is computed by the Haversine formula) d ij between them is smaller than ΔD (d ij < ΔD), there will be an edge e ij from v i to v j , and otherwise none.
According to the definition of POI-POI graph, the location network among POIs is thereafter formed. Location network captures the geographical characteristics of users' check-in activities. Intuitively, if two POIs v i and v j are geographically close, they will have a strong correlation in location network. When we know a user has checked in v i , we can conclude this user will also visit v j next with high confidence. In experiments, we obtain the location network of Gowalla with 25,517 nodes and 210,325 edges. e parameter settings of our pretrained node2vec model are as follows: D t � 128, l � 80, r � 10, k � 10, p � 1, and q � 0.5, where D t is the dimension of the learned network feature vector.

Location-Aware Social Recommendation.
e pretrained location network embeddings can be seen as the representations of POIs from geographical relations. Based on the assumption that nearby locations tend to be visited by similar users, that is, neighboring locations tend to have similar representations, we treat the learned embeddings as the prior of the latent POI factor in collaborative filtering model and modify the distribution of V as where T v ∈ R D t denotes the pretrained embedding of POI v from geographical relationships (D t is the dimension of T v ) and Y ∈ R d×D t is the weight matrix that transforms the learned POI representations from location network into the collaborative filtering latent space via YT v . As in SRank, the zero-mean spherical Gaussian distribution is placed on the weight matrix Y: en, the posterior probability of SRank can be further modified as By computing the log form of equation (19) and keeping the hyperparameters fixed, the objective function of our final recommendation method Geo-SRank is achieved: where λ Y � 1/σ 2 Y is the regularization parameter of Y. To find a local minimum of this objective function, gradient descentbased algorithm is utilized, and the latent factors can be updated by the following gradients: Algorithm 1 presents the training process of our Geo-SRank method. 8 Complexity

Computational Complexity.
e complexity of training the Geo-SRank method is mainly caused by computing the ranking loss (equation (20)) and the gradients of latent feature factors U, V, X, and Y. Assuming the average number of sampled triples is N, the average dimension of pretrained social network embedding is s, and the average dimension of pretrained location network embedding is t, the complexity of computing equation (20)

Applications in Cold-Start Situations.
e advantage of our feature-based recommendation method Geo-SRank is that it can be easily extended to the cold-start situations, i.e., there are no training data for the users/items in history. In the user cold-start situation, we can make predictions for users that no one has check-in records by using the social network representations r uv � (XS u ) T V v , where pure collaborative filtering models have trouble in learning the latent user factors. Here, the weight matrix X can be learned by known users and S u is the pretrained network embeddings. In the item cold-start situation, we can make predictions for coldstart items by using the location network representations r uv � U T u (YT v ), where the weight matrix Y is learned by known items and T v is the pretrained embeddings of items.

Experiments
In this section, several experiments on two real-world datasets are conducted to evaluate the recommendation performance of our feature-based method. To test the scalability of our method to the cold-start problem, we also conduct experiments on the situations with cold-start settings.

Datasets and Evaluation Metrics.
In our experiments, two real-world datasets Gowalla and Yelp [12] are exploited to evaluate our proposed method (Table 1). Gowalla was launched in 2007 and closed in 2012, which provides the location and social services that enable users to check in at places and make friends based on where they have visited.
e Gowalla dataset (https://snap.stanford.edu/data/locgowalla.html) we used was crawled by its API from February 2009 to October 2010. By filtering out those users with fewer than 15 check-in POIs and those POIs with fewer than 10 visitors, we can reach our final dataset, which contains 25,379 users, 32,623 POIs, 1,395,856 check-ins, and 118,717 social ties. Yelp is a local-search service powered by crowdsourced review forum, which publishes crowdsourced reviews about local businesses, as well as the online reservation service. Yelp was founded in 2004 and grew very quickly. By the end of 2017, Yelp has 148 million reviews. e dataset we used was collected by Liu et al. [12], which contains 30,887 users, 18,942 businesses, 561,746 check-ins, and 265,533 social ties.
To measure the performance of our recommendation approach, we exploit three popular measures Precision@k, Recall@k, and mean average precision (MAP) [12] as the evaluation metrics. In this Top-k item recommendation task, Precision@k measures how many relevant POIs can be predicted in the recommendation list. Here, the relevant POIs refer to the ones that have been labeled as positive in the dataset. Recall@k denotes how many relevant POIs can be predicted among the total relevant POIs in the dataset. e formal definitions of these two metrics are as follows: where M is the number of being tested users and L T (u) is the POI set that have been checked in by u in the test data. L k (u) represents the recommended Top-k POI list for user u (k is Require: the user-POI check-in matrix R, learning rate η, regularization parameters λ U , λ V i , λ V j , λ X , and λ Y , the pretrained network embeddings S and T; Ensure: the latent factors U, V, X, and Y; (1) e initialization of U, V, X, and Y; (2) while (L(t) − L(t − 1) > tolerate error) do (3) Randomly select one training example (u, i, j) from D R ; (4) r uij ⟵ r ui − r uj ; (5) Calculate U u by utilizing equation (23); (6) Calculate X by utilizing equation (21); (7) Calculate V i by utilizing equation (24); (8) Calculate V j by utilizing equation (25); (9) Calculate Y by utilizing equation (22); (10) Calculate L(t) (the loss error in t step) by utilizing equation (20); (11) end while (12) return U, V, X, and Y; ALGORITHM 1: e training process of Geo-SRank.
Complexity the size of this list). In this work, the list size k is set as 5. MAP measures the average precision (AP) over all users, which is computed by where Precision@i refers to the precision for Top-i recommendations.

Data Analysis.
To better understand the users' visiting behaviors in location-based social networks, we further investigate the social influence and geographical characteristics in Gowalla and Yelp and try to investigate the following questions: (1) Are users with friend relationships more likely to have similar check-in activities? (2) Are POIs with neighboring relationships more likely to be visited by the same set of users? To answer these two questions, we need to first define a method to measure the similarity between two users. Let A(u) denote the POI set that has been visited by user u in history. e similarity between user u and v can be computed as where |A(x)| represents the size of the POI set A(x) and 1/log(1 + |A(u)|) is the item that punishes the influence of popular POIs. For each user u, two similarities are computed, that is, (1) the average similarity between user u and his/her friends (x f (u)) and (2) the average similarity between user u and randomly selected users. For fairness, we choose the same number of users with their friend relationships. Figure 4 plots the kernel-smoothing density estimations based on the vectors x f and x r , from which we can observe that x r has smaller similarity values than x f . is result positively answers our first question, that is, users with friend relationships share more common interests than those without.
To answer the second question, the measurement of the similarity between two POIs needs to be predefined. By denoting B(i) as the user set that has checked in location i, the similarity between POI i and j is arrived at Based on this similarity, Figure 5 shows how the similarity of randomly chosen POI pairs varies with the increase of geographical distance, where we can find that, with the increase of the distance, the mean value of the POI similarity decreases in both Yelp and Gowalla data.
is result demonstrates the positive relationship between the POI similarity and geographical distance and provides us a positive answer to the second question: POIs with neighboring relationships are inclined to be visited by similar users.

Performance Comparison.
To evaluate the recommendation performance of our method, we conduct comparisons with several related recommendation methods. As the matrix factorization-(MF-) based method has been proved to have state-of-the-art performance, we mainly compare against the MF-based collaborative filtering models and the related ranking-based method (most of these methods are implemented in MyMediaLite [63]): (i) MostPopular: this is a basic recommendation method, which ranks the POIs according to how many users have visited them. (ii) WRMF [45]: this method is proposed for the oneclass collaborative filtering problem and provides a solution by giving small weights to negative samples and large weights to positive samples. (iii) GeoMF [4]: this is a state-of-the-art POI recommendation method, which extends the WRMF method by augmenting the user and POI latent factors. (iv) BPRMF [7]: this item recommendation method is proposed for implicit feedback, which learns the user interests from their pair-wise item preferences. is is the state-of-the-art POI recommendation method, which is only based on checkin matrix, without utilizing any context information.
(v) Rank-GeoFM [5]: this is a state-of-the-art POI recommendation method that develops a rankingbased matrix factorization model to learn the user's preferences and include the geographical influence simultaneously. (vi) IRenMF [47]: this is another state-of-the-art POI recommendation method that develops a weighted matrix factorization to model the geographical influence on POIs. (vii) SRank: this is our proposed social-aware POI ranking method, which incorporates the social network embeddings into the pair-wise ranking framework. Compared with Geo-SRank, the only difference is that it does not consider the location network information. (viii) Geo-SRank: this is our final recommendation method, which further extends the SRank method by utilizing the representations learned from a predefined location network. In experiments, we evaluate our method in both the noncold-start and cold-start situations, respectively. In the noncold-start situation, we randomly select 70% user-POI actions for training, 10% user-POI pairs for tuning, and 20% for testing. In the cold-start situation, we use the same dataset for testing and randomly select part of the training data for training (from 10% to 80%). To ensure the validity of these data, the data selection process is conducted five times independently. In these two situations, the regularization parameters of U, V, X, and Y are set as λ U � λ V i � 0.0025 and λ V j � λ X � λ Y � 0.00025. e dimensions of the latent factors U and V are set as d � 10.
e experimental results evaluated on Gowalla and Yelp with non-cold-start settings have been shown in Tables 2 and  3, from which we have the following observations: (1) Our Geo-SRank method achieves the best performance over all the baseline methods on both datasets. is demonstrates the superiority of the Geo-SRank method, that is, the effectiveness of our learned network embeddings and the usefulness of the social influence and the geographical Complexity information.
(2) Compared with basic recommendation methods (BPRMF and WRMF), the methods (i.e., GeoMF, Rank-GeoFM, IRenMF, and Geo-SRank) that consider the geographical information can significantly improve the POI recommendation performance. is result demonstrates the usefulness of the geographical information in helping users find the right POIs. (3) e method (i.e., SRank and Geo-SRank) that considers the social influence can perform better than BPRMF and WRMF, demonstrating the importance of the social influence. Geo-SRank can achieve the best performance over all the baseline methods, again, demonstrating the usefulness of both geographical information and social influence, and we can get a better result by considering them simultaneously. (4) MF-based methods can achieve a reasonable result with low-dimensional latent feature vectors, indicating the effectiveness of MF-based methods in the POI recommendation task. In experiments, we set the dimension of all the latent factors as 10. (5) From the results, we can also observe that the accuracy (i.e., Precision, Recall, and MAP) of all the POI recommendation methods is relatively low (less than 7%). One of the important reasons might be that the check-in data in LBSNs is very sparse (the density of check-in matrix is usually less than 0.5%), and thus, the recommendation methods suffer from the data scarcity problem. Second, compared with traditional movie rating or review data, the check-in data may not fully reflect the user's interests. For example, a user might check-in at a location where he/she is waiting for a friend or just passing by, which is not what he/she really likes.

Evaluation on Cold-Start Settings.
To evaluate the recommendation performance of our method in cold-start situations, we further conduct experiments with cold-start settings. In this experiment, we randomly select part of the training data for training. e training ratio is changed from 10% to 80%. In experiments, we compare our method with the baseline method BPRMF and the state-of-the-art POI recommendation method GeoMF. As BPRMF does not consider any other context information, it will use randomly generated feature values for new uses and items. But for GeoMF, as it not only exploits the basic user-item interactions to make predictions but also considers the geographical context, it will employ the location information to alleviate the cold-start problem. e experimental results with different training ratios are shown in Figure 6, from which we can find that, with the increase of the training data, the performance of these three methods both increases, which indicates that, with more training data, we can make our model more accurate. From this result, we can also find that our feature-based method can work well in these two datasets, which demonstrates that treating the learned network embeddings as prior knowledge of the latent factors is effective and can further improve the recommendations for those cold-start users. A larger margin between BPRMF and Geo-SRank can be found in Yelp than Gowalla, which is consistent with the results experimented on the non-coldstart data.

e Influence of the Recommendation List.
In POI recommendation, the system usually allows users to specify the POI numbers that shall be returned to her/him to avoid being overwhelmed by irrelevant POIs. Appropriate POI number helps to improve the user experience in recommender systems. erefore, we further investigate the performance of our method with varying the number of the recommended POIs. e Top-k recommendation result is shown in Figure 7, where only the Top-k most relevant POIs are considered. From this result, we can observe that, with the increase of the recommendation list, the hits of the relevant POIs are increasing, hence resulting in the increase of Recall@k and the decrease of Precision@k. e increase of Recall@k means the predictions can cover more relevant POIs, that is, the capability of the recommendation method to retrieve relevant POIs is increasing. e decrease of Precision@k represents the predictions can only cover less relevant POIs, i.e., the capability of the recommendation method to retrieve correct POIs is decreasing.

4.6.
e Influence of Latent Factor Dimensions. e dimension of the latent factors controls how much information can be used to represent users and items (In this experiment, we set U and V as the same dimension.). If the dimension is too small, the learned features will not be sufficient to represent the user and POI information. If the dimension is too large, not only will the learned features appear to be duplicated, but also the complexity of the algorithm will increase. Appropriate feature dimension plays a crucial role in our algorithm.    Complexity 13 Figure 8 shows the ranking performance of our method with the feature dimension changes from 10 to 80, from which we can observe that, with the feature dimension increases, the ranking performance of BPRMF and Geo-SRank also increases gradually. But when its value surpasses a certain threshold (such as 50), the increase begins to slow down, which indicates that when the dimension is big enough, there will be enough information for extracted features to represent the social and location networks. In our experiments, we find that, with very low dimension of these latent factors, the recommendation can achieve comparable results. To reduce the computational complexity of our method, in experiments both user and POI dimensions are set as d � 10.

Convergence Analysis.
To demonstrate the training efficiency of our feature-based recommendation method, we show the evaluation results with increasing training iterations in Figure 9. In this experiment, the learning rates of BPRMF and Geo-SRank are both set as 0.05. From this result, we can observe that, with the increase of the iterations, the Geo-SRank method can converge (due to the stochastic gradient descent algorithm we used, after convergence these two methods still fluctuate around a certain value) very fast in the Gowalla data, which almost converges within the same iterations with BPRMF (after about 20 iterations). But it converges a little slow in Yelp data (about 100 iterations), and BPRMF can also converge within 20 iterations. e reason of this result is that we introduce two external feature vectors in our method, and the increased random variables slow down the converge rate. From this result, we can also find our method can perform better than BPRMF throughout the whole learning process, which proved the effectiveness of these extracted network embeddings.

Conclusions and Future Work
In this work, we focused on how to make accurate POI recommendations in LBSNs and proposed a location-aware social recommendation method with pretrained network embeddings. To utilize the social relation and geographical information, we pretrained the network embedding method node2vec to learn the representations of users and POIs from the users' social network and the predefined location network, respectively. To incorporate the learned features into collaborative filtering model, we extended user and POI latent feature factors in a pair-wise ranking framework, where the learned embeddings are treated as their prior knowledge. Experimental results on two real-word datasets demonstrate the advantages of our method in the both noncold-start and cold-start settings. Although we only tested our method on Gowalla and Yelp, Geo-SRank can also be applied to other real-world settings. For example, when a  user planning a trip on Mafengwo (https://www.mafengwo. cn/) (a famous communication platform for travellers in China), he/she may want to ask his/her friends first and then pick out the POIs that have been visited by the users that have similar interests with him. Similarly, we can also help users to find out the restaurants that they want to have dinner with friends. In the above cases, both the social influence and geographical information help us make the right choices.
is work mainly investigates how to utilize the pretrained representations from social relation and location networks, but the check-in sequence is not considered. In real life, the users' check-in sequence is also an important aspect of their interest, for example, users always like to go to the playground and then go to the supermarket for shopping. In future, we plan to leverage the check-in sequence information to further enhance the POI recommendation performance.
Data Availability e data supporting this study are from previously reported studies and datasets, which have been cited. e processed data are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that they have no conflicts of interest.