^{1}

^{1,2}

^{1,2}

^{1}

^{2}

The recent decade has witnessed an increasing popularity of recommendation systems, which help users acquire relevant knowledge, commodities, and services from an overwhelming information ocean on the Internet. Latent Dirichlet Allocation (LDA), originally presented as a graphical model for text topic discovery, now has found its application in many other disciplines. In this paper, we propose an LDA-inspired probabilistic recommendation method by taking the user-item collecting behavior as a two-step process: every user first becomes a member of one latent user-group at a certain probability and each user-group will then collect various items with different probabilities. Gibbs sampling is employed to approximate all the probabilities in the two-step process. The experiment results on three real-world data sets MovieLens, Netflix, and Last.fm show that our method exhibits a competitive performance on precision, coverage, and diversity in comparison with the other four typical recommendation methods. Moreover, we present an approximate strategy to reduce the computing complexity of our method with a slight degradation of the performance.

The advent of Internet has confronted us with an exploding information era. We find that it is very difficult to select the relevant ones from countless candidates on the e-commerce websites. As an automatic way to help people make right decisions under the information overload, the recommendation system has become a significant issue for both academic and industrial communities.

During the last decade, lots of recommendation methods have been proposed, including collaborative filtering methods [

Latent Dirichlet Allocation (LDA) was first presented as a graphical model for text topic discovery by Blei et al. in 2003 [

In this paper, we propose a new content-unaware probabilistic recommendation method inspired by LDA model. Users’ collecting behaviors are probabilistic events, in which one user belongs to multiple user-groups and users in each user-group have different collecting preferences. In our method, the collecting process is regarded as two joined probabilistic processes intermediated by the user-group; that is, every user is a member of one latent user-group at a certain probability, while each user-group will collect various items with different probabilities.

Calculating the probabilities on the entire data set is time-consuming and space-consuming. In order to reduce the computing complexity of our method, we introduce an approximate strategy with a slight degradation of the performance, which samples a part of the data set to build a rough probabilistic recommendation model.

Many products on an e-commerce website are not popular; that is, the sale of every single product lies in the tail of sale curve, but the sales of all these unpopular products constitute a big portion of the whole income. That is the so-called long tail phenomenon. Therefore, a good recommender system must focus on both the accuracy and the diversity. The experiment results on three real-world data sets, MovieLens, Netflix, and Last.fm, show that our method exhibits a competitive performance not only on the precision and the coverage but also on the diversity.

People have different and multiple inner attributes, including physiological characteristics, preferences, taboos, and religious beliefs. These attributes can be clustered into lots of user-groups which can represent users with similar attributes. Actually, a user does not belong to only one user-group. For example, user

the users’ collecting behaviors are probabilistic events;

one user belongs to multiple user-groups and users in the same user-group have similar collecting preference.

The collecting action of users on items is therefore considered as a two-step probabilistic process; that is, users are observed as members of several latent user-groups and users in user-groups will collect items based on the group-item probability distributions. Here we assume that

As long as

Considering that, the Latent Dirichlet Allocation (LDA) is a probabilistic model that uses a latent topic to bridge documents and words. Using the latent topic, LDA constructs the documents via two probabilistic processes that chooses a topic after the first probability prediction and then collects words from the attributes of the topic-word according to the topic. Inspired by the LDA, the structure of our recommendation model is designed as a three-layer structure of Bayesian, that is, the user layer, followed by the user-group layer and the item layer. To construct it, parameters are used in pairs. The recommendation model is determined by the hyper parameters

Graphical model representation of our method.

In our method, the probability that a user

There are many approximate inference strategies to estimate parameters

Here, we use the posterior distribution

When the Markov chain is near the target distribution after adequate iterations, we recorded its current values of the implicit variable

Actually, the data set is updated every day. It is not only time-consuming but also space-consuming to use the entire data set to structure the recommendation model. To save time and space, we prefer to model with less data and the recommended items are only listed when required instead of preparing them in advance. In this paper, we present an approximation method to structure the approximate model of the probabilistic recommendation.

In the approximation method, we sample part of the users’ collection data from the data set to structure an imprecise probabilistic recommendation model. The imprecise model will serve as a guide to create a recommendation list from two sides. On one hand, the latent user-group vector

The processes of the approximate method are described as follows.

Choose part of users from the data set for constructing the approximate model, called approximating data.

Use the approximating data to initiate the Markov chain: random user-group

Use (

When user

Use (

Sample once in a certain number of times

By ranking the product of

The time consumption of the approximate model depends on the size of approximating data. Indeed, the performance of the approximate model depends on the size as well. In the experiment, we use different percentages of data as approximation data to find the optimal size. Approximate model is, however, imprecise owing to the use of data locality. Meanwhile, the performance oscillates when different data is chosen to do approximate modeling. Different strategies (random: randomly choose users within the entire data.; item degree: according to each user’s average degree of items sampled proportionally; user degree: according to the average degree of user sampled proportionally; quick classification: use a quick classification method to classify the users and then sample proportionally) are compared to find out the user distribution offered by which strategy is most similar to that of the entire data and has the same tendency of performance. In the experiment, we use the average value and the upper bound value to represent the performance of the approximate model.

Three benchmark data sets (Table

The basic statistical features of the three data sets.

Data set | Users | Items | Links | Sparsity |
---|---|---|---|---|

MovieLens | 6,040 | 3,952 | 1,000,209 | 4.19 × 10^{−2} |

Netflix | 10,000 | 6,000 | 701,947 | 1.17 × 10^{−2} |

Last.fm | 1,882 | 17,632 | 186,480 | 5.62 × 10^{−3} |

Three evaluation metrics were used to assess the recommendation’s effect in the experiment: precision, coverage, and diversity. Precision is a basic evaluating metric. It is defined as the proportion of users that accept the recommended items:

Different algorithm will provide different recommendation list to users. The union set of recommendation lists

For comparison, we present the results of the four recommendation methods which are the probabilistic spreading (ProbS), heat spreading (HeatS), user-based collaborative filtering (UserCF), and the association rule algorithm (ARule). User-based collaborative filtering algorithm is one of the most classic collaborative filtering methods. Based on the similarity of purchased items between users, it recommends the items that similar users have bought but not yet bought by the user himself. The association rule method is also widely used in recommender systems. This method concentrates on the latent relationship between items. To find these relationships, every user’s item list is analyzed to create a list of the most related items called association rule. Heat spreading method, a variant of probabilistic spreading method, has the highest rate of coverage and diversity in current recommendation algorithms, but it ignores accuracy. In the experiment, we use accuracy of recommendation as the lower bound of precision and use its coverage and diversity rate as the upper bound. Therefore, we use the enhance metrics to evaluate the performance, as shown in (

To evaluate the performance of the approximate model, missing rate

The recommendation performances of different methods on the MovieLens, Netflix, and Last.fm data sets are shown in Figures

The recommendation performances on MovieLens; our method uses experiential constant parameters:

The recommendation performances on Netflix; our method uses experiential constant parameters:

The recommendation performances on Last.fm; our method uses experiential constant parameters:

The performances of our method are far better than those of the other methods on MovieLens data set which has the most links in the experiment, with ProbS running a close second, while both UserCF and ARule performed significantly worse. When the length of recommendation is lower than 20, the performance of our method is at least twice as well as the other two methods. On the Netflix data set, our method consistently performs very well in terms of precision, coverage, and diversity. The precision of ProbS goes near to that of our method while its other performances are much worse. In addition, our method gets good comprehensive performance on Last.fm which is the sparsest data set in the experiment. When the recommendation list length is over 50, the precision of our method is lower than that of ARule, and coverage runs a close second. Furthermore, the consistency of its diversity could be rated as the best. The performances of the approximate model are shown in Figures

The performances of approximate model.

The missing rates and comprehensive rates on MovieLens and Netflix in which

Different metrics are drawn on different maps to show their tendency of coverage, as shown in Figure

Figure

In this paper, we proposed a method which makes use of users’ behaviors to give recommendation. Instead of modeling with tags or contexts, our method takes the collecting lists to construct a recommendation model without the contents of items. As shown in the experiment, our method exhibits an all-round competitive performance on precision, coverage, and diversity, in comparison with four typical classes of recommendation algorithms. To reduce the computing complexity of our method, approximate model is also proposed in this paper, where the adjusting parameters are the determinant of performance curve of approximate model. As shown in the experiment, the approximate method is feasible since the optimal value is under 20%. When precision is considered to be the most important metric, it is more appropriate to use less than 10% of the data to construct the approximate model.

The authors declare that there is no conflict of interests regarding the publication of this paper.

This work was supported by the National Natural Science Foundation of China (nos. 61300018 and 61103109), Research Fund for the Doctoral Program of Higher Education of China (no. 20120185120017), China Postdoctoral Science Foundation (nos. 2013M531951 and 2014T70860), Fundamental Research Funds for the Central Universities (no. ZYGX2012J071), and Special Project of Sichuan Youth Science and Technology Innovation Research Team (no. 2013TD0006).