A key issue to understand urban system is to characterize the activity dynamics in a city—when, where, what, and how activities happen in a city. To better understand the urban activity dynamics, city-wide and multiday activity participation sequence data, namely, activity chain as well as suitable spatiotemporal models, are needed. The commonly used household travel survey data in activity analysis suffers from limited sample size and temporal coverage. The emergence of large-scale spatiotemporal data in urban areas, such as mobile phone data, provides a new opportunity to infer urban activities and the underlying dynamics. However, the challenge is the absence of labeled activity information in mobile phone data. Consequently, how to fuse the useful information in household survey data and mobile phone data to build city-wide, multiday, and all-time activity chains becomes an important research question. Moreover, the multidimension structure of the activity data (e.g., location, start time, duration, type) makes the extraction of spatiotemporal activity patterns another difficult problem. In this study, the authors first introduce an activity chain inference model based on tensor decomposition to infer the missing activity labels in large-scale and multiday activity data, and then develop a spatiotemporal event clustering model based on DBSCAN, called STE-DBSCAN, to identify the spatiotemporal activity patterns. The proposed approaches achieved good accuracy and produced patterns with a high level of interpretability.
Cities, which are complex sociotechnical systems, are home to more than fifty percent of the world population [
To develop predictable activity models at the city level, three key challenges should be addressed: coverage, representativeness, and scale. Traditional approaches for urban activity modeling rely on household travel survey, which is known to be costly and have limited coverage. Some researchers also suggest that travel survey has recall bias as the respondents tend to forget details of activities other than major activities such as home and work activities [
Another challenge is how to model activity dynamics (spatiotemporal activity pattern) at the city level. Activity event (participation) data have three dimensions, which are location (including spatial information, such as longitude and latitude), time (including activity start time and duration), and characteristic (including activity type). The multidimension structure of the activity data makes the spatiotemporal activity pattern recognition a difficult problem. It requires building a model to combine multidimension information of the activity event data and calculate the proximity between activity events.
In this study, we focus on two tasks to analyze urban activity dynamics: (1) inferring the unobserved activity information from the mobile phone data; (2) based on inferred activity information, extracting the spatiotemporal patterns of the activities. Two models are proposed to tackle these tasks, namely, activity chain inference model and spatiotemporal activity pattern recognition model.
We develop a novel approach based on tensor-based collaborative filtering framework to infer large-scale individual-based activity chains by fusing mobile phone data and travel survey data. The proposed approach consists of two submodules: (1) a rule-based model to identify home and work activities, as home and work are highly regular activities and can be accurately inferred from multiday individual spatiotemporal trajectories; (2) a tensor-based collaborative filtering (TCF) model to fuse information from travel survey data and mobile phone data and infer noncommuting activities, which makes it possible to fully utilize the hidden information in the mobile phone data.
We develop a spatiotemporal event clustering algorithm based on DBSCAN (STE-DBSCAN) to solve the activity clustering problem. By defining the distance between activity events (
Our contributions can be summarized as follows: A new method for understanding urban activity dynamics using big data: with good coverage, representativeness, and scale of mobile phone data, we can find more plentiful and complicated patterns of urban activity. A new data fusion approach for city-wide activity information gathering, which can utilize the accurate ground truth activity information from travel survey together with the ubiquitous spatiotemporal information in mobile phone data. A flexible solution for spatiotemporal activity pattern recognition, which can sensitively balance the effects of time and space.
The following sections review the related works, introduce the data and the urban context feature extraction, describe the methodologies of the paper, present the experiment results, and conclude this work.
There are many works in the literature using data fusion on activity inference. Chen et al. [
Collaborative filtering is a data mining framework which is widely used to infer useful information from multisource partially observed data. Most of the work on collaborative filtering focused on the traditional problems involving only label and data features [
Many studies also have been done on this topic. Kumar et al. [
Two datasets from Shanghai are used in this study, which are travel survey data and mobile phone data (detailed statistics are shown in Table
Description of two datasets.
Data source | Data size | Period | Useful information |
---|---|---|---|
Travel survey data | 60,032 households | One day during Sep. 18–25, 2009 | Activity chains with time and location details |
Mobile phone data | 3,094,297 users | May 01, 2015–May 31, 2015 | Spatiotemporal trajectories |
One mobile phone user has 432 records per day.
In addition, we extracted a set of urban context features for activity inference, including time, geolocation (latitude and longitude), point-of-interest (POI), and road network information. POI data includes the location information of special points in the map, such as school, shop, and gas station. The POI data is collected via API of Baidu Map in 2012. The POI data is used to model the urban function of a region, and the road network data is used to model the accessibility of a region. A virtual grid reference is constructed by dividing the map into square cells of size 500 × 500 meters (considering that the positioning accuracy of base station in the mobile phone data is about 200 meters in downtown). The POI and road network information within each cell are extracted. The POI dataset contains 486,615 POIs and are divided into 8 categories, which are (1) schools, (2) companies, offices, banks, and ATMs, (3) mails and shopping malls, (4) restaurants, (5) gas stations, vehicle service locations, parking areas, and transportation facilities, (6) residences, (7) entertainment and living services, and (8) hotels, as suggested by [
POI distribution (a) and hierarchical road distribution (b) (red: freeway; blue: major road; green: local road).
The activity chain inference model contains three steps: (1) extracting the spatiotemporal trajectories to form a travel-stay chain for each mobile phone user; (2) identifying the home and work activities; (3) inferring noncommuting activities using a tensor-based collaborative filtering approach that fuses information from both travel survey and mobile phone data. Once the activity information is inferred, we build an activity spatiotemporal pattern recognition model to extract the spatiotemporal activity patterns at city level through clustering.
Due to the noise in signal, base station positioning results may jump at several nearby base stations. This situation is called ping-pong phenomenon or ping-pong handover [
STCS (shown in Figure
Illustration of STCS method handling small-scale ping-pong phenomenon.
Cumulative frequency of activity duration.
As home and work are highly regular activities (in general cases, residents usually stay home at night and work during daytime) and mobile phone data is long-term observation data (one month in this paper), we use a rule-based model to identify the types of these regular activities. Special cases like working all night are relatively rare, and transportation planning studies are more interested in the daytime activities. Residents’ home and work activity patterns in the travel survey data are illustrated in Figure
Cumulative frequency of home (a) and work duration (b) on weekday.
As mentioned before, our datasets are multidimensional (including spatial, temporal, and characteristic information), sparse (there is a lot of missing information needed to be inferred in mobile phone data), and partial (travel survey data has limited samples and recall bias), and we need a unified model which can take full use of the multisource data. Multidimensional data is often referred to as a tensor, and tensor decomposition is a standard technique to capture the multidimensional structural dependencies. By decomposing the partially observed tensor, the missing data in the tensor can be replaced using the product of the decomposed results. And because of the good performance on handling multiway, sparse, and partial data [
The user-activity-context tensor in this study is constructed using both travel survey data and mobile phone data (illustrated in Figure
Construction of user-activity-context tensor.
Tucker decomposition and CANDECOMP/PARAFAC (CP) are the two most popular methods to decompose a tensor. The Tucker decomposition of a 3D tensor is illustrated in Figure
Illumination of three-way Tucker decomposition.
The Tucker decomposition can be viewed as a generalized form of principal component analysis (PCA) or matrix factorization (MF). The Tucker decomposition of a 3D tensor can be illustrated as follows:
The tensor decomposition problem can be formulated as a least square problem (
There are several existing algorithms for solving the tensor decomposition problem [
Step 0: Initialize Step 1: From Step 2: From Step 3: From Step 4: If convergence occurs, calculate the core
Here, matrix
The notation
The above algorithm is applicable for fully filled tensor, but our case involves a lot of missing values. To handle the missing data, we fit the decomposed tensor only for the nonmissing data and use a new loss function:
Many studies solve the above least square problem using the stochastic gradient descent (SGD) [
The EM-based solution algorithm can be found in Algorithm
Step 0: Let Step 1: Decompose tensor And Step 2: Let Step 3: Convergence check: If
Key issues of spatiotemporal activity pattern recognition are how to measure the proximity between activity events (namely,
We introduce
Finally, STE-DBSCAN (shown in Algorithm
Step 0: Get Step 1: Set values of Step 2: Find and delete all noise. Step 3: Find all core point set Step 4: set Step 5: find the core point
We have conducted a series of experiments to test the proposed model named tensor-based collaborative filtering (TCF) model. In the upper part of the user-activity-context tensor, not all the survey data can be used and we make strict rules: (1) the activity chain of user has at least two activity categories; (2) the activity chain covers more than 12 hours; (3) Each activity has location information (longitude and latitude). Finally, we get 1634 activity chains from survey data. The activity inference model is implemented using the N-way Toolbox in MATLAB [
Different groups of input features have been tested before being selected as the final feature set, and different values of core tensor dimensions (
As ground truth activity labels of the mobile phone data are unknown to us, the activity labels in the travel survey data are randomly erased and used as the test set for validation. In our model, mobile phone data and travel survey data are fused together, and the label prediction problem is transferred to missing information imputation problem. We designed two test scenarios to evaluate the proposed framework.
Scenario 1 only uses fine grained travel survey data to construct the tensor. This scenario is designed to test the explanatory and predictive power of the model under different missing rates (different missing rates here mean different missing ratios of the known noncommuting activity labels in the travel survey data). We also compare our results with other machine learning methods which are used to infer activity types in other research works.
Scenario 2 combines different sizes of mobile phone data as well as travel survey data in the tensor. The sizes are tested as the multiples of the size of travel survey data. This scenario is designed to test the robustness of the model with different amount of unlabeled activity data.
Results of scenario 1 are presented in Figure
Performance of the model (
Confusion matrices of TCF results.
Actual\predicted | Shopping | Entertainment | Business | Other |
---|---|---|---|---|
Missing rate = 50% | ||||
Shopping | 1216 (0.771) | 57 (0.036) | 55 (0.035) | 249 (0.158) |
Entertainment | 179 (0.110) | 1203 (0.741) | 68 (0.042) | 173 (0.107) |
Business | 50 (0.094) | 24 (0.045) | 397 (0.749) | 59 (0.111) |
Other | 113 (0.079) | 50 (0.035) | 12 (0.008) | 1264 (0.878) |
Overall accuracy | 0.814 | |||
Missing rate = 80% | ||||
Shopping | 1375 (0.545) | 409 (0.162) | 28 (0.011) | 711 (0.282) |
Entertainment | 467 (0.180) | 1428 (0.550) | 1 (0.0004) | 700 (0.270) |
Business | 77 (0.091) | 81 (0.096) | 23 (0.027) | 667 (0.786) |
Other | 339 (0.099) | 330 (0.096) | 14 (0.004) | 2739 (0.800) |
Overall accuracy | 0.593 | |||
Overall accuracy after regarding business as other | 0.665 |
The values in parentheses are proportions.
Benchmark test result.
Model | Overall accuracy | F1-score | |||
---|---|---|---|---|---|
Shopping | Entertainment | Business | Other | ||
TCF (missing rate: 50%) | 0.814 | 0.776 | 0.814 | 0.748 | 0.857 |
Naïve Bayes | 0.405 | 0.302 | 0.142 | 0.206 | 0.552 |
SVM | 0.425 | 0.388 | 0.327 | 0.091 | 0.522 |
Decision tree (DT) | 0.731 | 0.691 | 0.729 | 0.741 | 0.762 |
Random forest (RF) | 0.773 | 0.749 | 0.765 | 0.790 | 0.795 |
We also compare our results with other machine learning methods (see Table
Confusion matrices of different models. (a) TCF (50% missing rate). (b) Gaussian Naïve Bayes. (c) SVM. (d) Random forest.
The test results of scenario 2 are presented in Figure
Robust performance of the model (
After inferring all activity labels over the city, we can mine the spatiotemporal activity patterns using STE-DBSCAN. We randomly select 100,000 inferred activity records derived from mobile phone signaling data on May 5, 2015 (Tuesday), in Shanghai to verify the performance of our spatiotemporal activity pattern recognition algorithm. To adjust the clustering results on time and space (we want to clearly identify different activity start time and duration from the clustering results, but we also want to make the activities happening at close places easily fall into the same cluster), we make the model more sensitive to time and less sensitive to space by emphasizing the time factors
Parameter selection result.
Number of clusters | |||||
---|---|---|---|---|---|
0.05 | 0.06 | 0.07 | 0.08 | 0.09 | |
218 | 115 | 71 | 59 | 36 | |
242 | 246 | 178 | 124 | 90 | |
316 | 204 | 169 | 152 | ||
262 | 342 | 403 | 432 | 199 | |
229 | 327 | 359 | 422 | 421 |
The
Cluster number distribution related to different activity types.
Considering spatiotemporal information and activity type, these 410 clusters depict the urban activity patterns in fine-granularity of time and space. The results also illustrate the complexity of the urban (activity) system. And urban dynamics of human activity can be modeled by the patterns with different space and time properties. In this section, because of the space limitations, we only pick partial patterns in the downtown of Shanghai to show the performance of our methods. Work is the most important activity in human life (some work-related patterns are shown in Figure
Sample activity pattern related to work. Size of the points represents the relative activity frequency.
Sample activity pattern related to shopping. Size of the points represents the relative activity frequency.
Sample activity pattern related to entertainment. Size of the points represents the relative activity frequency.
In this paper, we propose two models to analyze urban activity dynamics. In the first model, we infer multiday and all-time activity chains using large-scale heterogeneous and sparse data sources from urban areas. The proposed model overcomes the limitations of existing studies and utilizes the accurate ground truth activity information from travel survey together with the ubiquitous spatiotemporal information in mobile phone data. The activity labels of individual mobile phone records can be automatically annotated, which can lead to important applications in urban transportation planning and activity/location recommendation systems as well as targeted advertising systems. This approach can also be used to infer complete activity chains from not only mobile phone data and survey data (which we use here) but also any other type of massive partial and highly heterogeneous datasets. The dataset considered in this work is from Shanghai, China, but this method is transferable to other urban areas with similar data. The results also suggest that this approach is powerful and robust in handling missing values. The second model proposes the STE-DBSCAN algorithm, which can automatically capture detailed spatiotemporal activity patterns. The results can tell us when, where, and what activities happen in our city, and different spatiotemporal activity distributions are found at different times of the day. The discovered activity patterns also confirm the reasonability of the activity inference. Our work can be used to understand urban activity dynamics at a deeper level, which is important for urban planning, disease control, location/activity recommendation services, etc.
Future research can explore parallelized solution approaches for the proposed models to more efficiently handle huge volume of data. Better model performance can be achieved if more informative features are introduced, such as detailed urban land use and high-precision survey data captured by GPS devices. The results will be more reasonable if we can use datasets at the same time period, as we made an assumption that the way people travel did not change too much from 2009 to 2015. In addition, it is interesting if we can model how the way people travel changes with the city development. The activity spatiotemporal patterns can be used as an input to future activity prediction problem, which has wide applications in location/activity recommendation, targeted advertising, etc.
Some basic definitions for DBSCAN are provided below [ The Eps-neighborhood of a point A point A point Two points The border point is density-reachable from another core point but is not core point itself. The points which are not density-reachable from any other point are noise.
The mobile phone data and household travel survey data used to support the findings of this study have not been made available because we do not have the right to share them with the public. However, we can provide part of encrypted data. The POI data and the road network data used to support the findings of this study are available from the corresponding author upon request.
The authors declare that there are no conflicts of interest regarding the publication of this paper.
This research was sponsored by National Natural Science Foundation of China (71171147).