Mobile Information Systems
Volume 2016 (2016), Article ID 2316757, 12 pages
http://dx.doi.org/10.1155/2016/2316757

Research Article

Energy-Efficient Real-Time Human Activity Recognition on Smart Mobile Devices

Jin Lee and Jungsun Kim

Department of Computer Science and Engineering, Hanyang University, Ansan, Gyeonggi-Do 15588, Republic of Korea

Received 31 December 2015; Accepted 30 May 2016

Academic Editor: Wenyao Xu

Copyright © 2016 Jin Lee and Jungsun Kim. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Nowadays, human activity recognition (HAR) plays an important role in wellness-care and context-aware systems. Human activities can be recognized in real-time by using sensory data collected from various sensors built in smart mobile devices. Recent studies have focused on HAR that is solely based on triaxial accelerometers, which is the most energy-efficient approach. However, such HAR approaches are still energy-inefficient because the accelerometer is required to run without stopping so that the physical activity of a user can be recognized in real-time. In this paper, we propose a novel approach for HAR process that controls the activity recognition duration for energy-efficient HAR. We investigated the impact of varying the acceleration-sampling frequency and window size for HAR by using the variable activity recognition duration (VARD) strategy. We implemented our approach by using an Android platform and evaluated its performance in terms of energy efficiency and accuracy. The experimental results showed that our approach reduced energy consumption by a minimum of about 44.23% and maximum of about 78.85% compared to conventional HAR without sacrificing accuracy.

1. Introduction

Interest in u-health and wellness-care has recently been growing [1–3]. Various technologies that recognize the physical activities of users using various embedded sensors in smart mobile devices are actively studied. Recognized physical human activities can be used to develop applications that predict a falling accident or measure calorie consumption [4–7]. Such applications mainly use the triaxial accelerometer because it consumes the least power compared to other available sensors [8, 9]. Therefore, the use of “sensor” hereafter in this paper refers to a triaxial accelerometer.

These applications need the accelerometer to operate continuously without stopping in order to recognize different physical human activities in real-time. Unfortunately, this incurs unnecessary power consumption by the sensor and computational overhead; it is regarded as a big problem considering the limited power resources of smart mobile devices [10–12]. For example, while the battery life of LG Optimus Pro reaches up to over 60 hours when all applications and sensors are turned off, it decreases to 22 hours when a human activity recognition (HAR) application is activated with a sensor (100 Hz).

One facile solution is to blindly limit the usage of the accelerometer, but this may cause another problem of sacrificing the accuracy of human activity recognition. Another solution is to adopt a lower acceleration-sampling frequency (SF) for the sensor, but this may result in the loss of important sampling data. For this reason, previous studies have mostly focused on achieving a rather suboptimal balance between energy efficiency and HAR accuracy, instead of seeking optimal power consumption without sacrificing the HAR accuracy [13–16]. An analysis of the previous studies showed that they required the accelerometer to be operating at all times; as a result, the power consumption due to the continuous operation of the sensor itself and the accompanying data processing by the CPU remain unaddressed. In this paper, we argue that it is possible to save energy to great extent without continuous sensor operation.

In order to further improve the energy efficiency, we propose an approach that dynamically controls the variable activity recognition duration (VARD) for HAR. Our approach classifies a user’s activities as dynamic or static and controls the classification duration and sleep time for the HAR process based on two factors: the acceleration-sampling frequency and window size (WS). We performed experiments and conducted a thorough analysis of the result to show that the proposed VARD strategy performs well in terms of both energy efficiency and HAR accuracy.

The remainder of the paper is organized as follows. Section 2 presents an analysis of previous HAR approaches for efficient power consumption. Section 3 describes our initial motivations and a basic HAR system. Section 4 presents the impact of varying the SF, WS, and feature vector dimensionality (FVD) on the classification accuracy and the power consumption. Section 5 explains the VARD strategy. Section 6 reports on the evaluation results for our approach. Finally, Section 7 concludes with a summary and future directions.

2. Related Works

In this section, we first present a variety of accelerometer-based HAR technologies and then discuss relevant previous studies.

2.1. Human Activity Recognition Using Accelerometer

Early-stage researchers investigated the wearable sensor-based HAR; they demonstrated that the usage of wearable sensors can provide elevated accuracy in the area of HAR [17–20]. Recent wearable sensor-based HAR has been enhanced by some previous work. Hong et al. [21] presented a personalized HAR system using Bayesian network and support vector machine (SVM).

Due to the rapid advancement of smart mobile devices technology, many researchers focused on the mobile device-based HAR. Their work [4, 22–24] also turned out to be successful in providing high recognition rate. Torres-Huitzil and Nuno-Maganda [25] showed a position-independent HAR system using time-domain features and neural network. Vo et al. [13] presented a personalized HAR system through SVM, along with a -medoids clustering method. Albert et al. [2] studied a HAR system for Parkinson’s patients.

Smart mobile devices are promising platform for HAR because they not only are equipped with embedded built-in sensors but also are a natural part of everyday human daily life [26]. However, smart mobile device needs energy management due to its limited resources.

2.2. Human Activity Recognition with the Energy-Saving

A naive solution to reducing the power consumption of mobile devices is to limit the usage of the accelerometer. However, such an approach may negatively affect the HAR accuracy and therefore should be applied with caution.

Vo et al. [13] aimed to reduce the power consumption of the accelerometer and CPU by improving the HAR algorithm. Their approach relied on a SVM and time-domain features and reduced the power consumption by about 6.7% when compared to a conventional approach adopting SVM and fast Fourier transform (FFT). However, they focused more on the HAR accuracy than on reducing the power consumption.

Vo et al. [14] and Yan et al. [15] improved the power consumption efficiency by changing the SFs of the accelerometer and classification features. The key concept was identifying the best combination of SF and classification feature for a specific activity. Their approach reduced the power consumption by about 20%–25% compared to previous approaches. However, their approach also requires continuous operation of the triaxial accelerometer when the application is running.

Liang et al. [16] reduced the power consumption of HAR by using lower SFs. They proposed a hierarchical recognition algorithm that uses time-domain features, frequency-domain features, and similarity measurements. Their algorithm applies a decision tree instead of SVM. In their results, the battery life was extended by 3.2 h. However, because this algorithm tried to use a lower SF, the HAR accuracy was at best over 85%, which is less than that of other studies [13–15].

In this paper, we propose a new approach for HAR process that reflects the physical states of the mobile user. Our approach can secure a similar or higher HAR accuracy compared to previous approaches while providing better energy efficiency.

3. Human Activity Recognition on Smart Mobile Devices

In this study, our aim was to develop a lightweight HAR approach that uses the embedded accelerometer in smart mobile devices. To build a mobile HAR system on smart mobile devices, methods for sensor monitoring and real-time detection of user activity need to be considered, as depicted in Figure 1.

Figure 1: Human activity recognition using a single accelerometer.

Typical HAR can simply be defined as the process of interpreting raw sensor data to classify a set of physical human activities [27]. Statistical machine learning techniques are used to infer information about the activities from raw sensor readings; this process usually includes a training phase and predicting phase. The training phase requires collecting labeled data to learn the model parameters and build a training model from the collection. The predicting phase uses the training model to classify physical activities of users in the following sequence: preprocessing, segmentation, feature extraction, and classification. The following subsections explain the details of the proposed HAR process.

3.1. Collecting Acceleration

Physical human activities consist of basic movements such as walking, sitting, standing, and running. We selected the six most common activities as target activities, which have been recognized in previous works [8, 13–16, 19, 24]. Table 1 presents the target activities for our study.

Table 1: Classification of target activities.

Activity type
Activity

Static
Standing and sitting

Dynamic
Walking, running, ascending stairs, and descending stairs

We collected data from the triaxial accelerometer (MPU-6050; maximum range: 39.227 m/s²; resolution: 0.001 m/s²) on the LG Optimus Pro (Android Kitkat 4.4.2 OS) of two male subjects who are 28 and 32 years old, respectively. A smart mobile device was placed inside the back pocket of the pants of a subject.

With the Android operating system, four different SFs (NORMAL: 5 Hz, UI: 16 Hz, GAME: 50 Hz, and FASTEST) can be selected for the accelerometer. The FASTEST SF depends on the computational workload of each specific mobile device and thus can differ from device to device. For our device, the FASTEST frequency was 100 Hz. In this study, we collected training data for six activities from two subjects. For each activity, 30 samples were collected at four different SFs; thus, we collected 1440 samples in total. A sample was a unit with a single activity classification and corresponded to a window that contained the preset number of contiguous accelerometer data, which we called the WS. Section 4 discusses the experiments performed with the above samples. Figure 2 illustrates an example of the acceleration signals of human activities on each axis. This example was obtained at an SF of 100 Hz and WS of 128.

Figure 2: An example of acceleration signals of target activities on three axes.

3.2. Preprocessing Data

The preprocessing step consists of segmentation, the total magnitude (TM), and normalization. In the segmentation phase, the raw accelerometer data are segmented into windows with size , where accelerometer samples overlap between two consecutive windows. Feature extraction has been successfully performed on windows with 50% overlap in previous work [17]. The TM is the intensity (vibration) of a user activity and is a significant metric for discriminating between activities [8, 16, 24]. The TM is calculated according to , where is the magnitude of the sampled data on -axis. Figure 3 plots the acceleration on each axis, and the TM data are a sample of the “walking” activity.

Figure 3: Normalized amplitude of the “walking” activity on -, -, and -axes and the total magnitude.

Finally, the raw data and TM data are normalized to have values in range of (−1, 1) for later feature extraction and classification [28].

3.3. Extracting Features

The selection of proper features from raw data plays an important role in the HAR performance. In general, the relevant features extracted for HAR are grouped into three categories: (i) time-domain features such as the mean, standard deviation, energy, and correlation between axes [13–17, 23]; (ii) frequency-domain features such as the FFT coefficient, zero crossing rate, and autocorrelation of the magnitude [13–17, 29, 30]; and (iii) other features such as wavelet features [16, 29], the autoregressive coefficient [31], and discrete cosine transform coefficients [32].

The FFT coefficient demonstrates a higher average accuracy than the rest of the features [16, 31]. Thus, the first 20 FFT coefficients (first five for each of the three axes and five from TM; see Sections 4.1 and 4.2) are selected for each window, as illustrated in Figure 4. The FFT coefficients on each axis reflect the amplitude of basic waves which can be combined to reconstruct the original signal. For FFT, we utilized the decimation-in-time (DIT) Radix-2 FFT [33], which recursively partitions a discrete Fourier transform (DFT) into two half-length DFTs of the even- and odd-indexed time samples.

Figure 4: 3D acceleration and total magnitude after fast Fourier transform.

3.4. Classifying Activities

The extracted feature vectors can be classified by using the SVM classifier, which is widely used for HAR [13–15, 23]. LibSVM [34] was adopted to classify the dataset. SVM is a learning algorithm that separates training samples into their corresponding classes by maximizing the margin of a separating hyperplane between classes in order to solve the classification problem. SVM efficiently finds the complex hyperplane in nonlinear data by using the kernel trick. We used the radial basis function (RBF) kernel in order to map support vectors to multiple dimensions because there were 20 FFT attributes [23].

Human activities were classified into two activity types, as given in Table 1: (i) the static activity type (SAT) includes “sitting” and “standing” and (ii) the dynamic activity type (DAT) includes “walking,” “running,” “ascending stairs,” and “descending stairs.” The SAT is equivalent to a nonmoving relaxed state, and the DAT denotes active movement. Our strategy exploits the fact that humans are likely to maintain the same activity type for some time, especially for the SAT.

4. Tradeoff between Energy and Accuracy

The effects of the SF, WS, and FVD on the classification accuracy and power consumption were evaluated, and the FVD and combination of SF and WS were identified for application to our method. To obtain the readings, we turned off the network interfaces and display of our mobile device during the experiment. We used PowerTutor [35] utility to measure the power consumption.

4.1. Classification Accuracy and Acceleration-Sampling Frequency

We investigated the impact of different SFs on the classification accuracy with a WS of 128 and FVD of 20. Here, 2400 test samples were used (six activities × four SFs × 100 samples).

As shown in Figure 5, high SFs normally produced better predictions, especially for the DAT cases. The SFs of 50 and 100 Hz recorded an average accuracy of 90% or more in six activities and were sufficiently higher than the minimum SF of 20 Hz that is required to assess daily activities [36].

Figure 5: Accuracy across six activities with different acceleration-sampling frequencies.

4.2. Classification Accuracy and Feature Vector Dimensionality with Differing Window Sizes

Figure 6 illustrates how the classification accuracy changed with the number of coefficients for each WS. Using the first 20 FFT coefficients (first five for each of the three axes and five from TM) produced an accuracy of more than 90% for a WS of 128 or more. Our experiments showed a slightly different result compared to Preece et al. [29], who analyzed the discriminative ability of individual FFT coefficients. They found that applying the first 18 coefficients (first six on each of the three axes) produced the maximal accuracy. This discrepancy may be due to our incorporation of TM coefficients in our feature vectors.

Figure 6: Accuracy versus feature vector dimensionality with different window sizes.

4.3. Power Consumption and Feature Vector Dimensionality

For this experiment, we set the SF and WS to 100 Hz and 128, respectively. The SF of 100 Hz had the best classification accuracy, as shown in Figure 5, and the WS of 128 had a prediction accuracy of over 90%, as shown in Figure 6. Figure 7 plots the power consumption over 30 min against different numbers of FFT coefficients. The power consumption showed a quadratic increase with the dimensionality. Based on the results shown in Figures 3 and 4, we selected an FVD of 20 in our study. This had the least power consumption among FVDs with an accuracy of more than 90%.

Figure 7: Power consumption of feature vector dimensionalities at an acceleration-sampling frequency of 100 Hz and window size of 128 over 30 min.

4.4. Power Consumption and Acceleration-Sampling Frequency with Differing Window Sizes

Figure 8 illustrates the power consumption for different SFs and WSs with an FVD of 20 over 2 h. The results can be summarized as follows:

(i)The power consumption clearly increases with the SF. A high frequency mandates more frequent raw data collection.(ii)Larger WSs normally consume less power because they decrease the number of classifications, which take up a large proportion of the power consumption.

Figure 8: Power consumption at different acceleration-sampling frequencies and window sizes over 2 h.

Table 2 summarizes our investigations. We adopted SFs (50 Hz and 100 Hz), WSs (128, 256, and 512), and an FVD (20) which yielded an accuracy of 90% or more with low power consumption.

Table 2: Summary of tradeoff between energy and accuracy.

Observation
Accuracy
Power consumption

Higher SF
Higher
Increase

Bigger WS
Higher
Decrease

More FVD
Higher
Rapid increase

5. Experiments on the Variable Activity Recognition Duration Strategy

To monitor user activities on smart mobile devices in an energy-efficient manner, our study focused on two key ideas.

First, humans more often tend to maintain the same activity than change from one activity to another (e.g., walk-to-run and sit-to-stand). When one activity is recognized in succession, we assumed that the activity will be lasted for a while. Therefore, we focused on developing an energy-saving scheme that increases the classification duration this situation. If we increase the period in which an activity is recognized in a given time, the frequency of activity recognition will decrease. Consequently, this reduces the power consumption necessary for activity recognition. To increase the classification duration, we adopted a method that lowers the SF and/or increases the WS. We verified that a low SF and large WS consume less power, as shown in Figure 8.

Second, dynamic activity (e.g., walking and running) is more meaningful than static activity (e.g., sitting and standing) equivalent to a nonmoving relaxed state because it can be used as data for dynamic health information such as calorie consumption. Thus, we first classified a user’s activities as a DAT and SAT, as indicated in Table 1. And then, when an SAT is recognized, we gave a break to the HAR process in order to save more energy.

Based on these ideas, we applied different strategies for each type with regard to the classification duration. To control the duration, the SF and WS were used for the DAT, and a sleep time was additionally used for the SAT. We call this energy-saving scheme the variable activity recognition duration (VARD) strategy.

5.1. Variable Activity Recognition Duration Strategy for the Dynamic Activity Type

To increase the classification duration, we can lower the SF and/or increase the WS. However, a low SF and large WS are insensitive to rapidly changing activities because they yield fewer samples than a high SF and small WS. Therefore, our strategy is to start with a high SF and small WS to quickly identify changing activities. If the same dynamic activity is maintained for a long time, we assume that the same activity will continue and adopt a method to lower the SF and increase the WS.

To guarantee the energy efficiency and high accuracy of HAR, we can choose SFs of 50 and 100 Hz, as shown in Figure 5, and WSs of 128, 256, and 512, as shown in Figure 6. Each SF and WS can be combined for a total of six combinations. The classification durations of 〈100 Hz, 256〉 and 〈50 Hz, 128〉 are the same at 2.56 s.

However, the power consumption of 〈50 Hz, 128〉 (573 mWh) is less than that of 〈100 Hz, 256〉 (832 mWh), as shown in Figure 8. Another difference between the two combinations is that the larger WS provides better HAR accuracy because it extracts more precise features in the raw data with noise comprising the latter part of previous acceleration from the changing activity, as shown in Figure 9. These two differences have conflicting tendencies for the energy efficiency and HAR accuracy. If the classification durations overlap, we can choose the energy-efficient combination to focus on saving energy.

Figure 9: Comparison of feature extraction precision with noise.

Accordingly, we adopted four combinations for the strategy with the DAT, as listed in Table 3: 〈100 Hz, 128〉, 〈50 Hz, 128〉, 〈50 Hz, 256〉, and 〈50 Hz, 512〉. We used the repeating count of the same activity in order to check that the same activity is continuous. A threshold for this count was set, and we implemented a strategy of changing from the current combination to the next combination with a low frequency and large WS if the count carries over the threshold. The progression to each configuration away from the first combination causes the improvement in energy efficiency and marginal weakening of the HAR accuracy.

Table 3: Variable activity recognition duration configuration for dynamic activity type.

Activity type
Acceleration-sampling frequency
Window size

Dynamic
100 Hz
128

50 Hz
128

50 Hz
256

50 Hz
512

5.2. Variable Activity Recognition Duration Strategy for the Static Activity Type

Our strategy for the SAT is based on a similar concept for the DAT strategy. However, there is no need to recognize SAT often because there is less movement compared with DAT. Our SAT strategy, therefore, uses the sleep time during the HAR process along with the SF and WS for better energy efficiency compared to the DAT strategy. In addition, a DAT should be stably recognized in the SAT state because it is more important than the SAT for extracting processed information.

In our strategy, when an SAT is recognized during the classification of human activity, the process takes a break. After the break, the human activity is reclassified. As a result, the classification duration increases within a given time because this strategy incorporates a sleep time.

To ensure stable HAR accuracy while reducing energy consumption, this strategy involves Sleeping 0 s when an SAT is initially recognized and gradually increasing the sleep time in increments of 1 s whenever an SAT is continuously recognized.

The power consumption can be reduced with a break. Nevertheless, the extent to which the break can be increased while ensuring stable HAR accuracy needed to be evaluated. Therefore, we investigated the HAR accuracy with six combinations: 〈100 Hz, 128〉, 〈50 Hz, 128〉, 〈100 Hz, 256〉, 〈50 Hz, 256〉, 〈100 Hz, 512〉, and 〈50 Hz, 512〉. This was done in order to calculate the preferred maximum sleep time. In this experiment, the HAR accuracy was observed as the break was increased from 0 s to 60 s for each combination. The observation times for each break were 5 and 10 min. We made a total of 732 observations (6 × 61 × 2) of the HAR accuracy. Figure 10 plots the observed HAR accuracy.

Figure 10: Recognition accuracy of human activities with variable activity recognition duration. The circle of each combination represents the break where the human activity recognition accuracy violently fluctuates.

This experiment showed that the HAR accuracy became unstable every time the break was over a certain amount. The circular symbols in Figure 10 show the break after which the HAR accuracy badly fluctuated. This point was the limit to the break for each combination. The limit can be calculated by

where is an SF, is a WS, and is a constant of 30 s as determined in this experiment. Based on this limit, we can guarantee efficient power consumption and stable accuracy during HAR.

Figure 11 plots the measured power consumption of six combinations in 2 h with a preset maximum sleep time: 〈100 Hz, 128〉, 〈50 Hz, 128〉, 〈100 Hz, 256〉, 〈50 Hz, 256〉, 〈100 Hz, 512〉, and 〈50 Hz, 512〉. The power consumption increased with a larger WS relative to a small WS, and changes to the SF had less effect on the power consumption than changes to the WS. This is because the numbers of activity recognition processes for every combination within a given time are equal if the HAR process has a sleep time, and a large WS increases the computational cost of HAR. As a result, the samples with a large WS consumed more power. Therefore, using a small WS can ensure high energy efficiency.

Figure 11: Power consumption of each of six combinations with a maximum sleep time of 2 h.

As shown in Figure 10, however, the average accuracy is higher for large WSs than small WSs. Thus, we adopted three combinations for the SAT strategy: 〈100 Hz, 512〉, 〈100 Hz, 256〉, and 〈100 Hz, 128〉. As indicated in Table 4, we defined the VARD combination configuration for SAT strategy. We used the repeating count of SAT in order to check that the type is continuous and employed a strategy of changing from the current combination to the next combination with a smaller WS if the count carried over a threshold based on the sleep time limit. Progressing to further configurations away from the first combination increases the energy efficiency and destabilizes the HAR accuracy.

Table 4: Variable activity recognition duration configuration for static activity type.

Activity type
Acceleration-sampling frequency
Window size

Static
100 Hz
512

100 Hz
256

100 Hz
128

5.3. Real-Time Human Activity Recognition with the Variable Activity Recognition Duration Strategy

The VARD strategy can effectively guarantee not only classification accuracy but also energy efficiency because it does not need to constantly keep a specific SF and WS for HAR. Figure 12 represents our approach as a state machine diagram, and the strategy is described in Algorithm 1. In order to obtain a break, our HAR process is divided into a Sensing State and Sleeping State, as shown in Figure 12.

(1) Set the SF f with the initial and the WS with the initial size;

(2) Load the classification model for f and ;

(3) Load the dynamic configuration table as [: (100 Hz, 128), : (50 Hz, 128), : (50 Hz, 256), : (50 Hz, 512)];

(4) Load the static configuration table as [: (100 Hz, 512), : (100 Hz, 256), : (100 Hz, 128)],

(5) the repeating count of SAT , and the repeating count of the same activity ; ;

(6) Set the threshold th for

(7) and the maximum sleep time ;

(8) While   Do

(9)         Start the accelerometer; fill window with ;

(10)      Classify the current activity from the window with ;

(11)       If the current activity is DAT Then

(12)              ; ;

(13)              If the current activity is equivalent to the previous activity Then

(14)                      Increase ;

(15)                      If exceeds th Then

(16)                              Increase up to the size of the dynamic configuration table; ;

(17)                      End If

(18)              Else

(19)                      ; ;

(20)              End If

(21)              Update f and with the control table ;

(22)              Load the classification model for f and ;

(23)       Else

(24)              If is equivalent to 0 Then

(25)                      ;

(26)              End If

(27)              Update f and with the control table ;

(28)              Load the classification model for f and ;

(29)              Calculate based on (1);

(30)              If exceeds   Then

(31)                      Increase up to the size of the static configuration table; ;

(32)              End If

(33)              Stop the accelerometer;

(34)              Set the timeout of Sleeping with the repeating count of SAT ;

(35)              Delay for the timeout of Sleeping;

(36)              Increase ;

(37)      End If

(38) End While

Algorithm 1: Variable activity recognition duration algorithm. The elements (, ) of the dynamic and static configuration tables contain the acceleration-sampling frequency and window size where th is the threshold to maintain a combination in the dynamic configuration table, is the repeating count of the static activity type, is the count continuously kept of any activity, and is the index of an element in the configuration table. There are seven classification models for each element in the control tables, and is the maximum sleep time.

Figure 12: State machine for the variable activity recognition duration strategy.

The Sensing State repeats the following cycles: collecting, preprocessing, feature extraction, and classification. Our classifier in the HAR process uses a variety of training models for VARD configuration, as indicated in Tables 3 and 4. These models are built by an offline SVM using the training samples discussed in Section 3.1.

By classifying a recognized activity as a DAT or SAT, the HAR process transfers from the Sensing State to the state for each type. For a DAT, the HAR process goes into the Dynamic State to perform the DAT strategy. Otherwise, the SAT strategy is performed for the Static State. When the SAT strategy is performed, the HAR process transfers to the Sleeping State unconditionally and takes a break. This break time is set by the repeating count of SAT. After the break, the process returns to the Sensing State in order to reclassify the human activity.

When an event listener for the triaxial accelerometer is registered in the initial Idle State, the HAR process transfers to the Active State. The Active State comprises two substate machines: the Sensing State and Sleeping State. In the Active State, the process initializes the SF and WS and loads the classification model for this combination. It also sets a threshold for the repeating count of the same activity. The HAR process transfers to the Sensing State after the accelerometer is started. While this transition is performed, the repeating count of the SAT and repeating count of the same activity are initialized with zero. When all of the initializations are completed, the Sensing State begins so that a human activity can be recognized. This portion is equivalent to lines (1)–(10) in Algorithm 1.

When a recognized activity is a DAT, the HAR process is transferred to the Dynamic State. In this state, the repeating count of the same activity and maximum sleep time are initialized. In the Dynamic State, the current activity is checked to see if it is equivalent to the previous activity. If they are the same, the repeating count of the same activity is increased. If this count exceeds the threshold, then the current VARD configuration is changed to the next combination, and the count is initialized. If the current and previous activities are not the same, the repeating count of the same activity is initialized, and the VARD configuration is changed to the first DAT combination. In the Dynamic State, SF and WS are updated by the current configuration, and a classification model is loaded for the configuration. At the end of the Dynamic State, HAR is started. This portion is equivalent to lines (11)–(22) in Algorithm 1.

When a recognized activity is an SAT, the HAR process is transferred to a Static State. If the previous activity is a DAT, the VARD configuration is first changed to an SAT. In the Static State, SF and WS are updated by the current configuration, and a classification model is loaded for the configuration. If the repeating count of the SAT exceeds the maximum sleep time, the current VARD configuration is changed to the next SAT combination, and the count is initialized. Then, the HAR process is transferred to the Sleeping State, and the accelerometer stops. In this state, the sleep time is set by using the repeating count of the SAT, and the HAR process takes a break during this time. After the break, the repeating count of the SAT is increased, and the HAR process is transferred to the Active State. This portion is equivalent to lines (23)–(38) in Algorithm 1.

6. Performance Evaluation and Discussion

To evaluate the performance of the proposed algorithm, we performed independent experiments with regard to the recognition accuracy and power consumption. An application employing our approach was installed as an Android service that can operate in the background. The initial WS and SF were set to 128 and 100 Hz, respectively. The threshold th for a DAT was set to 10. The experimental results were as follows.

6.1. Energy Efficiency

Five cases were considered, each for a span of 12 h:

(i)No HAR: there is no HAR application running on the phone.(ii)Typical SVM: the SF is fixed at 100 Hz, and the WS is fixed at 128.(iii)VARD with DAT only: all activities are assumed to be DAT.(iv)VARD with SAT only: all activities are assumed to be SAT.(v)VARD with daily activities: daily activities include walking to the lab, moving on stairs, studying at a desk, and jogging.

We measured the battery level by using BatteryManager Android API and powered off the network interfaces and display of our mobile device during the experiment. Figure 13 compares the battery drainage time series in our experiment. HAR with VARD showed slow and stable power consumption of the smart mobile device over time. VARD with DAT represented only the maximum power consumption of our approach. This case clearly reduced the energy consumption by 23% compared to the typical SVM case. VARD with SAT represented the minimum power consumption and consumed 3% more power than with no HAR. Finally, VARD with daily activities showed a reduction of 36% in energy consumption compared to typical SVM. The increase in energy efficiency compared to typical SVM was computed by . The increase in efficiency was about 44.23% for VARD with dynamic activity only, about 78.85% for VARD with static activity only, and about 69.23% for VARD with daily activities.

Figure 13: Comparing power consumption of different types.

6.2. Human Activity Recognition Accuracy

The confusion matrix in Table 5 represents HAR errors for a real dataset (six activities × 100 samples). The confusion matrix shows that 5% of “walking” was misclassified as “ascending stairs” and 6% for opposite misclassification. Also, 8% of “sitting” was misclassified as “standing” and 5% for the opposite misclassification. The experimental results showed that the average HAR accuracy was 92.17%. If the activities “sitting” and “standing” are unified into a relaxation activity, the HAR accuracy for an SAT would be 99.5%.

Table 5: Confusion matrix of human activity recognition with variable activity recognition duration.

Classified as
a
b
c
d
e
f

a
92
5

b
8
94

c

1
91

6
4

d

2
97
2
2

e

5
1
89
4

f

2
2
3
90

Key: a, sitting; b, standing; c, walking; d, running; e, ascending stairs; f, descending stairs.

7. Conclusions

Conventional HAR using the built-in accelerometer in smart mobile devices still has high power consumption due to not only the sensor itself but also the accompanying CPU computation overhead. Inspired by such challenge, we presented a new approach for energy-efficient real-time HAR on smart mobile devices. The experimental results showed that our method can achieve greater than 64% average energy-saving as compared to conventional HAR (SVM). We also showed that the average HAR accuracy was about 92% with six different activities. Moreover, we reported on how the SF, WS, and FVD alter the battery power consumption behavior with HAR. This report may be helpful to the field of HAR. However, if the Sleeping State persists for a long time, sudden human activities such as a fall cannot be recognized properly. In order to solve this problem, future work on improving the accuracy for recognizing sudden activity changes is needed.

Competing Interests

The authors declare that they have no competing interests.

References

H. Yan, H. Huo, Y. Xu, and M. Gidlund, “Wireless sensor network based E-health system-implementation and experimental results,” IEEE Transactions on Consumer Electronics, vol. 56, no. 4, pp. 2288–2295, 2010.

M. V. Albert, S. Toledo, M. Shapiro, and K. Kording, “Using mobile phones for activity recognition in Parkinson's patients,” Frontiers in Neurology, vol. 3, article 158, 2012.

L. Tang, X. Zhou, Z. Yu, Y. Liang, D. Zhang, and H. Ni, “MHS: a multimedia system for improving medication adherence in elderly care,” IEEE Systems Journal, vol. 5, no. 4, pp. 506–517, 2011.

A. Anjum and M. U. Ilyas, “Activity recognition using smartphone sensors,” in Proceedings of the IEEE 10th Consumer Communications and Networking Conference (CCNC '13), pp. 914–919, January 2013.

J. Wang, Z. Zhang, B. Li, S. Lee, and R. S. Sherratt, “An enhanced fall detection system for elderly person monitoring using consumer home networks,” IEEE Transactions on Consumer Electronics, vol. 60, no. 1, pp. 23–29, 2014.

M.-W. Lee, A. M. Khan, and T.-S. Kim, “A single tri-axial accelerometer-based real-time personal life log system capable of human activity recognition and exercise information generation,” Personal and Ubiquitous Computing, vol. 15, no. 8, pp. 887–898, 2011.

S. Abbate, M. Avvenuti, F. Bonatesta, G. Cola, P. Corsini, and A. Vecchio, “A smartphone-based fall detection system,” Pervasive and Mobile Computing, vol. 8, no. 6, pp. 883–899, 2012.

Y. He and Y. Li, “Physical activity recognition utilizing the built-in Kinematic sensors of a smartphone,” International Journal of Distributed Sensor Networks, vol. 2013, Article ID 481580, 10 pages, 2013.

J. W. Lockhart, T. Pulickal, and G. M. Weiss, “Applications of mobile activity recognition,” in Proceedings of the 14th International Conference on Ubiquitous Computing (UbiComp '12), pp. 1054–1058, Pittsburgh, Pa, USA, September 2012.

H. Lu, J. Yang, Z. Liu, N. D. Lane, T. Choudhury, and A. T. Campbell, “The Jigsaw continuous sensing engine for mobile phone applications,” in Proceedings of the 8th ACM International Conference on Embedded Networked Sensor Systems (SenSys '10), pp. 71–84, Zurich, Switzerland, November 2010.

G. Raffa, J. Lee, L. Nachman, and J. Song, “Don't slow me down: bringing energy efficiency to continuous gesture recognition,” in Proceedings of the 14th IEEE International Symposium on Wearable Computers (ISWC '10), pp. 1–8, Seoul, Republic of Korea, October 2010.

Y. Wang, J. Lin, M. Annavaram et al., “A framework of energy efficient mobile sensing for automatic user state recognition,” in Proceedings of the 7th ACM International Conference on Mobile Systems, Applications, and Services (MobiSys '09), pp. 179–192, Kraków, Poland, June 2009.

Q. V. Vo, M. T. Hoang, and D. Choi, “Personalization in mobile activity recognition system using K-medoids clustering algorithm,” International Journal of Distributed Sensor Networks, vol. 2013, Article ID 315841, 12 pages, 2013.

Q. V. Vo, M. T. Hoang, and D. Choi, “Adaptive energy-saving strategy for activity recognition on mobile phone,” in Proceedings of the 12th IEEE International Symposium on Signal Processing and Information Technology (ISSPIT '12), pp. 95–100, Ho Chi Minh City, Vietnam, December 2012.

Z. Yan, V. Subbaraju, D. Chakraborty, A. Misra, and K. Aberer, “Energy-efficient continuous activity recognition on mobile phones: an activity-adaptive approach,” in Proceedings of the 16th International Symposium on Wearable Computers (ISWC '12), pp. 17–24, Newcastle, UK, June 2012.

Y. Liang, X. Zhou, Z. Yu, and B. Guo, “Energy-efficient motion related activity recognition on mobile devices for pervasive healthcare,” Mobile Networks and Applications, vol. 19, no. 3, pp. 303–317, 2014.

L. Bao and S. S. Intille, “Activity recognition from user-annotated acceleration data,” in Pervasive Computing: Second International Conference, PERVASIVE 2004, Linz/Vienna, Austria, April 21–23, 2004. Proceedings, vol. 3001 of Lecture Notes in Computer Science, pp. 1–17, Springer, Berlin, Germany, 2004.

N. Kern, B. Schiele, and A. Schmidt, “Recognizing context for annotating a live life recording,” Personal and Ubiquitous Computing, vol. 11, no. 4, pp. 251–263, 2007.

A. M. Khan, Y.-K. Lee, S. Y. Lee, and T.-S. Kim, “A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer,” IEEE Transactions on Information Technology in Biomedicine, vol. 14, no. 5, pp. 1166–1172, 2010.

I. C. Gyllensten and A. G. Bonomi, “Identifying types of physical activity with a single accelerometer: evaluating laboratory-trained algorithms in daily life,” IEEE Transactions on Biomedical Engineering, vol. 58, no. 9, pp. 2656–2663, 2011.

J.-H. Hong, J. Ramos, and A. K. Dey, “Toward personalized activity recognition systems with a semipopulation approach,” IEEE Transactions on Human-Machine Systems, vol. 46, no. 1, pp. 101–112, 2016.

V. Könönen, J. Mäntyjärvi, H. Similä, J. Pärkkä, and M. Ermes, “Automatic feature selection for context recognition in mobile devices,” Pervasive and Mobile Computing, vol. 6, no. 2, pp. 181–197, 2010.

M. Khan, S. I. Ahamed, M. Rahman, and R. O. Smith, “A feature extraction method for real time human activity recognition on cell phones,” in Proceedings of the 3rd International Symposium on Quality of Life Technology (isQoLT '11), Toronto, Canada, 2011.

J. R. Kwapisz, G. M. Weiss, and S. A. Moore, “Activity recognition using cell phone accelerometers,” ACM SIGKDD Explorations Newsletter, vol. 12, no. 2, pp. 74–82, 2011.

C. Torres-Huitzil and M. Nuno-Maganda, “Robust smartphone-based human activity recognition using a tri-axial accelerometer,” in Proceedings of the 6th IEEE Latin American Symposium on Circuits and Systems (LASCAS '15), pp. 1–4, February 2015.

M. F. A. bin Abdullah, A. F. P. Negara, M. S. Sayeed, D. J. Choi, and K. S. Muthu, “Classification algorithms in human activity recognition using smartphones,” International Journal of Computer and Information Engineering, vol. 6, pp. 77–84, 2012.

O. D. Incel, M. Kose, and C. Ersoy, “A review and taxonomy of activity recognition on mobile phones,” BioNanoScience, vol. 3, no. 2, pp. 145–171, 2013.

S. Wang, J. Yang, N. Chen, X. Chen, and Q. Zhang, “Human activity recognition with user-free accelerometers in the sensor networks,” in Proceedings of the International Conference on Neural Networks and Brain (ICNNB '05), pp. 1212–1217, Beijing, China, October 2005.

S. J. Preece, J. Y. Goulermas, L. P. J. Kenney, and D. Howard, “A comparison of feature extraction methods for the classification of dynamic activities from accelerometer data,” IEEE Transactions on Biomedical Engineering, vol. 56, no. 3, pp. 871–879, 2009.

Y. E. Ustev, O. D. Incel, and C. Ersoy, “User, device and orientation independent human activity recognition on mobile phones: challenges and a proposal,” in Proceedings of the 2013 ACM Conference on Ubiquitous Computing (UbiComp '13), pp. 1427–1435, Zurich, Switzerland, September 2013.

Z.-Y. He and L.-W. Jin, “Activity recognition from acceleration data using AR model representation and SVM,” in Proceedings of the 7th International Conference on Machine Learning and Cybernetics (ICMLC '08), pp. 2245–2250, Kunming, China, July 2008.

Y. Xue and L. Jin, “A naturalistic 3D acceleration-based activity dataset & benchmark evaluations,” in Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (SMC '10), pp. 4081–4085, Istanbul, Turkey, October 2010.

D. Jones, “Decimation-in-time (DIT) radix-2 FFT,” Connexions, vol. 15, p. 2006, 2006.

C. Chang and C. Lin, “LIBSVM: a library for support vector machines,” ACM Transactions on Intelligent Systems and Technology, vol. 2, no. 3, pp. 1–27, 2011.

PowerTutor, “A Power Monitor for Android-Based Mobile Platforms,” http://ziyang.eecs.umich.edu/projects/powertutor/

C. V. C. Bouten, K. T. M. Koekkoek, M. Verduin, R. Kodde, and J. D. Janssen, “A triaxial accelerometer and portable data processing unit for the assessment of daily physical activity,” IEEE Transactions on Biomedical Engineering, vol. 44, no. 3, pp. 136–147, 1997.