Correctly identifying human activities is very significant in modern life. Almost all feature extraction methods are based directly on acceleration and angular velocity. However, we found that some activities have no difference in acceleration and angular velocity. Therefore, we believe that for these activities, any feature extraction method based on acceleration and angular velocity is difficult to achieve good results. After analyzing the difference of these indistinguishable movements, we propose several new features to improve accuracy of recognition. We compare the traditional features and our custom features. In addition, we examined whether the time-domain features and frequency-domain features based on acceleration and angular velocity are different. The results show that (1) our custom features significantly improve the precision of the activities that have no difference in acceleration and angular velocity; and (2) the combination of time-domain features and frequency-domain features does not significantly improve the recognition of different activities.
The classification of human motion based on inertial sensors has been proven to have many important applications in the medical and health fields. In previous studies, time-domain and frequency-domain features are widely used for feature calculation.
There are many studies that use wavelet transform to extract features to classify human activities. However, the research of Preece et al. [
Some time-domain features are derived to classify human activities, such as the mean, median, variance, skewness, kurtosis [
We studied 12 kinds of activities and found it easy to confuse elevator up and elevator down. These two kinds of activities do not have obvious differences in acceleration and angular velocity, so the time and frequency-domain features based on acceleration and angular velocity cannot achieve good classification results. Therefore, for those motions that have no significant difference in angular velocity and acceleration, no matter how the features are extracted from the acceleration and angular velocity, it is difficult to achieve good results.
In addition, we begin to wonder if there is any essential difference between time-domain features and frequency-domain features based on acceleration and angular velocity. In order to solve this discredit, we have separately tested the effects of time-domain features and frequency-domain features. Then, we tested the combination of the two kinds of features and found that the combination of time-domain features and frequency-domain features was slightly higher than only time-domain features. From the experimental results analysis, we believe that, for human activity classification problem, the time-domain features and the frequency-domain features are two aspects of the same rules, and there is no essential difference.
Our contributions in this paper are two-fold: (1) to the best of our knowledge, this is the first time that features of motion classification based on velocity and displacement have been proposed, which solve some problems that cannot be solved by traditional time- and frequency-domain features; (2) As far as we know, for the first time, we have studied the difference between the time-domain features and frequency-domain features.
Indeed, the time-domain and frequency-domain features based on acceleration and angular velocity have achieved some success. However, for activities without obvious difference between acceleration and angular velocity, such as elevator up and elevator down, the traditional method of extracting features based on acceleration and angular velocity is difficult to work. In order to solve this problem, we carefully analyze the two activities of elevator up and elevator down, summarize the differences between them, and propose some new features for distinguishing such activities. After analysis, we sum up the following rules: When the elevator goes up, the speed is upward; when the elevator goes down, the speed is downward. When the elevator just starts to move or stops moving, its speed is small and its angular speed is large. When the elevator just starts to rise or fall, the direction of the speed is the same as the direction of the acceleration; when the elevator stops to rise or fall, the direction of the speed is opposite to the direction of the acceleration.
Based on the above evidences, we propose four features to distinguish these activities on each axis of the accelerometer. First, according to the first rule, we introduce three features, namely, the starting speed, the ending speed, and the displacement. Second, according to the second and third rules, we introduce the fourth feature. As some activities have just started and are about to stop, their speed is small and difficult to distinguish. Therefore, we extracted another feature to enhance the difference between two activities. When the velocity direction is the same as the acceleration direction, we use
Custom features.
Feature name | Description |
---|---|
End velocity along X | End speed along the |
End velocity along Y | End speed along the |
End velocity along Z | End speed along the |
Starting velocity along X | Starting speed along the |
Starting velocity along Y | Starting speed along the |
Starting velocity along Z | Starting speed along the |
Displacement along X | Displacement along the |
Displacement along Y | Displacement along the |
Displacement along Z | Displacement along the |
Velocity plus acceleration along X |
|
Velocity plus acceleration along Y |
|
Velocity plus acceleration along Z |
|
Suppose the time window we choose is
Data section of one time window.
We think these sampling points are equally time-distributed, so we have the following conclusions.
First, we derive the formula for the end speed of each axis. According to the kinematics formula, we can easily get the following formula and simplify it by combining it with Equation (
Next, we introduce the determination of the starting speed. If this window is the first window, we default to a starting speed of zero. Otherwise, the starting speed is the end speed of the previous window marked as
As for the displacement of each axis, since we think that there is uniform linear motion between each sampling point, the displacement formula can be derived as follows.
In the experiment, in order to calculate the last three features we defined in Table
Finally, we introduce the calculation method of gravity acceleration we used in the experiment. Since the tester is just wearing the device for data acquisition, it is generally at a standstill and the starting speed is zero, which is used in our experiments. Therefore, for the sake of convenience, in the experiment, we believe that the initial acceleration of each axis is the component of gravity acceleration and assume that the component of gravity acceleration in each axis remains unchanged. We take the average of the first 10 sampling points of each axis as the component of gravity acceleration in each axis, recorded as
In order to illustrate the validity of our custom features, we have selected the USC_HAD of University of Southern California as the verification dataset [
In order to verify the validity of our custom features, we extracted some common time- and frequency-domain features. In the time domain, we chose the mean, median, variance, skewness, kurtosis, and interquartile range as time-domain features. In the frequency domain, we choose peak frequency, median frequency, power spectral density, DC component, spectral energy, and information entropy as frequency-domain features. Then, we added our custom features to these time- and frequency-domain features and compared their results. In the experiments, we adopted the two commonly used models, SVM and random forest. For SVM, the kernel function we use is a polynomial kernel function. For RF, the number of decision trees we choose is 50.
First, in order to check whether the distinction between the lift of the elevator and the descending of the elevator is achieved, we tested the precision and recall of our custom features on both models. Precision and recall are often used as performance measures for classifiers in classification problems. Table
Precision and recall of the identification of elevators up when adding custom features and no custom features.
SVM (%) | RF (%) | |||
---|---|---|---|---|
No custom features | 50.60 | 55.23 | 56.82 | 52.08 |
Adding custom features | 74.01 | 74.50 | 78.05 | 81.53 |
Precision and recall of the identification of elevators down when adding custom features and no custom features.
SVM (%) | RF (%) | |||
---|---|---|---|---|
No custom features | 48.38 | 53.78 | 55.24 | 52.84 |
Adding custom features | 71.38 | 71.61 | 83.23 | 85.67 |
Also, we use the ROC (receiver operating characteristics) curve and corresponding AUC (area under the ROC curve) values to check whether the distinction between the lift of the elevator, and the descending of the elevator is achieved. For SVM, we conducted a test. The ROC curves for the two types of classification results for elevator up and elevator down are shown in Figure
ROC curve of elevator up and elevator down over SVM.
From the above figure, we can see that in the SVM, after adding our custom features, the ROC curves of elevator up and elevator down completely cover the ROC curve of the original feature. The AUC value for the no custom features’ ROC curve is 0.8572, while the AUC value for adding custom features’ ROC curve is 0.9729. The ROC curve shows that our custom features have achieved very good results in distinguishing between the elevator up and the elevator down.
In order to further explain the significance of our custom features we have extracted and verify the difference between time-domain features and frequency-domain features, we have compared our custom features with the time-domain features and frequency-domain features. We performed comparative experiments on four combinations of features over SVM. We conducted five experiments for each experiment. The detailed experimental results are summarized in Table
Classification accuracy of 12 activities on four combinations of frequency-domain features, time-domain features, and custom features over SVM.
1 (%) | 2 (%) | 1, 2 (%) | 1, 3 (%) | |
---|---|---|---|---|
First experiment | 89.67 | 83.93 | 90.46 | 92.21 |
Second experiment | 90.52 | 84.11 | 90.80 | 92.17 |
Third experiment | 89.78 | 83.07 | 91.27 | 91.94 |
Fourth experiment | 89.34 | 83.50 | 90.62 | 92.61 |
Fifth experiment | 89.29 | 83.93 | 90.21 | 92.25 |
From the experimental results, we can see that individual frequency-domain features and time-domain features can achieve good results. However, when frequency-domain features are combined with time-domain features, no significant improvement is obtained, indicating that there is no essential difference between features extracted from the frequency domain and features extracted from the time domain. The superposition of the two did not achieve better results. The time-domain feature is better when combined with our custom features than combined with frequency-domain features. Based on the comprehensive analysis, we believe that our custom feature is a supplement to traditional time-domain features rather than a redundant feature.
The traditional time-domain features and frequency-domain features are all based on the acceleration and angular velocity but there is no essential difference, so the superposition of the two will not bring significant improvement. Our custom feature is the mining of the rules of speed and displacement, which is very different from the traditional mining of acceleration and angular velocity. So, when these features are introduced in the time-domain feature, we can obtain certain promotion. Especially for those motions which have no obvious difference between angular velocity and acceleration but there is a clear difference in speed and displacement, we can achieve good results with these custom features. For example, there is no obvious difference in acceleration and angular velocity in the smooth upward movement of the elevator and the smooth descending of the elevator, but there is a clear difference in speed and displacement. So, when introduce our custom feature, we can obviously increase the recognition rate of the two kinds of motions.
To calculate these features we define, we must know the initial state of motion, especially the initial state of speed. In our experiment, we assumed that the initial state is zero. In the database we use, most of the data are collected after the tester reaches a steady state of various motion postures, which does not satisfy our assumptions. If we can record data when the tester just wears the inertial sensors, so as to meet our assumptions, we believe we can achieve better results. For other types of sports, such as running, walking, and station, there is a difference in speed, which will bring a higher recognition rate.
In this article, we begin with the elevator up and elevator down, which are indistinguishable based on the existing feature extraction methods and analyze the differences and rules between these two types of movements. Then, we have proposed four features on each axis of the accelerometer that have significantly improved the distinction between the two types of movements, elevator up, and elevator down. In the experiment, we found that the combination of frequency-domain features and time-domain features does not significantly improve the distinction of activities. The two kinds of features are two different aspects of acceleration and angular velocity, and there is no essential difference. From the experimental results, the time-domain features are better than the frequency-domain features and can more fully reflect the differences between different activities. Our custom features are not another response to acceleration, but instead, these features can be used to distinguish movements that differ in the speed of movement. In particular, it is of great significance to distinguish between movements that do not have a significant difference in acceleration and angular velocity but have a significant difference in speed.
In the experiment, for the sake of convenience, we assumed that the component of the gravitational acceleration remains unchanged, which is obviously not in line with the actual situation. Next, related personnel may consider introducing some basic theories of motion analysis in order to accurately calculate the components of the gravitational acceleration and thus more accurately calculate the features we introduce. We believe that when the features of velocity and displacement are introduced, we can make a great breakthrough in the existing human motion classification problem and to some extent get rid of the dilemma that some motions cannot be accurately identified on the features of acceleration and angular velocity.
The USC_HAD data used to support the findings of this study are included in Reference [
The authors declare that there are no conflicts of interest regarding the publication of this paper.
This work was supported by the Natural Science Foundation of China (No. 61571268).