Data Feature Extraction Method of Wearable Sensor Based on Convolutional Neural Network

With the rapid development of society and science technology, human health issues have attracted much attention due to wearable devices' ability to provide high-quality sports, health, and activity monitoring services. This paper proposes a method for feature extraction of wearable sensor data based on a convolutional neural network (CNN). First, it uses the Kalman filter to fuse the data to obtain a preliminary state estimation, and then it uses CNN to recognize human behavior, thereby obtaining the corresponding behavior set. Moreover, this paper conducts experiments on 5 datasets. The experimental results show that the method in this paper extracts data features at multiple scales while fully maintaining data independence, can effectively extract corresponding feature data, and has strong generalization ability, which can adapt to different learning tasks.


Introduction
Wearable devices are smart devices that can be installed on people, animals, and objects, and can sense, transmit, and process information [1]. Sensors are the core carrier, and wearable sensors are an extension of human senses. With the rapid development of the Internet of everything and society, human health issues have also received widespread attention. Various health indicators and parameters of the human body can also be connected to the Internet for monitoring and analysis. Perceive human behavior through wearable sensors, which can be used in medical health, sports fitness, disease prevention, and treatment, especially in the daily monitoring of elderly people [2][3][4][5][6][7][8]. With the intelligent development of wearable devices, sensors such as accelerometers, magnetometers, and heart rate meters are built into smartphones and smart bracelets [9]. e sensors can continuously track and collect human behavior data, fully protecting personal privacy as well. Users only need to wear their mobile phones or smart bracelet to complete data collection.
Most of the data collected by wearable sensors are sequence data. Currently, models based on recurrent neural networks (RNN) and convolutional neural networks (CNN) are mainly used to predict and identify sequence data [10,11]. Although the accuracy of the RNN-based data prediction and recognition model is slightly higher than the accuracy of the CNN-based model, the cyclic nature and data-dependent characteristics of the RNN model make it difficult to achieve high parallelization on the edge side, which leads to low computing efficiency on the edge hardware platform. CNN can achieve higher computational parallelism and higher applicability. erefore, this paper uses a CNN-based model to extract the data features of wearable sensors and perform human behavior recognition [12,13]. Figure 1 shows the process of feature extraction of wearable device sensing data and the output results based on CNN. First, the human body's acceleration, heart rate, and other data are collected through the sensor in the wearable device, and the original data is scanned with a fixed time window to obtain samples as the input of the CNN model. en, the model is used to identify its specific behavior category. After obtaining each behavior category, an individual behavior dataset is generated.

Wearable Sensors and Data Collection.
Wearable devices first appeared in the middle of the 20th century. e traditional method is to place sensors directly on the user to collect user motion data. e sensors used in data acquisition equipment mainly include accelerometers, gyroscopes, magnetometers, smartphones, barometers, pressure sensors, and EMG signal detectors, among which accelerometers and gyroscopes are more common. e accelerometer mainly measures the force of the device and can sense acceleration in any direction; the gyroscope judges the current movement state of the device through its rotation state (angle and angular velocity). With the development of smart devices, researchers generally design embed sensors in mobile devices. Chacon et al. designed a new pulse oximeter by embedding sensors in portable medical equipment, providing real-time monitoring of the physical condition of the elderly [14]. For the feature extraction of wearable sensor data, complex behaviors in general life can be expressed through various basic action combinations. For example, eating snacks includes several basic actions such as holding and eating.
Gu et al. introduced the emerging pattern to describe the discriminative pattern of significant changes between data categories and identified sequential, interlaced, and concurrent activities in a unified framework, thereby identifying sensor features and classifying activities [15]. Wang et al. proposed a hierarchical recognition method to infer ongoing activities in a multistage process to better distinguish similar activities and improve overall performance [16]. In terms of complex behavior recognition,  Chang et al. considered the interaction between people and the surrounding environment using sensors distributed on people and objects to collect behavioral data, which improves the extraction of data features of complex behaviors [17].

Data Feature Extraction.
In human behavior recognition, the most representative algorithm is the improved dense trajectories (IDT) algorithm [11]. It is a dense optical flow trajectory algorithm based on dense trajectories (DT). Before CNN was applied, IDT was the best algorithm for human behavior recognition. With the development of CNN, Ye et al.
proposed the two-stream method to extract behavioral features in video space and time. Compared with IDT, the twostream method is more robust [18]. e convolutional 3D method proposed by Torfi et al. can directly process video [19]. e effect is very good in spatiotemporal feature extraction. is method effectively improves the recognition speed, but the accuracy is low. Jin et al. put forward the timesensitive network (TSN) method [20]. By segmenting the long video, the problem of the inability to model the long-term structure in the two-stream method is solved, but the TSN method has a complex structure. e recognition speed is slow. Wu et al. discussed the deep local video feature (DLVF) method based on the TSN method, assigning different weights to different segments in TSN and fusing the judgment results of different segments more reasonably [21]. Chanti et al. envisioned a behavior recognition method based on temporal and spatial weighted gesture motion characteristics through dynamic time warping and Fourier time pyramid algorithm modeling, and finally improved the accuracy of human behavior recognition [22].

Wearable Sensor Data Feature Extraction
Method Based on CNN e data feature extraction studied in this paper is suitable for sensors installed on people, animals, and objects, but the sensors must move with the appendages so that the relevant data features can be extracted, and their behaviors can be analyzed to obtain a behavior set. For example, when a person is running, their running behavior is recognized through the data characteristics of parameters such as legs, hands, and heart rate.
For the extraction of wearable sensor data features, the acceleration data, angular velocity data, and magnetic field data output by the wearable sensor are first fused by the extended Kalman filter fusion algorithm. en, the preliminary state estimation of the human behavior represented by the wearable sensor output data is obtained, and the CNN model is used to process the collected data in parallel to get the result of human behavior recognition.

Data Fusion.
At present, the commonly used human behavior data feature decomposition methods are estimation methods based on Kalman filtering, which mainly include unscented Kalman filtering, extended Kalman filtering, and volume Kalman filtering [23]. e estimation methods based on non-Kalman filtering include the complimentary filtering algorithm and the gradient descent algorithm. rough the comparison of several methods, this paper selects the extended Kalman filter method to identify the behavior of the sensor. e Kalman filter is an algorithm designed for linear systems at the beginning of its design and cannot be applied to nonlinear systems, but in practice, most systems are nonlinear, and it is impossible to directly use the Kalman filter to solve nonlinear systems. In response to this problem, scholars began to transform the Kalman filter to improve and optimize its performance so that it can be applied to nonlinear systems. In this process, an extension of the Kalman filter to nonlinear systems has been produced.
is paper uses the data fusion algorithm based on the extended Kalman filter to realize the human behavior recognition capability of the inertial sensor. Figure 2 shows the specific implementation process of the data fusion algorithm based on the extended Kalman filter. As shown in the figure, the system has three different types of input data. e gyro data is measured in deg/s, and the angular velocity is measured on three axes. e accelerometer data measures acceleration in g.
e magnetometer data measures the magnetic field in Gaussian units. e gyro data is used as an input to update the prior state estimation. At the same time, the accelerometer and magnetometer data are used as a fixed reference to correct the error in the prior state estimation. e following is the specific implementation process of this method.
In the data fusion algorithm based on the extended Kalman filter, it is defined that n { } represents the navigation coordinate system and b { } represents the sensor coordinate system. First, the initial Euler angle is calculated using the accelerometer and magnetometer data acquired at the initial moment. e initial pitch angle x 0 and the initial roll angle y 0 are obtained from the acceleration data, and the initial yaw angle z 0 is obtained from the magnetic field data which are as follows: where a k � [a x , a y , a z ] T defines the acceleration data at the initial moment, and m k � [m x , m y , m z ] T presents the magnetic field data at the initial moment. rough the conversion between the Euler angle and a quaternion, the quaternion of the initial moment state can be obtained. e quaternion is a four-dimensional vector that can express various poses in space without any singularity. e posture represented by the quaternion can be understood as a coordinate system rotated by another coordinate system. en, the final coordinate relative to the original coordinate can be regarded as a rotation of the original coordinate, assuming that the angle of rotation is θ, e state transition equation in the continuous domain mainly uses quaternion differential equations, which can be defined as follows: where _ q represents the differential of the quaternion, Ω(ω t ) is the rotation matrix derived from the nature of the quaternion, expressed by the angular velocity data ω t � [ω x , ω y , ω t ] T measured by the gyroscope at the time t and has the following form: e application field of the Kalman filter is the discrete domain. For this reason, it is necessary to convert the quaternion differential equation in the continuous domain into the discrete domain. By displaying the derivation, the continuous-time system can be rewritten as follows: To transform this equation into a discrete-time equation, the time step T between each execution of the algorithm is used in the digital system. In this way, we can know the following: Because the bias of the gyroscope is also considered in the system, this paper defines the state vector of the system as a unit quaternion and gyroscope bias. e update equation of the state vector can be obtained through the abovementioned analysis, which is as follows: where q t is the optimal estimated value of the quaternion of the unit behavior at the previous moment, q − t+1 is the suboptimal estimated value of the quaternion of the unit behavior at the current moment, and Φ t is the state transition matrix. e specific expression is as follows: e quaternion norm is not retained in the state equation. Since the angular position can only be correctly represented by a unit quaternion, it is necessary to introduce a normalization unit to ensure the integrity of the data.
e Jacobian matrix of the state transition equation is solved as follows: It is solved by assuming that the noise between each axis of the accelerometer, gyroscope, and magnetometer is independent of each other. In this system, the state transition matrix that the system relies on is described by the gyroscope data, so the process noise covariance matrix directly depends on the gyroscope. In the measurement model, the measurement noise comes from the noise of the accelerometer and magnetometer. erefore, the process noise covariance matrix Q t and the measurement noise covariance matrix R t can be determined as follows: In the abovementioned formula, ω n is the gyroscope measurement noise, ω bn is the gyroscope bias noise, a n is the accelerometer measurement noise, and m n is the magnetometer measurement noise. erefore, the prior error covariance at time t can be further solved as follows: P t−1 represents the posterior error covariance matrix of the previous filter iteration. e behavioral state quaternion obtained by (6) is calculated from the output data of the gyroscope and the optimal behavioral state at the previous moment. It is the behavior obtained during the long-term operation by relying only on the output of the gyroscope. e state is not reliable, so it is necessary to fuse the accelerometer and magnetometer data to modify the state to obtain the final optimal behavior.
To calculate the measurement equation of the magnetometer, first convert the magnetic field data from the sensor coordinate system to the navigation coordinate system to obtain the magnetic field data in the navigation coordinate system. m b represents the magnetic field data in the sensor coordinate system, and m n represents the magnetic field data in the navigation coordinate system, which are as follows: In the measurement model, it is assumed that the magnetic field in the navigation coordinate system has no data in the east direction and only in the north and sky directions. After the magnetic field data is processed and then converted to the carrier coordinate system, the measured value m b of the magnetic field strength can be obtained, which is as follows: e measurement equation of the accelerometer is calculated. In the navigation coordinate system, the reference direction of the gravity field is g � 0 0 1 T , and the measured value of acceleration a b is obtained from the equation: e obtained magnetic field data and acceleration data are normalized, and the processed data is denoted as Z t . Finally, the total measurement equation can be obtained from the following equation: e Jacobian matrix of the measurement equation is calculated as follows: rough the calculated prior error covariance, the Jacobian matrix of the measurement equation, and the measurement noise covariance matrix, the Kalman gain can be calculated as follows: e posterior error covariance matrix is updated, which is as follows: Finally, a more accurate posterior state estimate can be obtained, which is as follows:

Feature Extraction of Sensor Data Based on CNN.
e basic calculation process of the CNN designed in this paper is shown in Figure 3, where N pairs of feature data are taken and weight parameters are considered as input. e corresponding N pairs of data are multiplied, and then the N results of the multiplication are accumulated.
Convolution calculation is the largest calculation part of the CNN model. To improve the computational efficiency of convolution calculations, this paper uses multiple channels and multiple filters to perform calculations in parallel. e specific process is shown in Figure 4. In the figure, the input feature data size is H in × W in , the convolution kernel size is K x × K y , the input channel is CH in , the number of convolution kernels is K, and N is the channel parallelism.  Journal of Healthcare Engineering

Journal of Healthcare Engineering
Multichannel parallelism cuts the feature data and each weight data into multiple sub-blocks along the input direction. e input channel dimension of each sub-block is N, and N corresponding data is taken out from the feature data sub-block and weight sub-block, respectively. Multifilter parallelism is the simultaneous calculation of input weights and multiple convolution kernels. Figure 4 shows the process of the sub-block convolution operation. e specific calculation process is as follows: (1) As shown in Figure 4(a), the feature data and the first data in the n channel directions of the 2 convolution kernels are taken out at the same time, and the corresponding data taken out respectively are multiplied, then accumulated in the channel direction, and the result is output. (2) As shown in Figure 4(b), the weight remains unchanged, and the feature data is slipped, so that the first data on the N channels of the weight is multiplied by part of the feature data and accumulated in the direction of the channel. (3) As shown in Figure 4(c), the weight data is changed so that the calculation can cover all the feature data and weight data of the sub-block, and all the data of the 3×3 convolution kernel on n channels and the entire feature data are calculated. e result of multiplying the feature data corresponding to the 9-weight data of the 3×3 convolution kernel and accumulating in the channel direction. (4) e next N channel sub-block is taken along the channel direction to repeat the abovementioned calculation, and the result is accumulated to output the characteristic data. e maximum pooling calculation is shown in Figure 5. e feature data is divided into multiple n × n sub-blocks along the length and width, and the maximum value is taken in each sub-block. e length and width of the output feature data are the original feature data lengths. And the width of 1/n.
Pooling is an operation used to reduce the size of data. As shown in Figure 6, to improve the computational efficiency of the pooling module, this paper designs two subfunctions, horizontal pooling and vertical pooling, and uses pipeline parallel calculation to obtain the output results. At the same time, multichannel parallel acceleration is performed on each subfunction.
Assuming that the input feature data is pooled with the maximum value of n × n and the step size is also n, the specific calculation process is as follows: (1) As shown in Figure 6(a), the input feature enters the horizontal pooling module, the first data is compared with the subsequent input n − 1 data, and the maximum value is recorded. Take the input n data as a group, and repeat the abovementioned operations to obtain the result of horizontal pooling. Currently, the height of the intermediate data remains unchanged, and the width is 1/n of the original width. (2) As shown in Figure 6(b), first record the first W out data of the intermediate feature input, and compare the n − 1 sets of W out data input later with the first set of data at the corresponding positions to obtain the maximum value, and output the result. Take the input n groups of W out data as a group.

Results and Discussion
e experiment in this paper is based on the three typical datasets of MHEALTH, WHARF, and USCHAD in the UCI_HAR dataset.
ese three datasets based on wearable devices are represented in the field of human behavior recognition and can reflect the overall performance of this method. To better illustrate the advantages of the method in this paper, experiments were also carried out on two relatively new datasets, the Stanford-ECM Dataset and DATAEGO [24,25]. Among them, the Stanford-ECM Dataset was published in 2017. e DATAEGO dataset was made publicly available in 2018. Since the dataset may have problems such as unbalanced sample distribution, and these problems may cause overfitting of the model, it is necessary to prove through experiments whether the improvement in model performance is caused by overfitting. is paper also conducts experiments based on the dataset of the Stanford-ECM Dataset and analyzes the reasons for its performance improvement. To verify the versatility of the method, this paper also uses the network in the classification part of the zero-sample learning task. e semantic relationship parameters between the visible class and the unknown class and the prototype features of the visible class are synthesized as the virtual prototype features of the unknown class. e input and output of the network model are the semantic attributes of each category. e category is determined according to the semantic attributes. Furthermore, this is compared with the method proposed by Chen et al. to verify the performance of the algorithm in this paper [26].

Experimental Evaluation Index.
In the experiment of this paper, three evaluation indicators are used, namely, precision A, recall rate R, and F 1 value. In the field of machine learning, A is the most common evaluation index, which indicates how many of all samples that the model judges to be positive are true positive samples, which is as follows: where TP defines a real example, which means that the positive sample is predicted to be the number of samples in the positive class; TN presents a true negative example, which shows that the negative class sample is predicted to be Journal of Healthcare Engineering the number of samples in the negative class; FP describes a false positive example, which describes that the positive class sample is predicted to be the number of samples that are predicted to be negative; FN defines a false negative, which explains the number of samples that are predicted to be positive. R, known as recall rate, which indicates how many positive samples are judged by the model as positive samples, is also one of the common evaluation indicators in the field of machine learning. Its calculation method is as follows: e function of the F 1 value is to consider both A and R when evaluating the model and balancing precision and recall. It is also one of the commonly used evaluation indicators. e calculation method is as follows:

e Process of Sample Generation and the Division of
Datasets. For sensor data, samples are obtained by sliding on the original data by setting a fixed time window. ere are two ways for the time window to slide on the original data: one is to slide without overlap and the other is to slide with half overlap. e two sample generation methods have their advantages and disadvantages. Among them, the main advantages of generating samples in a nonoverlapping manner are that there is no overlap between the samples and that the experimental results will not cause deviation, but the number of samples generated in this way will be relatively small. With half overlap, the advantage of the method of generating samples is that it can generate as many samples as possible, but its disadvantage is that overlap will bring high deviations, resulting in better results than actual. When training the model, the dataset needs to be divided into a training set, a validation set, and a test set. Since the three datasets used in this paper have user information, crossvalidation is used to train the model in the process of training the model. at is, one user data is reserved for model verification in each training, and the data of other users is used for the model verification train. Doing so means that each user's data will be used as a validation set, which means that multiple experiments are required. Considering that there may be situations where the quality of the data collected by the user is low or beyond expectations, the maximum and minimum values are removed in the final evaluation of the model, and the average value is taken to evaluate the performance of the model more comprehensively.

Experiment and Result Analysis.
is paper conducts experiments on the five abovementioned datasets and compares different methods. To ensure comparability between the methods, this paper sets the same experimental conditions and experimental end conditions during the training model.
at is, the training model process ends when the loss value is less than or equal to 0.2 or after 200 iterations of training are completed. is paper uses Adadelta as the optimizer, and the Softmax classifier as the classifier. e learning rate of the model is 0.001, and the maximum number of iterations is 200.
It can be seen from Table 1 that compared with the method of Chen et al., the method in this paper has obvious advantages in the three commonly used evaluation indicators. is fully illustrates the necessity of data fusion and the reasonable setting of convolution kernels in data feature extraction for such tasks. e method in this paper has achieved better results on the MHEALTH dataset, and the accuracy reaches 94.08%, which is 2.21% higher than Chen et al.'s method, which is enough to prove the advantage of this method. In addition, the method in this paper also has obvious advantages in the WHARF dataset and the USCHAD dataset, but the effect is not significantly improved in the MHEALTH dataset. e reason is that the dataset is collected in different ways. e MHEALTH dataset has strict data during the collection process. e collection standard regulates the collection action. However, the WHARF dataset and the USCHAD dataset did not develop such strict collection standards during the collection process. Figure 7 shows the experimental results of the two methods on the Stanford-ECM Dataset. It can be seen from Figure 7 that the method in this paper has achieved better results in 8 categories. With the same results of the method, 5 categories have achieved slightly worse results. However, the overall performance is still significantly improved, indicating that the performance improvement is not caused by the data distribution but the function of the method itself.

Conclusions
is paper proposes a method for feature extraction of wearable sensor data based on CNN. is method first uses a Kalman filter to fuse the data to obtain a preliminary state estimation, and then uses CNN to recognize human behavior and obtain the corresponding behavior set. is paper conducts experiments on five datasets. e experimental results show that this paper can extract data features at multiple scales while fully maintaining data independence and can effectively extract corresponding feature data. In addition, the method in this paper has strong generalization ability and can adapt to different learning tasks. At present, wearable devices are mainly smart watches and bracelets, concentrated in medical and health, outdoor sports, audiovisual entertainment, and other fields. With the intensification of society's aging phenomenon, the wearable device market will usher in tremendous development in areas where health is the appeal. In the future, the accurate extraction of data characteristics of wearable sensors can be used to analyze users' various behaviors, the health index, and other data, which also provides strong support for disease prevention and treatment, health monitoring, and other aspects. At the same time, the accurate extraction of wearable sensor data features can complete various tasks such as object detection, face recognition, and image analysis, bringing a brand new experience to users. e accurate extraction of data features of wearable sensors can provide humans with more accurate and reliable basic datasets. rough these datasets, more intelligent and digital behavior parameters can be provided for artificial intelligence and help the development of artificial intelligence.
Data Availability e datasets used and/or analyzed during the current study are available from the author on reasonable request.

Conflicts of Interest
e author declares no conflicts of interest.