Gait Recognition for Human-Exoskeleton System in Locomotion Based on Ensemble Empirical Mode Decomposition

As exoskeleton robots are more frequently applied to impaired people to regain mobility, detection and recognition of human gait motions is important to prepare suitable control modes for exoskeletons. +is paper proposes to explore the potential of the ensemble empirical mode decomposition (EEMD) method to help analyze and recognize gait motions for human subjects who wear the exoskeleton to walk.+e intrinsic mode functions (IMFs) extracted from the original gait signals by EEMD are utilized to act as inputs for classification algorithms. Evident correlations are found between some IMFs and original gait kinematic sequences. Experimental results on gait phase recognition performance on 14 able-bodied subjects are shown. +e performance of the composing signals extracted from the original signals as IMF1 ∼ IMF8 is investigated, which indicates that IMF8 might be helpful when wearing exoskeleton and IMF5 might be helpful when walking without exoskeleton on gait recognition. And the similarity of joint synergy between wearing and without wearing exoskeleton is analyzed, and the result shows that the joint synergy might change between with and without wearing exoskeleton. +e quantitative results show that based on some IMFs of the same orders, these machine learning algorithms can achieve promising performances.


Introduction
Exoskeletons are encountered increasingly and frequently in assisting the movement of people in various aspects, helping them regain locomotion ability and reconstructing their lost gait. In order to achieve better gait transition for exoskeletons with different gait modulation modes switched, it is important to make accurate recognition of different gait phases for the human-exoskeleton system. In the past decades, researchers and engineers have developed a variety of methods for detecting and recognizing human gait with and without wearable robots. Aertbelien and Schutter developed a statistical model method to learn the joint gait trajectory and the variations for the control of the lower limb exoskeleton [1]. Brinker et al. proposed a gait recognition model that can fast adapt to a novel user's movements in exoskeleton control by using previously collected other users' data and generate one side of leg exoskeleton movement based on the other side of the leg movement [2]. Kim et al. presented a novel shape-based and model-based combined gait recognition algorithm by extracting modebased gait cycle based on the prediction-based hierarchical active shape mode [3]. Recently, Torricelli et al. presented a methodology to predict the human joint motion based on exoskeleton motion by combining personalized skeletal models of human subject with a kinematic model of the exoskeleton [4].
As some previous works mentioned, the human-exoskeleton system is a hybrid and complicated cognitive dynamics system. In particular, different subjects with exoskeletons may present different physical and dynamic parameters to be identified, which makes the whole dynamic motion model hard to establish and personalize [5]. When the control system of exoskeleton wants to perform precise control tasks, it needs to know the accurate gait status of the human-exoskeleton system [6]. However, classification and recognition of gait phase through primal modeling and identification of the dynamic human-exoskeleton may be cumbersome. In this instance, utilization of some suitable machine learning algorithms driven by immediately acquired gait data might be feasible for the exoskeleton system to recognize different gait motions.
Since it is quite difficult to measure multiple joint torques and ground reaction forces simultaneously for the human-exoskeleton system, employing gait data acquired by motion capture devices is more convenient for the control system to process and configure [7]. Due to different influence factors such as subjects' biomechanics and physiological movement diversity, gait kinematic signals can be still characterized as nonlinear and nonstationary signals [8].
us, it would be suitable to use the empirical mode decomposition (EMD) method to analyze and extract the potential features concealed in gait data, as an alternative [9]. Some researchers have utilized such pathway to analyze gait signals [10]. For example, Cui et al. proposed a novel concept as step stability index (SSI) which evaluates subjects gait stability and fall risk by utilizing the decomposed the 1st ∼ 4th intrinsic mode functions (IMFs) from its original signals [11]. Wen et al. found a new way based on improved EMD using Gaussian process to denoise gait accelerator data, and it gets better performance on denoising gait data [9]. Ren et al. discovered a good performance of neurodegenerative disease characterization by utilizing EMD to extract gait rhythm features from gait rhythm fluctuations which is based on signals of vertical ground reaction force [12]. Wang et al. achieved to characterize and differentiate the single waist accelerator signals through EMD between 5 gait patterns with high accuracy [13].
In this paper, we propose a framework of gait phase recognition for a human-exoskeleton system based on EMD. In such EMD-based recognition framework, eight IMF components are extracted from the original gait signals of 14 able-bodied subjects whose kinematics data are acquired by the optical-marked motion capture device. Six kinds of machine learning algorithms are applied for gait phase classification and recognition as follows: support vector machine (SVM), Kmeans, decision tree, logistic regression, Naive Bayes, and random forest methods. e extracted IMFs are used as the input for the training models with these six algorithms.
e gait recognition results based on the 5th ∼ 8th IMFs with some of these algorithms generally show promising performance. In the meanwhile, nonlinear fitting is used to examine the correlation between the decomposed IMFs and the original gait signals, and such fitting results could help to improve quantitatively evident observations on the correlations between IMFs and original gait data addressed in [11]. e difference and similarity of joint synergy between wearing with or without exoskeleton is analyzed. To our best knowledge, there is little work specifically focusing on human-exoskeleton gait phase recognition based on the EMD method in such manner.

Materials and Methods
In this section, experiments are divided into two parts. In the 1st part, kinematics data with and without exoskeletons during locomotion for the 14 subjects are measured by the optical motion capture system. In the 2nd part, another 3 subjects' joint angles are captured by the inertial measurement unit (IMU) system. For the 1st part and 2nd part of experiments, IMFs are extracted from the measured original kinematics data by ensemble empirical mode decomposition (EEMD) algorithm, and the method of evaluation of the correlation between them is proposed. For the 1st part experiments, joint synergy is extracted from joint angle data.

Experiment Setup.
In the 1st part of experiments, 14 able-bodied subjects (all males, 22.88 ± 1.32 years old, 173.65 ± 5.22 cm height, and 54.59 ± 5.21 kg weight) took part in optical-object motion capture experiments to capture their kinematics data with and without exoskeletons during locomotion. In the 2nd part of experiments, another 2 ablebodied subjects and 1 paraplegia subject (all males, 25.66 ± 2.44 years old, 177.10 ± 19.33 cm height, and 62.13 ± 1.91 kg weight) also participated in the IMU-based motion acquisition experiments to measure their joint angles. All motion capture studies were performed with their consent. e experiments followed the institutional guidelines of the University of Electronic Science and Technology of China, and all the experiment operations were in accordance with the Declaration of Helsinki. ey were all instructed to utilize the lower limb exoskeletons to perform the normal walking tasks during part of the optical-object motion capture experiments. e lower limb exoskeleton system used in the experiments was developed by the University of Electronic Science and Technology of China. Such lower limb exoskeleton system is with four active degrees of freedom (flexion/extension) of motion in hip and knee joints, and its ankle joints are with two passive degrees of freedom of motion (dorsi/plantar flexion). e subjects are required to use crutches to maintain the balance during their locomotion for safety reasons.
In the 1st part of experiments, during optical-object motion capture experiments, 39 infrared markers were attached to each subject's body for the VICON (Oxford Metrics Limited, UK) motion capture system according to the Plug-in Gait full body model [14]. e VICON system captured the locomotion of these 39 markers by 8 cameras with a frame rate of 100 Hz. e locomotion on elbow, shoulder, hip, knee, and ankle joints was acquired and converted to the angles of these joints through the algorithm provided by VICON. Each subject was required to perform 12 experiment sessions with different conditions, and a group of 30 joint angle signals (i.e., hip joint, knee joint, ankle joint, shoulder joint, and elbow joint at X, Y, and Z axes on both left side and right side, respectively) were captured correspondingly for each experiment session.
In the 2nd part of experiments, 7 IMU sensors (Perception Neuron, Noitom Limited) were bonded to the subjects lower body. e Perception Neuron system captured the locomotion of these 7 sensors with a frame rate of 120 Hz. Raw data on the angles of hip, knee, and ankle joints were captured.

Experimental Protocol.
In the 1st part of experiments, all the 14 subjects were required to perform walking tasks with wearing or without wearing exoskeleton on 6 different floors with different frictions and materials, which include normal floor, wood floor, ceramic floor, carpet, plastic lawn, and floor with cobble. e first session of the test for each subject was to let him walk on 6 different floor conditions wearing an exoskeleton, and the second session of the test was to let him walk on 6 different floor conditions without wearing an exoskeleton. ese two sessions are independent and separate. Each subject was told to walk about 5 m for several times for both test sessions. For each experiment in different floor conditions with or without wearing exoskeleton, we clipped some of experiment sessions when some of makers are found to be lost on camera during the motion capture. Figure 1 shows the subjects walking on the floor with different materials.

EEMD Computation and IMF Extraction.
In each experiment session, we need a group of 18 joint angle signals (hip joint, knee joint, and ankle joint at X, Y, and Z axes on both left side and right side, respectively) from 30 joint angle signals mentioned above to perform gait recognition. According to the EEMD design principle [10], the 18 original gait time signals x(t) for each experiment session are to be decomposed into the following n intrinsic mode function (IMF) components, respectively: where c j denotes the jth IMF component (i.e., termed as IMFj) decomposed from the gait sequence x(t) and r n is the residue of x(t). e EEMD algorithm for gait signals can be described by the following algorithm (Algorithm 1). Figure 2(a) shows one of the 18 joint angle signals generated by one experiment session and its decomposed 8 IMF signals. Figure 2(b) shows the FFT of Figure 2(a); this shows that with the increase of IMF order number, the frequency of components the IMF contains decreases.

Gait Recognition.
For the purpose of investigating how the various floor materials would affect the gait on subjects and how the different machine learning classifications and the traits of IMF would affect gait recognition when it comes to processing with VICON data in the part one experiments, we try to make gait recognition based on the 14 subjects' motion capture data, by applying classification/recognition approaches with the extracted IMF components. Before making recognition, first an algorithm is developed to tag gait phase status for every frame of dataset according to the signals of toe and heel marker on Z axis captured by VICON. e tagged phase status contains four tags as left and right foot both on land, left foot on land and right foot off land, left foot off land and right foot on land, and the uncertain mode, where gait phase status information was used to generate dataset Y as label set. en, we remove those frames labeled as uncertain mode in Y set and their corresponding frame in the VICON data. We make gait recognition on 3 perspectives: gait recognition modes A, B, and C.

Gait Recognition Mode A.
In this mode, we randomly divided the dataset which we acquired from each experiment session into two parts as training set and testing set with 2 : 1 ratio for the whole gait dataset of each experiment session. e original training dataset X and labeled dataset Y were composed of 8 IMFs as IMF1 ∼ IMF8 dataset. And we let the IMF1 ∼ IMF8 dataset signal train different machine learning models (SVM, Kmeans, decision tree, logistic regression, Naive Bayes, and random forest) and use the corresponding testing set to count the recognition accuracy.

Gait Recognition Mode B.
In this mode, for each subject, we set the dataset on the Normal Floor as the training set and let the others of the same subject as the testing set. Because one subject's experiment with the floor with cobble was abandoned, we got 69 accuracies for the original dataset gait recognition. en, the 8 IMFs as IMF1 ∼ IMF8 dataset were generated by decomposing the original dataset, and we let the IMF1 ∼ IMF8 dataset signal to evaluate different machine learning algorithms.

Gait Recognition Mode C.
In this mode, for each floor material, we set the dataset of the same subject as the training set and let the others on the same floor material as the testing set. Because one subject's experiment with the floor with cobble was abandoned, we got 77 accuracies for the original dataset gait recognition. en, we decompose the original data into 8 IMFs as IMF1 ∼ IMF8 dataset, and we let the IMF1 ∼ IMF8 dataset signal go through the same procedure as its original signal dataset X had been processed.

Evaluation of Correlation between IMF Components and
Original Gait Trajectory. For the kinematics data of the subjects measured by VICON, 12 experiment sessions in the 1st part of experiments are performed on each subject to capture and thus produce a group of 18 original gait signals (i.e., hip, knee, and ankle angles for X, Y, and Z axes on both left side and right side, respectively) for each experiment session. For the joint data acquired by IMU in part two experiments, 3 experiment sessions are performed on all 3 subjects, and thus we generate a group of 3 original gait signals (i.e., left hip, left knee, and left ankle). For all those original signals S n mentioned above, we divide them with dividing parameter p ∈ [0, 1] into S 1,n and S 2,n . It means that S 1,n is the former p part of S n and S 2,n is the latter (1 − p) part of S n . And after the IMFs (IMF1 n ∼ IMF8 n , n ∈ 1, 2, . . . , 12 × 18 { }) are extracted from these original gait Mathematical Problems in Engineering (1) For different N noisy signals w i (t), i ∈ N, we generate different x i (t) and calculate its corresponding decomposing results, IMF1 i ∼ IMF8 i . is means we extract IMFs for N trials for different noisy signals w i (t). (1) in the whole dataset, the number of extrema and the number of zero crossings must either equal or differ at most by one; (2) at any point, the mean value of the envelope defined by the local maxima and the envelope defined by the local minima is zero.
which is a cubic spline line connecting all the local maxima of h j,k (t) and l k (t) is the lower envelope of h j,k (t) which is a cubic spline line connecting all the local minima of h j,k (t) (8) end while (9)   Mathematical Problems in Engineering signals S n , IMF1 1,n ∼ IMF8 1,n and IMF1 2,n ∼ IMF8 2,n are also generated, respectively, for two data sequences S 1,n and S 2,n . In order to investigate the correlation between the original gait signals and IMFs by cross validation in a nonlinear fitting manner, first, we construct the matrix U n for each S n as follows: (2) Next, we produce a number of new signal matrices S (3) us, we describe the variance between S 2,n and S 2,n by index VAF n (variance accounted for) counted through the following equation: As VAF n > 0 is closer to 1, it indicates that they may possess more correlation evidence.

Joint Synergy Extraction.
In addition, it is also important to further compare the joint synergy for subjects between them with wearing exoskeleton and without wearing exoskeleton to walk. We adopt principal component analysis (PCA) to extract the principal components of the five joints' angle data (i.e., ankle, knee, hip, shoulder, and elbow joints' angle) in the part one experiments. Some researchers have found that utilizing PCA to analyze the movement coordination patterns for human-exoskeleton might be useful [15,16]. Firstly, we generate the following joint motion matrix D acquired for each individual.
where D j (j ∈ ANK, KNEE, HIP, SLDR, ELBW { }) denotes the data about the degree of ankle, knee, hip, shoulder, and elbow joints which contain the information about joint synergy. e method of PCA creates a new set of variables called principal components. Each principal component is a linear projection of the original variables. We can generate PCA through the following equations: Mathematical Problems in Engineering where Σ denotes the covariance of the matrix D we constructed, α l denotes the direction that the original variables project on and also is the lth eigenvector for matrix Σ, and λ l is the lth principal component corresponding to α l .

Results and Discussion
In this section, the results of gait recognition modes A, B, and C for the human-exoskeleton system based on EEMD with the six machine learning methods are presented. e influence of floor materials on gait recognition has also been analyzed. Additionally, joint synergy for the subjects who walk with and without exoskeletons is also drawn. Figure 3 shows the results of gait recognition mode A for the decomposing signals IMF1 ∼ IMF8 of the original signals based on all the six machine learning methods (SVM, Kmeans, decision tree, logistic regression, Naive Bayes, and random forest) for the 14 subjects who walked on flat floors with different friction situations with and without exoskeletons. Tables 1 and 2 show the detailed information of Figure 3. In this mode, we could figure out the performance for the six learning algorithms recognizing the gait status on each experiment session. According to these two subfigures in Figure 3, recognition performance with IMF5 ∼ IMF8 can be generally better than that with IMF1 ∼ IMF4. From Tables 1 and 2, conclusion can be made that when IMF1 ∼ IMF4 are used for the six algorithms, the recognition performance is not satisfactory since the accuracy generally cannot reach 80%. e performance of IMF5 ∼ IMF8 is generally good for recognition. ese comparative results might indicate that IMF5 ∼ IMF8 may be considered to be involved in the recognition algorithms. Besides, in these results, especially when SVM, Kmeans, logistic regression, decision tree, and random forest approaches are applied for IMF5 ∼ IMF8, promising recognition accuracy can be generated. Otherwise, in Tables 1 and 2, we can see that when we know the gait mode for someone in certain floor material, the gait recognition can reach 98% through IMF6, which means that IMF contains information for gait recognition. Furthermore, Figure 4 shows the frequency that the maximum accuracies may occur in each order of IMF in all experiments, which stands for the probability that each IMF contributes to the recognition accuracy for the six algorithms. We generally can see that IMF7 and IMF8 have a higher probability to enhance the recognition accuracy for subjects who walk with exoskeletons, and IMF4 ∼ IMF8 can have a higher probability for subjects who walk without exoskeletons. As we can see, the signal in IMF1 ∼ IMF8 that can get the highest probability changes between wearing exoskeleton and without wearing exoskeleton. e reason behind this phenomenon may be that the gait locomotion mode changes between wearing exoskeleton and without wearing exoskeleton.

Gait Recognition Mode A Results.
In conclusion, we can see that when it comes to recognizing gait with VICON data for each experiment session, all six algorithms are useful for recognition, and IMF7 ∼ IMF8 might be useful for recognition when wearing exoskeleton, while IMF4 ∼ IMF8 might be helpful for recognition when walking without wearing exoskeleton. Figure 5 shows the results of gait recognition mode B with the original signal's decomposing signals IMF1 ∼ IMF8 for the same subject and various floor materials. Tables 3 and 4 show the detailed information of Figure 5. In this mode, we could evaluate the performance of the learning algorithm when we acquire one subject's gait data in one floor material to establish the recognition model and try to recognize the same subject's gait on other floor materials based on the model. Kmeans could get good performance when wearing with or without exoskeleton as their highest recognition average accuracy all exceed 80% when with or without exoskeleton in both Figure 5(a) and 5(b).

Gait Recognition Mode B Results.
When subjects wear exoskeleton, we can see that for different gait recognition algorithms, generally the higher the IMF order number is and the higher the accuracy is in Figure 5(a).
us, the accuracy of IMF8 is the highest  (80.94%) with Kmeans according to Table 3. Kmeans, logistic regression, and random forest have a good performance with IMF8 in Table 3. Figure 5(b) shows that when subjects walk without exoskeleton, generally IMF4 ∼ IMF6 have relatively better performance among IMF1 ∼ IMF8, and IMF5 is the highest (80.24%) with Kmeans. Additionally, Figure 6 shows the probability for IMF1 ∼ IMF8 that each IMF may be the highest accuracy among IMF1 ∼ IMF8 for the six algorithms. From Figure 6, we could generally conclude that IMF8 has the highest probability to get the maximum accuracy among IMF1 ∼ IMF8 when wearing exoskeleton and IMF5 is most likely to get the maximum accuracy among IMF1 ∼ IMF8 when walking without exoskeleton. Signal in IMF1 IMF8 that can get the highest probability changes between wearing exoskeleton and without wearing exoskeleton. As with mode A, signal in IMF1 ∼ IMF8 that can get the highest probability in mode B changes between wearing an exoskeleton  and without wearing exoskeleton. is shows again that gait locomotion mode may change between wearing exoskeleton and without wearing exoskeleton.
In general, the result of this section tells us that when we tend to utilize someone's gait data on the normal floor as the model to recognize the same subject's gait in other materials, Kmeans may be a more helpful algorithm to analyze gait recognition among the six algorithms. Also, IMF8 may be more helpful for analyzing gait recognition when subjects walk with exoskeleton on flat floor, and meanwhile IMF5 may be more useful to recognize gait among IMF1 ∼ IMF8 when subjects walk without exoskeleton on flat floor.

Gait Recognition Mode C Results.
e results of gait recognition mode C that the decomposing data of the original data as IMF1 ∼ IMF8 for the same floor material and various subjects can be analyzed from Figure 7. Tables 5  and 6 show the elaborate information for Figure 7. In this   mode, we can compare the performance of different machine learning algorithms when we establish the recognition model by one subject experiment session and recognize gait with the same floor material in different subjects.
When it comes to the performance of IMF1 ∼ IMF8 while wearing an exoskeleton in Figure 7(a) and Table 5, Kmeans is also the best algorithm compared to others and IMF8 contains the highest accuracy among IMF1 ∼ IMF8 with Kmeans (69.18%). In Figure 7(b) and Table 6, we can see that Kmeans performs best and IMF5 gets the highest accuracy among IMF1 ∼ IMF8 with Kmeans (72.87%).
Furthermore, Figure 8 shows the probability for IMF1 ∼ IMF8 that each IMF may be the highest accuracy among IMF1 ∼ IMF8 for the six algorithms. In Figure 8(a), we could say that the most likely to get the maximum accuracy among IMF1 ∼ IMF8 when wearing exoskeleton is IMF7 and IMF8 especially for Kmeans, Decision tree, logistic regression, and random forest. From Figure 8(b), we   can see that IMF4 ∼ IMF6, especially IMF5 can get the maximum accuracy among IMF1 ∼ IMF8 when walking without exoskeleton. In sum, from the result of mode C, we could generalize that when we try to recognize various subjects' gait based on another subject with the same floor materials, Kmeans may be better to perform gait recognition among the six algorithms on mode C. IMF8 may be more helpful for analyzing gait recognition when subjects walk with exoskeleton on flat floor, and meanwhile IMF5 may be more useful to recognize gait among IMF1 ∼ IMF8 when subjects walk without exoskeleton on flat floor. is is the same as mode B.

Correlation Analysis between IMF Components and
Original Gait. As addressed in previous gait recognition results, we see that there might be potential evident correlation between the latter IMFs and the original signals. Now the aforementioned nonlinear fitting approach is adopted to investigate the trajectory correlation between the latter IMFs and the original signals. Table 7 shows the VAF results for finding correlation between the IMF combination and the original gait signals acquired from the VICON system on the 14 subjects in part one experiments and IMU sensors on the 3 subjects in part two experiments. We can see that IMF4 ∼ IMF8 of some combinations can be used to identify the original gait signals. It reveals that the combination of IMFs may be utilized to identify the original gait trajectory.

Joint Synergy
Results. In addition to gait motion recognition results, in order to investigate the possible similarity of joint synergy, we performed PCA on the five joints: elbow, shoulder, hip, knee, and ankle joints, to extract the principal components which can represent joint synergy indexes. For these five joints on X axis, from Figure 9, we could see the comparison on the joint synergy for the subjects who walk with and without exoskeleton.

Conclusions
In this paper, EEMD-based gait recognition modes A, B, and C are proposed for the human-exoskeleton system. SVM (support vector machine), Kmeans, decision tree, logistic   regression, Naive Bayes, and random forest methods are used as training algorithms for such framework, and the performance of these six algorithms is discussed. us, we estimate how various algorithms, IMF order number, different floor materials, and various subjects would affect the result of recognition. And the results show that when it comes to gait recognition among various subjects and various floor materials, Kmeans might be better on performance whenever with or without wearing exoskeleton. For the contribution of gait recognition with IMF1 ∼ IMF8 among various subjects and various floor materials, IMF8 might be helpful when wearing exoskeleton, and IMF5 might be useful when walking without exoskeleton. And floor materials have little influence on gait recognition. e correlation of original gait data and their decomposing signal IMFs through EEMD is investigated, which reveals that the combination of IMFs may be utilized to identify the original gait trajectory.
At last, joint synergy of five joints for the subjects who walked with and without exoskeletons is also drawn, which showed that the joint synergy might change between with and without wearing exoskeleton.
Data Availability e VICON and IMU data used to support the findings of this study were supplied by Jing Qiu under license and so cannot be made freely available. Requests for access to these data should be made to Jing Qiu, qiujing@uestc.edu.cn.

Conflicts of Interest
e authors declare that they have no conflicts of interest.