Robust Personal Identi ﬁ cation Using Wearable Devices Based on LSTM and CNN

,


Introduction
According to the Personal Information Protection Act [1], it is stated that the personal information manager cannot handle "unique identification information" without the separate consent of the information subject. Here, the unique identification information may include a social security number, a passport number, a driver's license number, and so on. In addition, it is emphasized that personal information such as emotion, ideology, or religion, including personal information, should be classified as sensitive information, and efforts should be made to prevent infringement or leakage of the information. When we deal with sensitive data, the most important thing is that a clear method for identifying individuals should be presented.
A typical method used to distinguish whether a user is a legitimate person is a method using personal biometric data such as fingerprint, iris, and face shape. There is also the possibility that human gait can be used as a means of a new user authentication system that distinguishes individuals. As such, gait analysis is used in various application fields such as personal health status check, criminal detection through CCTV image analysis, and new authentication means of security systems.
Inertial sensors are attached to the person's body to collect walking data. By analyzing the data collected in this way, the gait pattern of the subject is found. A gait analysis method using a CCTV or infrared camera requires a special experimental environment or an expensive image processing device, while the price of inertial sensors is relatively low. In addition, recent advances in wearable technology have made wearable devices essential consumer devices for healthcare, fitness, location tracking, and many other uses, including the ability to effectively manage and control long-term conditions such as Parkinson's disease, stroke, diabetes, and dementia [2]. For this reason, recent gait analysis studies tend to mainly use these sensors. In most gait analysis studies using inertial sensors, the task of extracting the gait cycle from the collected gait data is performed first. The gait cycle is defined as from one heel strike (HS) to the next HS of the same foot. In one gait cycle, the heel contact of both feet and the toe off (TO) occur repeatedly. That is, it is necessary to identify the gait cycle to understand the individual's gait pattern and characteristics.
In this study, the personal identification system that we proposed aims to identify a specific individual by training walking data and the gait cycle (HS and TO). Numerous studies have been conducted on personal identification utilizing deep learning models [3][4][5][6][7][8][9][10][11][12][13]. In previous studies [7][8][9][10], there were limitations in not properly reflecting reality in the data collection environment, which will be discussed further in Section 2. For example, there was no change in the type of shoes used when collecting gait data, or the experiment was conducted using only gait data collected on flat ground [8]. In a real environment, various types of shoes may be worn, and different environments might exist. Even the same person can produce different results depending on the walking environment. This study identifies individuals from walking data collected by replacing the shoes of walking testers, a method that has not been applied in real-world environments in previous studies. We constructed CNN (convolutional neural network) model for identification and, as a result, confirmed that the model using inertial data significantly lowered the recognition performance from 99% to 81% when the shoes worn by the individual changed. As such, if the performance of identification systems is affected by environmental changes, these systems have limitations in being used as a tool to identify sensitive personal information.
The ultimate purpose of this study is to overcome these limitations and to confirm whether it is possible to maintain the constant performance of the identification system based on data collected in various environments. For this purpose, we discuss how to extract HS and TO, which are key information for gait cycle identification, using an long short-term memory (LSTM) model and apply it to the system. To do this, the first step is to accurately determine when HS and TO occurred. In this study, the LSTM model is trained with the subject's gait data to identify the gait cycle of a person by finding the HS and TO points of both feet using only the inertial sensor, and the model produced HS and TO with 95% precision and 93% recall. Then, the gait cycle obtained in this way is applied to the individual identification CNN model, and the model identified individuals who changed shoes with over 90% accuracy.
In this study, we changed the type of shoes to reflect the real environment, which is a new method that has not been done before, but the type of shoes is not the only variable that can reflect reality. For example, factors such as the shape or slope of the ground, gait imbalance due to disease, etc., can change the walking pattern and affect the identification. Therefore, as a future research issue, it is necessary to find variables that can reduce the difference between the real environment and the experimental environment.
Our contributions are summarized as follows: (i) This study identified individuals with over 90% accuracy using gait data collected while changing shoe type, a factor that has not been utilized in existing gait studies to apply to the real world. (ii) This study mitigated the decrease in identification accuracy with changing shoe types, which can be varied in the real world, by using gait cycle data from an LSTM model.
The structure of the study is as follows: Section 2 examines research trends on individual gait identification using gait analysis, and the limitations of existing gait identification studies are identified as a literature review. Section 3 describes in detail how gait data is collected from the experiment participants and the sensors used for data collection. Section 4 introduces the preprocessing method of the collected gait data. The linear interpolation method and low pass filtering method are applied to the raw data. Section 5 explains the process of finding HS and TO using LSTM. In Section 6, we present in detail the personal identification method using CNN proposed in this paper. We introduce how to train the CNN model by using both the HS and TO found by the LSTM mentioned in Section 5 and the gait data of the subject as a training data set. It explains in detail how personal identification is smoothly performed through the data acquired while the subject changes the type of shoe. Finally, in Section 7, the findings of this study are discussed, and future research tasks are presented.

Related Work
Numerous studies have been conducted to collect and analyze gait data for personal identification purposes. Gait data refers to inertial data obtained through accelerometers, gyroscopes, and other sensors, encompassing various temporal and spatial data related to walking. Many studies have shown that each gait data has unique characteristics [3][4][5][6][7][8][9][10][11][12][13]. Consequently, gait data has been proposed as an authentication method in the security and authentication fields, similar to fingerprint, iris, and face recognition [11][12][13][14][15][16][17][18][19][20][21][22][23]. Gait research can be broadly categorized into studies that utilize machine learning techniques and those that do not. In this section, we review the trends in each type of research and provide a brief overview of the data collection environment for identifying gait data owners.
Most studies that do not employ machine learning techniques primarily focus on the medical field. These studies extract gait features such as balance, speed, posture, and stride for patients with specific diseases [24][25][26][27][28][29] and develop algorithms to assess the severity of the disease based on these features [24,28]. In these studies, the extraction of features such as balance, speed, posture, and stride length from continuous gait often revolves around identifying important events in the gait cycle, such as HS and TO. The algorithm's performance is then evaluated using metrics provided by medical professionals, such as the UPDRS (Parkinson's Disease Evaluation Index). While many medical studies aim to identify diseases or analyze walking patterns, relatively few studies specifically related to user authentication exist in the security field.
In contrast, gait analysis studies using machine learning techniques have been conducted for various purposes, ranging from identifying specific diseases to user authentication. Machine learning-based research on gait pattern identification and recognition has gained attention as a means of user authentication using biometric data, diverging from traditional gait analysis methods. However, many studies focusing on gait identification or authentication heavily rely on 2 Journal of Sensors datasets with limited collection environments [4, 7-9, 14, 15].
These datasets include open data sources such as RecodGait [3], OU-ISIR [2,4,17,21], ZJU-GaitAcc [13], WISDM (wireless sensor data mining) [3,21,22], HMOG [20,22], and Motion Sensor [22], which have the following limitations: (i) An authentication system trained on data from one position may misclassify data from another position. (ii) An authentication system trained on data from controlled environment may misclassify data from uncontrolled environment. (iii) An authentication system trained on data collected while wearing one type of shoe may misclassify data collected while wearing another.
For instance, the OU-ISIR dataset contains data collected exclusively from treadmills, while the dataset used in the study [8] consists of data collected from thighs. In addition to location and collection position limitations, all of the aforementioned studies do not accurately reflect real-world environments where gait data can be utilized for authentication purposes. These studies often collect data while wearing a specific type of footwear, neglecting the potential variations in shoe types that may affect authentication performance. Thus, the collection environment of gait data needs to be carefully considered. One limitation of previous studies is that they solely trained the authentication system using data collected while wearing a single type of shoe, failing to capture real-world scenarios. In our study, we collected gait data using various shoe types to better reflect the real-world environment.
In gait research, many types of models have been used to learn from gait data to produce specific outputs. In general gait studies, recurrent neural network (RNN)-based models are often chosen to learn the context of inertial sensor values, which are time-series data [30][31][32][33][34][35][36][37][38]. Representative types of models are LSTM and gated recurrent unit (GRU), and both models are used for processing time-series data and solving the long-term dependency problem that occurs in simple RNNs. The above two models, which are improved models of RNN, are suitable for processing inertial sensors because they can memorize or use time-series data using internal memory. In a study on user authentication, the LSTM model outperforms other models [11]. Other classification models have used CNN, support vector machine (SVM), and random forest (RF) [31,32,39]. In studies for authentication and identification, CNN [2, 6, 8-10, 12, 21, 22] and LSTM [10,11,21] have shown particularly good performance. In addition, other models used include SVM [4,6,7,19], K-nearest neighbors [6,16], and RF [7,16]. The model's performance is evaluated according to the evaluation metrics. Evaluation of model performance is typically conducted using widely used metrics such as mean absolute error, mean squared error, root mean squared error, precision, recall, and F1-score. This study analyzes the performance by referring to the above metrics for evaluating the model.
This section provides an examination of trends in gait studies with and without machine learning. Many existing gait studies used datasets for model training limited by factors such as fixed sensor positions, controlled data collection environments, and fixed shoe types. Although some studies have attempted to account for these factors, none have explored the impact of changing shoe types to better reflect real-world environments. Our study aims to address this gap by collecting data using different shoe types, which is a novel approach. We built a model using LSTM and CNN, which demonstrated excellent performance in previous studies, to identify individuals based on the collected data.

Data Collection
Data collection is divided into two main categories. First, it goes through a process of collecting walking data to obtain a gait cycle and obtaining gait data for personal identification. In this section, these two data collection processes are explained in detail.

Data Collection for Gait Cycle.
We obtained an approval for our experiment by the Institutional Review Board (IRB) of Gangneung-Wonju National University (GWNUIRB-2020-33) and collected gait data from 50 subjects. The subjects consist of 25 healthy males and 25 healthy females, whose ages range from 20 to 29 years. All participants agreed to be collected personal information such as height, weight, and name. They also heard a detailed explanation of how the experiment was conducted and were also fully familiar with the precautions for the experiment. All subjects repeatedly walked at their natural walking speed on a 33 m long flat land a total of 10 times.
A Shimmer shown in Figure 1 has a 3-axis accelerometer and a 3-axis gyroscope. It measures acceleration and angular velocity when the subject walks during the experiment. Tables 1 and 2 show the detailed specification of the accelerometer and gyroscope of Shimmer3 IMU sensor. As represented in Figure 2, all subjects attach the Shimmer to the left  wrist of them to collect the 3-axis direction of acceleration and angular velocity. A pressure data collection device shown in Figure 3 collects the pressure applied to the heel and forefoot. All of the subjects wearing the Shimmer and the shoes attached to the pressure data collection device participated in the experiment. What we have learned from numerous previous gait analysis studies is that when people walk, they repeat certain cycles, which is called the gait cycle. The gait cycle is composed of the action of HS and TO. Figure 4 denotes the HS and TO within one gait cycle. Accelerometers and gyroscope sensors were mainly used to collect gait data in most previous gait analysis studies. It is not an easy task to analyze the data collected from these sensors to find HS and TO and to find the gait cycle from these results. In this study, the exact time point at which the HS and TO actions occured was obtained from the pressure data collection device, and the acceleration and angular velocity values at that time were synchronized. The reason for this is that, as described above, it is difficult to accurately determine the time points of HS and TO with only accelerometers and gyroscope sensors.

Data Collection for Personal
Identification. Ten subjects, four men and six women, walked twice back and forth on a 20 m flat surface, changing the type of shoes. The types of shoes are divided into type 1 and type 2. In type 1, the shoes of the participants in the experiment were used as they are, and in type 2, slippers provided by us were used. The external appearance of the shoes used in the experiment is shown in Figure 5(a) shows shoe type 1, and Figure 5(b) represents shoe type 2.
The Shimmer is attached to the left wrist with a strap to collect acceleration and angular velocity in Figure 6. Figure 6 shows that the subject walks on a 20 m flat surface while changing the type of shoes. The ultimate reason for collecting gait data while changing the type of shoe is to show through

Heel strike
Step width Step length Stride length Toe off Gait cycle FIGURE 4: HS and TO within one gait cycle [44].
ðaÞ ðbÞ   Journal of Sensors this study that there is no problem in identifying a specific person with the gait data even if there is an environmental change in which the type of shoe changes.

Data Preprocessing
We preprocess the collected gait data as follows: first, linear interpolation is applied to the walking data to compensate for missing data from datasets collected at 20 ms intervals; and second, low pass filtering is conducted to the collected raw data to remove noise. This section describes these two preprocessing methods.

Linear Interpolation.
The sampling rate of the pressure data collection device and the Shimmer sensor is set to 20 ms in data collection. However, examining the result of the data, the data is not sampled exactly every 20 ms. In other words, there exist cases in which data measurement does not occur according to the sampling rate we configure for the experiment. Linear interpolation shown in Equation (1) is used to the collected data to solve this problem.
where x: target time; y: target value; x 0 , y 1 : time before and after x; y 0 , y 1 : value before and after x.

Low Pass
Filtering. It can be hard to capture the repeated ambulation pattern from raw data due to noise generated inside the device and caused by environmental factors during movement. In our work, we remove noise using a low pass filter provided by Butterworth in the Python SciPy library and properly adjust the two values of Order and Cutoff to eliminate noise from the raw data. The same values for Order ( = 2) and Cutoff (0.2) were applied to all subjects' gait data and were modified (Order = 1−2, Cutoff = 0−1) until they represented the optimal performance of the LSTM model, which will be discussed in detail in Section 5. Figure 7 shows the x-axis acceleration with filtering applied. The acceleration applied with filtering is shown in bolder black, and the acceleration without filtering is shown in gray. The reason for doing this is to extract only features related to gait by filtering out noise in the data. The same filtering mechanism is applied to the angular velocity data, and we obtain the same results as shown in Figure 7.

Recognition of a Gait Cycle Using LSTM
We use LSTM, the most suitable recurrent neural network model, for time series because gait data is basically based on time. All deep learning training is conducted using the Keras library, which is easy to construct neural networks. After training the data obtained from the Shimmer with the LSTM model, the process of discriminating HS and TO constituting the gait cycle is described in detail in this section.

Training Dataset.
It is not easy to find out when HS and TO occur only with inertial data obtained from a sensor attached to the subject's wrist. Therefore, after collecting pressure data through the pressure data collection device mentioned in Section 3, the task of finding the time of occurrence of HS and TO is first performed based on this data, and then, we examine the relationship between the wrist trajectory and pressure data as shown in Figures 8 and 9. Figures 8 and 9 divide the acceleration direction of the upper body and wrist according to the foot in contact with the ground. Ultimately, we intend to perform the task of discriminating the timing of HS and TO only with the acceleration and angular velocity collected from the sensor. It is considered that it might not be appropriate to simply analyze the magnitude of the acceleration only in the movement direction of the wrist in analyzing the magnitude of the acceleration of the wrist. This is because not only the movement direction of the wrist but also the movement direction of the upper body affects the acceleration of ambulation. The upper body, which is the center of the pendulum  Journal of Sensors movement, is not stationary. The upper body, which is the center of the pendulum motion of the arms, is not stationary and continues to move in the walking direction (+). When it is TO, it accelerates in the (+) direction, and in HS, it decelerates in the opposite (−) direction. Let's think about the acceleration caused by the movement of the upper body when walking. During HS, the heel touches the ground and decelerates due to the frictional force, and in TO, acceleration occurs due to the force pushing the ground with the forefoot. Let's look at the acceleration direction of the wrist indicated in bolder line in Figures 8 and 9. It can be seen that acceleration and deceleration are applied in the (+) direction when the left foot HS and TO, and in the (−) direction when the right foot HS and TO.
The direction of acceleration of the upper body and wrist, which we looked at previously, affects the acceleration shown in Figures 10 and 11.
Let's look at Figure 10 first. Figure 10 shows the acceleration of the left foot's HS and TO in bolder gray and bolder black, respectively. Looking at the acceleration during HS and TO, it can be seen that the acceleration increases in the (+) direction, and the magnitude of the acceleration applied during TO is larger than that of HS. Conversely, looking at Figure 11, it can be seen that the acceleration in the (−) direction increases during the HS and TO of the right foot, and the magnitude of the acceleration is smaller during the TO compared to the HS. Table 3 summarizes this phenomenon. According to the "Direction of acceleration" in Table 3, it can be seen that the "Magnitude of the acceleration" is neutralized ("(+)+(−)" or "(−)+(+)") or amplified ("(+)+(+)" or "(−)+(−)"). It can be seen that the direction of acceleration of the wrist and upper body is the criterion for distinguishing the HS and TO of both feet through this. In conclusion, we analyze the acceleration direction of the upper body and the wrist and the pressure data obtained from the pressure data collection device to find out the relationship between the acceleration of the wrist and HS and TO. Finally, it can be said that the points of the HS and TO of the two feet can be recognized only by the acceleration of the wrist.  Unlike TO of the left foot and HS of the right foot, which are well characterized by the large magnitude of the acceleration, the HS of the left foot and the TO of the right foot are smaller in comparison, making it difficult for the LSTM to capture the characteristics well. Therefore, in this study, by taking the inverse number indicated in Equation (2) for the inertial data at that time, the characteristics were made more prominent. By doing this, we construct the training data of the LSTM by extracting features (M acc , M gyro ) from Equation (2) that can well capture the gait cycle from the inertial data as follows: where M acc : momentum of acceleration, M gyro : momentum of angular velocity, X acc , Y acc , Z acc : acceleration of each axis, X gyro , Y gyro , Z gyro : angular velocity of each axis. Two features, M acc and M gyro and class information, as described in Table 4, are used as feature for the LSTM model. The class is divided into four positive classes of HS and TO, which is determined from the pressure data collection device, and one NONE negative class that is not both HS and TO.

Window Construction.
When you look closely at the gait, it can be seen that the HS and TO of both feet appear repeatedly and continuously, and this is called the gait cycle mentioned above. Due to the characteristics of the gait cycle, in discriminating HS and TO, it is better to deal with a data set that occurred continuously for a specific time rather than individually analyzing inertial data related to walking. Therefore, we define this data set as a window and calculate the average gait cycle time from the pressure data of 50 subjects. The average gait cycle calculated this way is about 1.2 s, and the data generated during this time is managed as one set. A window is defined as a set of inertial data generated for about 1.2 s and consists of 60 data samples (sampling rate: 20 ms). Let us look at Equation (3), showing the window's composition.
where i: sampling time, w: size of window (1.2 s), Window i : constructed window at i. Among the inertial data collected at a sampling period of 20 ms, 60 M acc and M gyro extracted based on 1.2 s are composed of one window.

LSTM Model Configuration and Validation.
In this study, training is conducted using LSTM, a recurrent neural network model. We set the test_size ( = 0.4) and shuffle ( = true) parameters in Scikit-learn's "train_test_split()" function to randomly split the dataset 6 : 4 for training and testing, respectively. The configuration of the LSTM and description of designing each neural network layer are presented in Table 5. When training the model, we set the save_best_only ( = true) parameter in TensorFlow's "ModelCheckPoint()" function to save the model only when the model performance improves and used 10% of the test dataset for validation. The loss and accuracy on the training epoch of training and validation data are shown in Figure 12. Looking at all the values in Figure 12, we can see that overfitting occurs from about the 12th epoch, and checking the accuracy calculated by the model, we can see that the training set identifies the gait cycle with an accuracy of about 90% and the validation set with an accuracy of about 80%. Through Figure 12, it is expected that the accuracy of the test set could be about 80%. However, this study uses our evaluation criteria, which will be described in detail in Section 5.4, rather than evaluating the LSTM itself.    Table 4. Therefore, the Softmax activation function used for multiclass classification is applied to the output layer to produce a value between 0 and 1. Figure 13 shows the class categorized with the highest probability through the output of the model. In Figure 13, "predict" marked with X is the class classified by the model, and "r_value" marked with a solid line is the class measured by the pressure data collection device. Looking at the number of "predict" marked with X in Figure 13, only 24 out of a total of 300 data samples were classified as HS or TO, and all the rest were categorized as NONE. There are much more NONE classes than HS and TO. "r_value" also has the same aspect as "predict." For example, consider the case of classifying a total of 300 data samples into five classes. If each class has 60 data samples, it is said to be evenly distributed. However, as shown in Figure 13, most data samples are concentrated in the NONE class. This phenomenon is called a class imbalance problem. The learning result of the model constructed in Section 5.3 has a class imbalance problem. It is known that it is generally appropriate to calculate precision and recall rather than accuracy when evaluating models with class imbalance problems. Therefore, we evaluate the model by calculating precision and recall through the evaluation criteria of Equation (4) and Figure 14.

Evaluation and Analysis.
Precision ¼ num of cases in which r value in tolerance num of predict Recall ¼ num of cases in which predict in tolerance num of r value : Prior to calculating precision and recall, "predict" shown in Figure 13 is compared with "r_value" to find out which of the five classes belongs. First, looking at the six results identified as "TO R " among "predict" in Figure 13, all four except for the second and fifth "predict" exactly match "r_value." In particular, the second and fifth "predict" exist before or after 20 or 40 ms based on "r_value." In this way, if "predict" exists within a certain time based on "r_value," it is determined that "r_value" and "predict" are identical. In order to establish a standard for a certain time, we check how the accuracy of the model results changes as time is changed, and the allowable range for time is defined as "Tolerance." When calculating precision, as shown in Figure 14, "Tolerance" is applied based on "predict" to calculate how many "r_value" exist within the allowable range. In addition, when calculating recall, "Tolerance" is applied based on "r_value" to determine how many "predict" are within the allowable range. "Tolerance" is set at a maximum interval of 1 s in consideration of the average gait cycle time of the research participants, and the results of calculating precision and recall are shown in Table 6. When "Tolerance" is set to AE0.5, the average precision is 95.98%, and the average recall is 93.18%, indicating that HS and TO are well classified within one gait cycle.   Journal of Sensors

Personal Gait Identification Using CNN
In this study, we construct CNN deep learning models A and B to confirm the effect of the gait cycle on identifying individuals with gait data. The experiment is conducted twice according to the input data of the deep learning models, as shown in Figure 15, and inertial data and gait cycle data are used as input data. Inertial data that includes acceleration and angular velocity values are used for training models A and B. Gait cycle data, that is, the HS and TO of both feet obtained from the LSTM configured in Section 5, is used for the training of model B only to confirm how the gait cycle affects the identification by comparing the results of identification with only inertial data and with gait cycle data together. Training and testing of the models are conducted by changing the types of shoes, divided into sneakers (shoe type 1) and slippers (shoe type 2). When training the models, the inertial data obtained from shoe type 1 is provided to the deep learning model A, and the inertial data and gait cycle data obtained from shoe type 1 are entered into the deep learning model B. After model training, the data for test is divided into four types, as shown in Figure 16, according to the training data and the types of shoes. The first test data (Test A-1 in Figure 16) is the inertial data from shoe type 1, and the second (Test A-2 in Figure 16) is the inertial data from shoe type 2. The third test data (Test B-1 in Figure 16) is the inertial data and gait cycle data obtained through shoe type 1, and the fourth (Test B-2 in Figure 16) is the inertial data and gait cycle data obtained from shoe type 2. This section describes the process of entering these four test data into deep learning models A and B and evaluating the performance of personal identification.

CNN Model Configuration and Validation.
In this study, we implement a CNN model for gait pattern identification.
As with the LSTM model, we randomly split the dataset 6 : 4 for training and testing. Since the inertial data to be input is time series numerical data, we use a 1D nested layer, i.e., conv1d layer, for 1D vector processing. The output of the model is the probability per test subject's ID. When training the model, as with the LSTM model, we saved the model only when the model performance improved and used 10% of the test dataset for validation. The model configuration is shown in Table 7, and the loss and accuracy on the training epoch of model B are shown in Figure 17. Looking at all the values in Figure 17, we can see that overfitting occurs from about the 18th epoch, and checking the accuracy calculated by the model, we can see that the training set identifies the individuals with an accuracy of about 95% and the validation set with an accuracy of about 90%. Through Figure 17, it is expected that the accuracy of the test set can be about 90%.
Layer 1, 4, 5, 9 (conv1d): Perform a 1D convolution, which is the process of extracting relevant features by sliding a filter based on the input data from models A and B. Layer 2, 5, 7 (max_pooling1d): Reduce the size of the feature map output from the conv1d layer, and use the maximum value of that feature map (maxPool) to highlight and extract the most important features and pass them to the next layer.  Table 8, and a confusion matrix is obtained.

Journal of Sensors
Performance is then evaluated using four metrics: accuracy, precision, recall, and F1-score. The four test data are entered into the deep learning models and classified using the standards presented in Table 8 to obtain a confusion matrix, as shown in Figure 18, which visually represents the classification result for each model. In Figure 18(a), the X-axis represents the actual class, and the Y-axis represents the class classified by the model. For example, the rectangle marked 6,619 in the upper left corner of Figure 18(a) means that 6,619 of the 6,630 data of P1 are correctly classified as TP, and the rest of the X-axis (0, 5, …, 0) are misclassified as FN. And the rest of the Y-axis (8, 4, …, 2) means it is misclassified as FP, and any square other than TP, FN, or FP is correctly classified as TN. Figure 18(b)-18(d) can be interpreted in the same way, and the larger the number of squares placed on the diagonal (the darker color), the better the model performs. Through this, it can be confirmed that the classification performance of the model is high for the result of the first test data (Figure 18(a)). However, looking at the numbers on the diagonal in the result of the second test data (Figure 18(b)), it can be seen that it has lower classification performance compared to the first test data (Figure 18(a)). The cause of the low identification performance is due to the change of the experimental environment as the shoe type changed. Comparing the results of the third test data (Figure 18(c)), Conv1d_3 (Conv1D) 10 Max_pooling1d_3 (MaxPooling1D)  11 Flatten (Flatten) 12 Dense (Dense) and the fourth test data (Figure 18(d)), it can be seen that the same performance degradation occurs. However, looking at the misclassified values in the second test data (≥100 : 61, ≥200 : 22, ≥300 : 1) and in the fourth test data (≥100 : 27, ≥200 : 1, ≥300 : 0), it can be seen that the number of misclassified has decreased. A total of 10 experiments were conducted to obtain the identification results of the four test data, as shown in Figure 18, and all the results obtained by the experiments showed the above characteristics. This means that the degree of performance degradation due to changes in the experimental environment can be reduced by using the gait cycle data. In order to more accurately evaluate these visually processed classification results, accuracy, average precision, average recall, and average F1-score are calculated and shown in Table 9. The closer the number is to 1, the higher the performance.
Accuracy: The percentage of all classification cases in which a specific person and another person are correctly classified.
Precision: The percentage of cases classified as specific persons that are actually specific. It means how accurate the classification is.
Recall: The cases of correctly classified in which a specific person is correctly classified as a specific person. It means how much the actual value has been matched.
F1-score: It refers to the harmonic mean of precision and recall and is mainly used when the imbalance between classification classes is severe. The LSTM model is constructed and used in consideration of the recurrence of the class imbalance problem of HS and TO.
When evaluating the results based on accuracy for the second test data, we can see a drop of about 0.18 from the result of the first test data (0.99). However, for the fourth test data, which contains both inertial data and gait cycle data, only about 0.09 falls from the results of the third test data (0.81). It is confirmed that identification performance deteriorates when data of different shoe types are entered into the deep learning model in which trained data of one shoe type only; that is when changed the experimental environment. However, the performance degradation that occurred when using inertial data only (deep learning A test result: 0.99 > 0.81) could be improved by entering gait cycle data together (deep learning B test result: 0.99 > 0.9). As a result, it shows that using gait cycle data can mitigate performance deterioration caused by changes in the experimental environment.

Conclusion
This study, which uses a deep learning model to identify the owner of gait data, confirms the performance degradation of the identification model when the type of shoe is changed. In addition, we suggest a method to minimize performance degradation using gait cycle data and evaluate the results obtained.
Previous studies on walking have shown that each person's gait data has unique characteristics, which have also identified the owner of the gait data or proposed an authentication system. The results were positive, but most of those studies have extremely restricted gait data collection environments, making it difficult to apply them to real-world environments where numerous variables exist. In this study, to reflect the natural environment as much as possible, walking data collected by changing the type of shoes (sneakers, slippers) is used to enter and train a CNN identification model, and the results are evaluated and analyzed.
Inertial data for gait cycle classification collected for 50 participants, 25 males and 25 females in their 20s, are preprocessed through linear interpolation and filtering and entered into LSTM. As a result, HS and TO of both feet are classified with an average precision of 95.95% and recall of 93.18%. The gait cycle acquired for personal identification through the model configured in this way is provided to the identification model.  Inertial data for personal identification is collected for 10 participants, five males and five females in their 20s, for each type of shoe, and the gait cycle data is obtained through the LSTM model. The inertial data and the gait cycle data are composed of four test data and given to the CNN model. As a result, it is confirmed that the identification accuracy decreased from 99% to 81% when data from different shoe types are entered into the deep learning model trained with only one type of shoe. And the performance degradation is improved to about 90% through gait cycle data. As a result, it showed that using gait cycle data can mitigate performance degradation due to changes in the experimental environment. In this study, we reflected the real-world environment by changing the type of shoes, but the type of shoes is not the only variable that can reflect the reality. For example, factors such as the shape or slope of the ground, gait imbalance due to age, fatigue, disease, etc., can change the walking pattern and affect the identification. Therefore, as a future research topic, we propose to find variables that can reduce the difference between the real-world environment and the experimental environment and apply personal identification as a service.

Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
The authors declare that they have no conflicts of interest.