HIT HAR: Human Image Threshing Machine for Human Activity Recognition Using Deep Learning Models

In recent days, research in human activity recognition (HAR) has played a significant role in healthcare systems. The accurate activity classification results from the HAR enhance the performance of the healthcare system with broad applications. HAR results are useful in monitoring a person's health, and the system predicts abnormal activities based on user movements. The HAR system's abnormal activity predictions provide better healthcare monitoring and reduce users' health issues. The conventional HAR systems use wearable sensors, such as inertial measurement unit (IMU) and stretch sensors for activity recognition. These approaches show remarkable performances to the user's basic activities such as sitting, standing, and walking. However, when the user performs complex activities, such as running, jumping, and lying, the sensor-based HAR systems have a higher degree of misclassification results due to the reading errors from sensors. These sensor errors reduce the overall performance of the HAR system with the worst classification results. Similarly, radiofrequency or vision-based HAR systems are not free from classification errors when used in real time. In this paper, we address some of the existing challenges of HAR systems by proposing a human image threshing (HIT) machine-based HAR system that uses an image dataset from a smartphone camera for activity recognition. The HIT machine effectively uses a mask region-based convolutional neural network (R-CNN) for human body detection, a facial image threshing machine (FIT) for image cropping and resizing, and a deep learning model for activity classification. We demonstrated the effectiveness of our proposed HIT machine-based HAR system through extensive experiments and results. The proposed HIT machine achieved 98.53% accuracy when the ResNet architecture was used as its deep learning model.


Introduction
e human healthcare systems have a vital role in our daily life. Due to the busy lifestyle, these days, the lack of exercise causes serious health issues. Emerging technologies such as human activity recognition (HAR) systems [1] can monitor the users' activities in the healthcare system. Recent research trends in HAR show its wide variety of applications that include health and fitness monitoring [2], assisted living [3], context-enabled games and entertainment [4], social networking [5], and sports tracking [6]. In HAR, the system tracks the user's movements and classifies the user's activities based on the sensor reading. e existing HAR system includes vision-based [7], radiofrequency-based [8], or wearable sensor-based approaches [9]. e most common and low installation cost-based HAR technique is the wearable sensor-based approach. e sensor-based technique is location independent, and the user can easily hold the sensor during their activities. e sensor-based HAR approaches achieved a remarkable classification accuracy, and smartphone or smartwatch-based HAR is the most common system used for activity recognition. However, the sensor errors, sensor type, sensor position in the human body, and user's complex activities make the system more challenging for activity recognition. e HAR system has worst classification results when the user is in complex activity motion. On the other side, when the HAR system uses radio frequency (RF) signals for activity recognition, the system takes advantage of the wireless communication features to classify the user's activities. Compared with the sensor-based HAR approach, RF-based HAR is device-free, and the system does not need any physical sensing module. e device-free characteristics of radio frequency-based HAR provide reduction in energy consumption and privacy protection compared with the sensor or vision-based HAR systems. However, indoor channel conditions, non-line of sight conditions, and signal interference affect the performance of HAR, and the system faces difficulties in maintaining high accuracy levels. Besides these HAR approaches, the vision-based HAR system uses a camera that records the user's activities in a video sequence. e vision-based approach uses computer vision algorithms for activity recognition. Based on the camera type used in the HAR system, the video sequence from the vision approach is in the form of RGB videos [10], depth videos [11], or RGB-D videos [12]. Compared with sensor-based or radio frequency-based HAR approaches, the vision-based approach shows higher classification results for users' complex activities. However, user privacy, energy consumption, and deployment cost are the main challenges for the vision-based HAR approaches. In this paper, our research focuses on the vision-based HAR approach, and we propose a human image threshing (HIT) machine-based HAR system that addresses some of the existing vision-based HAR challenges. Our HIT machinebased HAR system uses a smartphone camera as an input device to record the users' activities. A mask region-based convolutional neural network (R-CNN) further processes the recorded activity videos for human body detection, a facial image threshing machine (FIT) for image cropping and resizing [13], and a deep learning model for activity recognition. Our HIT machine can generate HAR images from activity videos, human body detection from images, data cleaning and removal of irrelevant data, and activity classification using a deep learning model. We tested our HITmachine with different HAR experiments based on deep learning models, including visual geometry group (VGG) [14], Inception [15], ResNet [16], and EfficientNet [17] models. e results from the HIT machine show that the system always maintains the classification accuracy for activity recognition. We analyzed our HIT machine results with conventional HAR approaches that include inertial measurement unit (IMU) and stretch sensor-based approaches. e results show that the HIT machine outperforms the traditional sensor-based approaches with a higher level of accuracy for activity recognition. We also tested our pre-trained deep learning models with unseen HAR datasets and analyzed the classification performance. e key contributions from our HIT machine are stated as follows: (i) We created a HAR dataset using a smartphone camera, IMU sensor, and stretch sensor. Our dataset consists of nine activities: sitting, standing, lying, walking, push up, dancing, sit-up, running, and jumping. It has 36, 558 image samples from smartphone cameras, 97,454 data samples from IMU sensors, and 7,850 data samples from stretch sensors. We used these datasets to validate our HIT machine, and the deep learning models can use our HAR datasets for training and testing without any computational complexity. We also collected HAR datasets for unseen datasets and tested them with pre-trained deep learning models. (ii) We proposed a HIT machine for activity recognition, and our HIT machine shows accurate classification results for basic (sitting, standing, and walking) and complex (running, jumping, and lying) activities. We tested our HIT machine with different deep learning models and analyzed the classification performance in terms of a confusion matrix, accuracy, loss, precision, recall, and F1 score. We also tested the pre-trained models with unseen HAR datasets and compared the performance of each model. We validated our HIT machine results with sensor-based HAR results and proved the impact of the HIT machine for activity recognition. e rest of the paper is organized as follows: Section 2 discusses the existing HAR systems, recently proposed HAR systems with their advantages, and current HAR challenges for practical implementation. Section 3 presents our proposed HIT machine-based HAR system, including mask R-CNN, FIT machine, and deep learning models. Section 4 discusses our HAR experiments with the validation of our HIT machine in terms of the impact of various deep learning models, analysis of unseen datasets for pre-trained models, and the result comparison with conventional HAR approaches. Finally, Section 5 concludes our HIT machinebased HAR approach with future research directions.

Related Work
HAR has been studied for applications in healthcare monitoring, smart homes, security, medical imaging, robot/ human interaction, personal assistants, and surveillance [18][19][20]. Many researchers have discussed various HAR approaches based on the technologies or algorithms used for activity recognition [21][22][23][24][25]. In this paper, our literature focuses on related work for HAR approaches that include sensors [26,27], Wi-Fi [28], Wi-Fi, and sensors [29], vision [30,31], and RFID [32]-based activity recognition. e HAR approaches from [26][27][28][29][30][31][32] provide significant performance improvements for HAR applications. However, the diversity of age, gender, and number of subjects, postural transitions, number of sensors and type of sensors, different body locations of wearable sensors or smartphones, missing values or labeling error, similar postures and datasets having complex activities, lack of ground truths, selection of appropriate datasets, and selection of sensors [33,34] create challenges to the HAR implementation. is paper proposes a HIT machine-based HAR system to address some of these challenges with higher classification results. e sensors-based HAR approaches are the most common and popular HAR systems. In sensor-based HAR, the system uses wearable sensors, smartphones, or smartwatches to collect data and identify the users' activity based on the sensor readings. Some of the recent HAR systems 2 Computational Intelligence and Neuroscience which take advantage of wearable sensors are discussed in [35][36][37][38][39]. ese systems achieved a remarkable recognition accuracy in real time. However, mounting a wearable sensor in the human body is challenging, and the wearable sensor's position determines the system's performance. e wearable sensor-based HAR systems still need to optimize the location of sensors in the human body for complex activity. An alternative method for activity recognition is the smartphone-based HAR systems [40][41][42][43]. In smartphone-based HAR, the user holds the smartphone and performs the activities. Compared with wearable sensor-based approaches, the smartphone-based method is simple and easy to implement in any place without any external sensors. However, the position in which the smartphone is held and the modes such as texting and calling affect the system's performance. e smartphone or wearable sensors-based HAR approach still needs to improve the classification performance at a certain level, and current systems use deep learning models for activity recognition [44][45][46][47]. e deep learning HAR-based systems include convolutional neural network (CNN) [48], long short-term memory (LSTM) [49], LSTM-CNN [50], deep recurrent neural networks (DRNN) [51], generative adversarial networks (GAN) [52], extreme learning machine (ELM) [53], graph neural network (GNN) [54], and semi-supervised deep learning models [55,56]. ese systems use raw sensor reading or extract the signal features in the time/frequency domain for activity recognition. When the system uses the signal in the time domain, it extracts the variance, mean, maximum, minimum, and range values and uses these features as model inputs. On the other hand, If the signal is in the frequency domain, the system extracts the amplitude, skewness, kurtosis, and energy information as to its features and uses this input to the model. Compared with the raw input signal-based deep learning HAR approach, the feature-based approaches show better classification results [2]. However, the deep learningbased HAR approaches are not free from challenges. A large number of data samples for training, training time, the complexity of feature extraction, and human resources required for data collection are some of the main challenges of deep learning-based HAR approaches. ese challenges reduce systems performance and require further classification improvements. e RF-based HAR approaches use physical sensors, such as pressure, proximity, FM radio, microwave, or RFID for activity recognition [57][58][59][60][61]. In a radio frequency-based approach, the system takes advantage of the body attenuation and the channel fading characteristics for activity recognition. e basic principle of RF-based HAR systems is that the propagation of RF signals is affected by the human body movement, resulting in attenuation, refraction, diffraction, reflection, and multipath effects. ese pattern differences in the received RF signals are the key ideas for activity recognition. Different activities lead to various patterns inside RF signals, and the system can use these features for classification. e RF-based systems consist of signal selection, model, signal processing, segmentation, feature extraction, and activity classification. In signal selection, the system uses Wi-Fi, ZigBee [63], RFID [64], frequency-modulated continuous-wave radar (FMCW) or acoustic devices. e system uses phase, frequency, amplitude, or raw signal for activity recognition depending on the signal selection. ese factors determine the model of the HAR system. When the model is defined, the system uses signal processing techniques, including noise reduction, calibration, and redundant removal. After this, the system uses signal segmentation in the time or frequency domain. When segmentation performed, the time domain, frequency domain, time-frequency domain, or spatial domain features are extracted for classification. e deep learning models use extracted features for activity recognition. Compared with the wearable sensor-based HAR approach, the RF-based approach exploits the wireless communication features for activity recognition. ese systems do not use any physical sensing module, thus reducing energy consumption and user privacy concern. Some of the RF-based HAR approaches are discussed in [65][66][67][68]. e RF-based systems discussed here have enhanced the HAR classification performance and opened many applications for detection, recognition, estimation, and tracking. However, the wireless channel conditions, signal interference, non-line-of-sight (NLOS) conditions, multi-user activity sensing, and limited sensing range make the systems more challenging. ey require new theoretical models and open datasets for accurate classification.
e system uses a video sequence for activity monitoring when considering a vision-based HAR approach for activity recognition [69,70]. e vision-based approach is best for multi-user activity recognition when privacy is not a significant concern. ese systems use different computer vision algorithms on activity videos to predict the user's activities from videos or images. Some of the vision-based HAR approaches are proposed in [71][72][73][74][75][76][77]. ese visionbased systems effectively use the video or image sequences and classify the users' activity by taking advantage of the recent deep learning models. Several review papers on the vision-based HAR systems are discussed in [78][79][80]. From vision-based HAR review discussions, the authors from [81] focus on the high level of visual processing, including human body modeling, understanding of human actions, and approaches to human action recognition. In [82], the authors presented the current state-of-the-art development of automated visual surveillance systems. ey discussed the necessity of intelligent visual surveillance in commercial, law enforcement, and military applications. In [83], the paper reviews the advances in human motion capture and analysis from 2000 to 2006 and discusses the problems for future research to achieve automatic visual analysis of human movement. e review paper [84] analyzes the approaches taken to date within the computer vision, robotics, and artificial intelligence communities to represent, recognize, synthesize, and understand action. In [84], the authors pay more attention to identifying actions at different levels of complexity. Machine recognition of human activities is reviewed in [85], and the authors present a comprehensive survey of efforts to address the vision-based HAR systems. e paper [80] focuses on pedestrian detection, and [86] introduces a HAR system that recognizes the human Computational Intelligence and Neuroscience 3 behaviors from transit scenes. e most recent HAR systems are presented in [87][88][89]. ese systems tried to improve the feature extraction techniques by introducing object detection, skeleton tracking, and human body poses. e visionbased HAR systems discussed here still have some challenges, such as processing high-quality videos or images, the complexity of the vision algorithms, the requirement for a higher graphics processing unit (GPU) processing power, the installation cost of the camera, and challenges from vision systems such as camera viewpoint, lighting, human body appearance, occlusion, and background clutter. ese challenges make it more difficult for the vision-based approaches for real-time health monitoring. So far, we have discussed different types of HAR approaches based on their technologies and algorithms used for activity recognition. In this paper, our research mainly focuses on the vision-based HAR approach, and we used our smartphones for data collection. We also collected data using IMU and stretch sensors, and the results from these sensors are compared with our proposed HIT machine. e experiment results show that the HIT machine is a practical HAR approach for healthcare applications and needs only a basic smartphone model for activity recognition.

Proposed HIT Machine-Based HAR System
e HIT machine consists of HAR dataset creation, data preprocessing, human body detection using mask R-CNN, image cropping and resizing, data cleaning and removal of irrelevant data, deep feature extraction, model building, and activity classification. Figure 1 shows the framework of our proposed HIT machine-based HAR system.
We first started our data collection in the HIT machine by using android and iOS smartphones that record activity videos. Next, the HIT machine performs the data aggregation on the activity video sequences. e data aggregation gathers all activity data and presents it in a summarized format. Followed by the data aggregation process, our system uses a mask R-CNN algorithm for human body detection. After this, the HIT machine operates the FIT machine for image cropping and resizing when the human body is identified from images. e cropped and resized activity images are ready for the model to use for training and testing. Our HIT machine also used a data cleaning process that removes the unnecessary images from the HAR dataset. After the data cleaning process, the images are ready to be used for model training and testing. We extracted the features from the activity images and created a deep learning model that classifies user activities into nine groups. e output of the HIT machine is the classification results of user activities which include sitting, standing, lying, walking, push up, dancing, sit-up, running, and jumping. e flowchart of the proposed HIT machine is presented in Figure 2.
In the flowchart, the system starts with HAR datasets. e datasets include HAR images from smartphones, accelerometer and gyroscope readings from IMU sensors, and stretch sensor readings. e HAR image dataset is then divided into training, testing, and unseen datasets. We used our HIT machine in the image HAR dataset for human body detection and activity recognition. e HIT machine includes human body detection, data preprocessing using a FIT machine, and deep learning models for classification. A mask R-CNN-based object detection algorithm is used for human body detection. A FIT machine is used for data preprocessing, including image cropping, resizing, data cleaning, and data segregation. A deep learning model is used for the training, and the model classifies the user activities into different categories. e system uses deep learning models of VGG, Inception, ResNet, and Effi-cientNet. On the other hand, conventional HAR approaches use IMU and stretch sensor data for activity recognition with a CNN model. e CNN model also uses the HAR image dataset for activity recognition, and we compared the effect of our HIT machine (with and without HIT machine) for activity recognition. Further discussions of mask R-CNN, FIT machine operation, and the deep learning models are added in the following subsections.

Mask R-CNN.
In computer vision, mask R-CNN is widely used for object detection tasks [90]. e mask R-CNN separates different objects from a video or an image. e algorithm provides the object bounding boxes, classes, and mask information, and our HIT machine can effectively utilize this information for human body detection. e mask R-CNN from our HIT machine operates in two stages. First, the algorithm generates proposals about the regions where an object is located in the input image. Second, the algorithm predicts the object class and refines the bounding box. e algorithm also adds a mask in the pixel level of the object based on the first stage proposal. Compared with Fast/Faster R-CNN-based object detection approaches, the mask R-CNN-based approach has additional features such as a binary mask for each region of interest (RoI). Our system utilizes this binary mask feature for human body detection. Figure 3 shows the structure of mask R-CNN. e mask R-CNN consists of a backbone, a region proposal network (RPN), a region of interest alignment layer (RoTAlign), an object detection head, and a mask generation head. e backbone of mask R-CNN is the primary feature extractor which uses residual networks (ResNets) with or without feature pyramid networks [91]. When our HAR images are fed into a ResNet backbone, the images go through multiple residual bottleneck blocks and turn into a feature map. e feature map contains the abstract information of input images, including different object instances, classes, and spatial properties. e feature map data are then fed into the RPN layer. In this layer, the network scans the feature map and RoI where the human body is located. e next step is to find each RoI from the feature map. is process is referred to as RoIAlign in Figure 3. e RoIAlign extracts the feature vectors from the feature map based on the RoI suggested by the RPN layer. e feature vectors are then converted into a fix-sized tensor for further processes. e outputs from RoIAlign are then processed by two parallel branches: object detection branch and mask generation branch. e object detection branch is a fullyconnected layer that maps the feature vectors to the final classes and bounding box coordinates. e mask generation branch feeds the feature map into a transposed convolutional layer and convolutional layer. e output of mask generation branch is one binary segmentation mask that is generated for one class. en the system picks the output mask based on the class prediction from the object detection branch. Figure 4 shows the human body detection using our HIT machine for nine activities.
As shown in Figure 4, the mask R-CNN accurately detects the human body for nine activities without any detection error. e mask R-CNN used here is straightforward and has a small computational overhead that enables a fast system and rapid experimentation. For more details on mask R-CNN and its implementation, refer to [92][93][94].

FIT Machine.
e HIT machine effectively uses our previously proposed FIT machine for image cropping and resizing [13]. e FIT machine is used to correct missing HAR datasets, remove irrelevant data, merge datasets on a massive scale, and crop and resize images. Our FIT machine     Computational Intelligence and Neuroscience converts input activity video sequences into the image output samples that consist of cropped, resized, and categorized activity images. e FIT machine contains a data receiver, a multi-task cascaded convolutional network (MTCNN), an image resizer [95], and a data segregator as the pre-trained Xception algorithm model [96]. e data receiver converts activity video sequences into images, and the MTCNN identifies the human faces from the activity images. e MTCNN used here consists of P-Net, R-Net, and O-Net layers. When the architecture detects the human faces, the input images enter the P-Net layer, which chooses the possible face frames from the input images. e R-Net layer in the MTCNN uses P-Net outputs as its inputs. e R-Net layer inspects the given initial frames from P-Net, then removes the face frames that do not reach a threshold score. Followed by the R-Net, the O-Net uses the output from the R-Net at the end. In the O-Net layer, it selects the best face frames from the given output from R-Net. Next, the images are passed through an image resizer, reducing the image size to 224×224 pixels. e last part of the FIT machine is a data segregator, which segregates the activity images into adequately labeled directories. e data segregator contains a pre-trained Xception model made by a depth-wise separable convolution layer. e depth-wise separable convolution layer used in the model splits each channel of the input and filter separately. e layer convolves them by each channel and later separates one element of 3 channels to be convoluted until all aspects have been convoluted.
e architecture also has some shortcut structure that skips over the block of the depth-wise separable convolution layers. e model uses a categorical crossentropy loss function as the metric loss measurement. For more details on the FIT machine, refer to [13].

Deep Learning Models.
e last stage of the HITmachine is the deep learning models. Our HAR dataset is trained with deep learning models and classifies user activities into sitting, standing, lying, walking, push up, dancing, sitting, running, and jumping. e HAR dataset consists of image samples, and our system considers four image classification models VGG, ResNet, Inception, and EfficientNet, as the deep learning models. Figure 5 shows the deep learning models used by our HIT machine.    Computational Intelligence and Neuroscience e most common image classification model is the VGG model introduced by the visual graphics at University of Oxford [14]. e VGG model consists of 13 convolution layers, five pooling layers, and three dense layers. e VGG model is sequential in nature and uses many filters one after another. e architecture uses a stack of convolutional layers with different depths in different architectures followed by three fully-connected (FC) layers. e first two FC layers have 4,096 channels each, and the third FC performs the 1,000-way classification. e last layer is the soft-max layer that is used to normalize the classification vector. All the hidden layers in the VGG architecture use rectified linear unit (ReLU) as the activation function. e ReLU activation function is computationally efficient, and its results are in faster learning. e ReLU function also reduces the likelihood of vanishing gradient problems and improves the classification performance. Figure 5(a) shows the architecture of the VGG network.
Next, our HIT machine used a deep learning model, which was developed by Google [16]. e GoogLeNet or Inception is a smaller network than the VGG model and uses an Inception module.
e Inception module performs convolutions with different filter sizes on the input images, performs Max Pooling, and concatenates the result for the next Inception module. e architecture uses a 1×1 convolution operation which reduces the parameters drastically.
is architecture is designed to solve the problem of computational expense, overfitting, and other deep learning model issues. e Inception model takes advantage of the multiple kernel filter sizes within the CNN, and rather than stacking them sequentially, it orders them to operate on the same level. Figure 5(b) shows the Inception architecture used by our HIT machine. e architecture has nine inception modules stacked linearly and has 22 layers deep (27, including the pooling layers). It uses global average pooling at the end of the last inception module. Compared with VGG networks, Inception networks are more computationally efficient in terms of the number of parameters generated by the network and the computational cost incurred. For more details on the Inception model, refer to [16].
Our HIT machine also analyzed the impact of the ResNet architecture for activity recognition. e main idea of ResNet architecture is to avoid poor accuracy when the model uses deeper layers. is model is mainly designed for the gradient vanishing problem. Figure 5(c) shows the ResNet architecture used by our HIT machine. e ResNet architecture is a 34-layer plain network inspired by VGG-19 networks, which adds shortcut connections. ese shortcut connections then convert the ResNet architecture into the residual network. e first two layers of the model are the same as those of the Inception model. e model uses a 7×7 convolution layer with 64 output channels and a stride of 2 followed by the 3×3 maximum pooling layer. e major difference with ResNet is the batch normalization layer which is added after each convolutional layer. e inception model discussed previously uses four modules which are made up of Inception blocks. However, the ResNet architecture uses four modules which are made up of residual blocks. Each residual block uses several residual blocks with the same number of output channels. e first module from the architecture uses the number of channels that are the same as the input channel numbers. From the first residual block of each subsequent module, the number of channels is doubled compared with the previous module, and the height and width are halved. Compared with Inception architecture, the ResNet model is more straightforward, easy to modify, easy to optimize, and achieves higher accuracy when the depth of the network increases. For more details on ResNet architecture and its implementation, refer to [15].
At last, our HIT machine used a model called Effi-cientNet from Google for activity recognition [17]. In EfficientNet, a new scaling method called compound scaling is introduced. e model ResNet discussed before follows a conventional approach of scaling the dimensions arbitrarily and adding more layers. However, if the model scales the dimensions by a fixed amount simultaneously and does so uniformly, the model achieves better performance. e user can decide the scaling coefficients. EfficientNet architecture is a convolutional neural network architecture with different scaling methods. In EfficientNet, the architecture uniformly scales all depth/width/resolution dimensions using a compound coefficient. Compared with conventional ways that arbitrarily scale these factors, the scaling method in the EfficientNet architecture uniformly scales network width, depth, and resolution with a set of fixed-scaling coefficients. Figure 5(d) shows the EffientNet architecture used by our HIT machine. e main building block of this architecture consists of mobile inverted bottleneck Convolution (MBConv), to which squeeze-and-excitation optimization is added. e MBConv layer is similar to the inverted residual blocks used in MobileNet v2 [97]. e MBConv creates a shortcut connection between the beginning and end of a convolutional block. e input activation maps are first expanded using 1×1 convolutions, increasing the depth of the feature maps. 3×3 depth-wise convolutions and pointwise convolutions follow this, and this structure reduces the number of channels in the output feature map. e shortcut connections connect the narrow layers, while the wider layers are present between the skip connections. is form of structure decreases the overall number of operations required as well as the model size. For more details on the EfficientNet architecture and its implementation, refer to [17].

Experiment Results and Analysis
We collected HAR datasets from different users to validate our proposed HIT machine-based HAR approach. ere were 10 volunteers for data collection, consisting of five members for the training dataset and five for the unseen dataset. e demographic information of participants is given in Table 1.
We used Samsung galaxy note eight and iPhone 11 pro smartphone models for video recording. e smartphones were kept stationary during the initial stage of the experiment and moved their positions based on the user's motions. e users made their activities within the 15 m experiment area. We also used the IMU and stretch sensors and recorded the sensor reading from the users' activities during the Computational Intelligence and Neuroscience experiment time. Our conventional HAR approaches use the sensor reading for activity recognition, and we compared these HAR results with our HIT machine approach. Figure 6 shows the smartphones, IMU and stretch sensors, and experiment area involved in the HAR data collection. Table 2 summarizes our system configurations and hyperparameters used for model training and testing.
We started the analysis of the HIT machine by implementing deep learning models, such as VGG, Inception, ResNet, and EfficientNet. We tested these models with our HAR dataset, and Figure 7 Figure 7(a). e VGG model has a higher classification accuracy for sitting, sit-up, standing, and walking activities. e model has the highest misclassification error for running. Some of the running activity is misclassified as walking. Figure 7(b) shows the classification results from the Inception model. is model achieved a 93.18% classification accuracy with 0.13 model loss, 93.18% precision and recall, and 93.11% F1 scores, which are worse performances than the results obtained by the VGG model. Furthermore, Figure 7(c) shows the best classification results from our HIT machine based on ResNet architecture. e ResNet architecture showed the best model accuracy with the least classification errors. However, the model loss is higher than other models and needs higher computation time than VGG and Inception models. is model maintains the classification accuracy for basic and complex activities,  Subjects  1  2  3  4  5  6  7  8  9  10  Age  30  21  35  22  28  26  30  32  35  23  Height (cm)  175  180  172  160  174  162  176  165  168  159  Weight (kg)  80  84  87  60  70  58  78  62  85 57 19% precision, and 89.94% recall and F1 score, which has worse HAR performance than VGG, Inception, and ResNet models. e higher level of classification error from Effi-cientNet shows that this model is unsuitable for our HIT machine-based activity recognition. Figures 8 and 9 show the deep learning models accuracy and loss plots, and Table 3 summarizes their performance.
In Table 3, we used the accuracy, loss, precision, recall, and F1 score parameters for performance evaluation. e following equations from [98] define these parameters.
where the variables TP, TN, FP, and FN are defined as true positive, true negative, false-positive and false-negative in a given experiment. In the loss function, y i is the ith scalar value in the model output, y i is the corresponding target value, and the output size is the number of scalar values in the model output. From the results in Table 3, the ResNet architecture outperforms the other deep learning models with an average value of 98.53%. ese results indicate that the system trained with the ResNet model is the best choice for activity recognition. When we consider the training time results from Table 3, it shows that the ResNet-based HAR approach has a higher training time (600 s) than other models. is is due to the deep architecture of ResNet, and the system takes more time to train the model. However, the activity recognition results from ResNet compensate for the training time when considering the overall system performance (6.44% of classification improvements than EfficientNet model-based HAR system). In the case of VGG model-based HAR, the system achieved the most down training time (240 s) compared with other models and reached good classification results for activity recognition. e Inception model-based HAR system has a 60 s time difference for model training compared with the EfficientNet model. e EfficientNet has a lower training time (300 s) than the Inception model-based (360 s) HAR system. However, the EfficientNet-based HAR approach shows worst classification results than other models.
To further validate our HIT machine performance, we tested the pre-trained deep learning models with unseen HAR datasets. We collected another set of HAR datasets and tested them with our pre-trained models. Table 4 summarizes the results for unseen datasets from pre-trained models.
e results in Table 4 show that the ResNet architecture achieved 72.13% for classification accuracy with 72.25% precision, 72.92% recall, and 72.95% F1 scores. ese results outperformance the other pre-trained models. However, the computational complexity of this architecture makes it more practically challenging for real-time HAR applications. e classification accuracy from the Inception model shows that the model reached 65.17% for classification accuracy with 65.21% precision, 65.08% recall, and 65.59% F1 scores. e results from the Inception-based pre-trained model give better results than VGG and EfficientNet pre-trained models. In the case of VGG based pre-trained model, the system shows 61.85% for classification accuracy with 61.73% precision, 61.48% recall, and 61.47% F1 scores.
e Effi-cientNet pre-trained model-based HAR system shows 57.42% for classification accuracy with 57.48% precision, 57.43% recall, and 57.72% F1 scores. ese results show the worst classification results compared with other pre-trained models, and the approach is unsuitable for image-based HAR systems.
Next, we validated our HIT machine results with sensorbased HAR approaches and image-based HAR without HIT machine. Figure 10 shows the classification results from our HIT machine, HAR without HIT machine, and sensor-based techniques. is analysis uses a 2D CNN model for activity recognition. e CNN model is computationally lighter than other deep learning models and easily fits IMU and stretch sensor datasets. Figure 10(a) shows the classification results from IMU sensor-based HAR approach. e results show that the IMU sensor approach reached 90.71% of classification accuracy with 0.27 model loss, 90.47% precision, 90.71% recall, and 90.00% F1 scores. e activities that include running, sitting, sit-up, standing, and walking have higher classification errors due to the similarities of IMU  Computational Intelligence and Neuroscience sensor data. e model fails to classify these activities, increasing the classification errors in the HAR system. When the system uses a stretch sensor instead of an IMU sensor, the classification performance has a 3% improvement. e stretch sensor-based HAR system achieved 93.80% of classification accuracy with 0.27 model loss, 94.16% precision, 93.80% recall, and 93.20% F1 scores. Figure 10(b) shows the classification results from stretch sensor-based HAR approach. e stretch sensor data are more stable than the IMU sensor and have accurate HAR results. e activities that include sitting and walking have higher classification errors than the IMU sensor-based approach. e stretch sensor-based HAR approach is reasonable if the system cost is not a primary concern. e prohibitive cost of the stretch sensor makes the system more challenging for practical health care applications. Next, we analyzed a HAR approach that uses image data without a HIT machine. Figure 10(c) shows the results from a HAR without HIT machine. e HAR system without HIT machine reached 90.98% of classification accuracy with 0.20 model loss, 91.24% precision, 90.98% recall, and 90.90% F1 scores. ese results indicate the significance of the HIT machine. Compared with the results from Figure 10(d), the system without a HIT machine has a higher classification error and shows the worst performance for both basic and complex activities. e results from Figure 10(d) show the classification performance of the HIT machine, which has the best performance compared with other HAR approaches. e system achieved a 6.01% accuracy improvement compared with the IMU sensor-based approach and 2.4% accuracy improvement compared with the stretch sensor-based approach. e system also has a 5.3% accuracy improvement compared with the HAR approach without HIT machine. Our proposed HIT machine-based HAR system show 96.28% of classification accuracy with 0.09 model loss, 96.26% precision, 96.28% recall, and 96.27% F1 scores. Table 5 summarizes the performance of each approach in terms of accuracy, loss, precision, recall, and F1 score. From Table 5 results, the HIT machine shows the highest classification results than the sensor-based and without HIT  machine-based HAR approaches. e results indicate the impact of the HIT machine-based activity recognition for complex activities. e training time results from Table 5 indicate that the stretch sensor-based HAR system shows the best training time (120 s) than the other HAR systems. is is due to the small number of data samples from the stretch sensor dataset. In the case of the IMU sensor-based HAR approach, the system has a 300 s training time, which is 180 s higher than the stretch sensor-based HAR approach. Also, the classification accuracy from the IMU sensor-based HAR approach is 3.09% lower than the stretch sensor-based approach. e proposed HIT machine-based HAR approach shows 340 s training time, which is lower than HAR without HIT machine-based approach (480 s). e training time     results from our proposed HIT machine indicate that the approach reduced 140 s of training time compared with HAR without a HIT machine-based approach.
From the experiment and result analysis, it can be seen that the HIT machine-based HAR approach has a significant role in activity recognition. e proposed HAR system addresses the primary vision-based HAR system's challenge, such as processing high-quality images. We used image cropping, resizing, and data cleaning to make the system can perform the high-quality images without compromising the classification results. Our system takes advantage of the mask R-CNN algorithm, which is computationally lighter than other vision algorithms. e proposed method also solves the camera viewpoint and background clutter issues by considering the smartphone camera's wide-angle feature. e classification results from the HIT machine show that the proposed HAR approach is a valid method for healthcare applications, including abnormal activity detection, elderly care in homes, and disabled assistance. e extended versions of HIT machines are helpful in other applications, including intelligent environments, indoor navigation [99], security and surveillance, and people monitoring [100].

Conclusion
is paper proposed a HIT machine-based HAR system for healthcare applications. e proposed HIT machine approach effectively utilizes the advantages of the mask R-CNN for human body estimation and enhances the performance of the HAR. e classification results from our experiments indicate that the proposed HIT machine has better classification results than conventional sensor-based HAR approaches. e traditional sensor-based HAR systems are not free from sensor errors, showing very poor classification results for complex activities. e proposed HITbased HAR system is suitable for basic and complex user movements and maintains its classification accuracy in all user motions. Our HAR classification results and analysis show the influence of the HIT machine for activity recognition. e proposed HIT machine-based HAR system is a suitable healthcare option if HAR systems use a camera as their input device. We validated our proposed HIT machinebased HAR system for human activity recognition through extensive experiments and analysis. To improve the classification performance, we intend to use a sensor fusion technique that combines the image and sensor data for activity recognition in our future work. Furthermore, we will consider the most popular public datasets (UCI-human activity recognition using smartphones dataset) for future research and compare our HAR datasets' performance with public datasets.

Data Availability
e data used to support the findings of this study have not been made available because of the privacy of the research participant.

Conflicts of Interest
e authors declare that they have no conflicts of interest or personal relationships that could have appeared to influence the work reported in this paper.