Human Motion Posture Detection Algorithm Using Deep Reinforcement Learning

,


Introduction
Nowadays, with the widespread of the Artificial Intelligence (AI) in various fields, surveillance system has been born, which gradually expands the advantage of deep learning in the field of visual computers and clarifies the development direction of image processing technology [1][2][3]. After deep learning is optimized, related algorithms and models have been applied to identify, track, and detect human body's motion postures, and excellent results have been achieved [4,5].
ere are many researches on the basic theory of detection and tracking of human body's motion postures with enhanced deep learning. Starting from the field of surveillance and security, traditional surveillance technology is widely used in the military field, such as customs defense and borders. e detection and tracking technology based on deep reinforcement learning can be used to assist manual completion of designated tasks [6,7]. e application of deep reinforcement learning human motion posture detection to sports items can provide services for sports training viewing [8]. For skiing and short track speed skating events, it can reproduce the three-dimensional posture of the human body and improve the standardization of athletes' own movements. In the competitive arena, human motion posture detection can restore live scenes and reconstruct human motion targets, enhancing the audience's visual experience [9,10].
In view of the serious loss of details and low detection clarity of the above methods, this paper proposes a human motion posture detection algorithm based on deep reinforcement learning. e perception ability of deep learning is used to match the feature points of human motion, and by locating feature points of human motion posture, the position and direction of the human motion posture feature are determined, and the human motion posture feature is obtained. is method analyzes the contour of the human body motion posture, uses the training characteristics of the deep learning network and the reinforcement learning network to obtain the human body motion posture change information, obtains the general direction of the human body motion posture, and realizes the design of the human body motion posture detection algorithm. e contribution of this paper is as follows: (1) the algorithm in this paper uses deep reinforcement learning to detect human motion posture, determine the position and direction of human motion posture features, and obtain human motion posture features, which can improve the accuracy of human motion posture detail feature extraction. (2) e paper proposes an antigenantibody binding method to detect human motion posture.
is method uses the color histogram distribution of the human body motion posture as an antigen, searches for an area similar to the human body motion posture in the human body motion image, and uses the candidate region of the human body motion posture as an antibody. At the same time, this method realizes the extraction of human body motion posture features by calculating the affinity between the antigen and the antibody. (3) In this paper, the deep learning network is used to obtain the information of the human body movement posture change to improve the clarity of the human body movement posture contour, thereby obtaining the general direction of the human body movement posture, which can effectively reduce the amount of detail loss.

Related Work
e human motion posture detection algorithm has a wide range of research at home and abroad as it has huge economic and social value. Aiming at the problems in the recognition of human posture data, Cai et al. [11] extracted a small number of key sequence frames from human motion images and designed a recognition method based on this. On the premise of preselecting the original motion posture sequence, they constructed the initial key frame sequence of the human body posture.
is sequence is combined with the frame elimination algorithm to obtain the key frame sequence of posture, and the posture model is trained using the Baum-Welch method to recognize the posture. e method has better performance in the accuracy of human body posture recognition. Considering that the sliding window algorithm often fails to detect when collecting human body posture data, Zheng et al. [12] produced a human body posture data set by collecting time series data. In addition, they predicted and classified the human body's persistent behavior posture and sudden behavior posture by comparing long short-term memory (LSTM) with other networks and algorithms. e method can improve the accuracy of human posture detection by 4.49%. Gao et al. [13] designed a recognition algorithm using the posture signal sequence and used a three-layer recognition algorithm to recognize different postures. e results show that the recognition algorithm fulfills the requirements. Ren [14] designed a monitoring instrument for human hand movement, using nanogenerators to monitor hand movement posture. Liu et al. [15] combined human prior knowledge and performance capture of the human motion capture system. To improve usability of the system while maintaining high fidelity, the model was updated while capturing motion. Experiments show that this method has a certain degree of convenience. Zhao and Chen proposed a method for detecting and recognizing the posture of a moving human body based on sensor technology [16], which uses inertial sensors to recognize shooting, passing, dribbling, and catching. A shaft inertial sensor is placed on the arm to collect the experimental data. After smoothing and normalization, the frequency domain features are extracted, and 30 feature vectors are obtained by principal component analysis. e method is improved after dimensionality reduction. However, the details of the human posture are severely lost. HSIU-YU LIN et al. proposed a posture analysis method of convolutional regression neural network (RNN) in fall detection [17] and proposed a continuous deep learning model. e model receives a set of continuous images and is used to classify posture types using Microsoft Kinect as our nonwearable sensor. In addition, RNN is used to build a LSTM architecture, and our detection model is constructed by recognizing human postures in fall detection.
is method extracts features from preprocessed high-resolution RGB images.
is method obtains the body shape with real motion and depth information, but the definition of human body posture detection is low.

Recognition of Human Motion Posture Features.
When recognizing the posture features, the method uses deep learning to match the perception of human movement feature. e specific steps are as follows: Step 1: Establish the posture scale space using the deep learning network, and define the human body motion posture image L(x, y, σ) as the product of deep learning network algorithm G(x, y, σ) and the original image I(x, y). Namely, Step 2: Compare the posture feature in the scale space with the neighboring feature points to obtain the specific position and locate the human motion posture feature points. Namely, where k means space scale, and L(x, y, kσ) refers to new feature point.
Step 3: To recognize the human body motion posture, Equation (3) is used to determine the position and direction of the posture feature. Namely, 2 Mobile Information Systems (3) Step 4: Suppose x is the characteristics of human body motion posture represented by P, and y is the characteristics of human body motion posture represented by q. e Euclidean distance between the two features is calculated as e perception ability of deep learning is used to match human motion feature. By locating the posture feature points, the position and direction of the human motion posture feature are determined, and the human motion posture feature is recognized.

Analysis of the Human Motion Posture Contour.
Assuming that the camera device is stationary when detecting the posture, the differential image of posture is defined as D(k), and k is any pixel in human motion image. If it is within ω(k), the accumulated difference image is D(k). Conduct pixel normalization processing of human motion images, namely, If the pixel is static in ω(k), then the normalized difference of all pixels is D(k)/σ, all of which comply to normal distribution.
Choose an appropriate confidence value 10 − 6 ≤ a < 10 − 2 , and according to the χ 2 distribution, get the threshold of human motion posture analysis l a , in which a � p(Δ 2 > l a |H 0 ). en, it can be deemed that all the pixels that meet Δ 2 > l a have correlated movements.
According to the above process, the contour of the human body movement posture is analyzed.

Extracting of Human Motion Posture Features.
Suppose q � q u , u � 1 . . . m means the color histogram distribution of the human body motion area, which is defined as an antigen, and x 0 is the center of the area of the human body movement posture [18,19]. e human body motion posture image to be solved is x i , i � 1, . . . , n. ere are n pixels in x i , and the gray quantization level of each pixel is m, and u � 1, . . . , m is the characteristic value of human body motion postures. Construct the probability density estimation model of human motion posture: where b(x i ) means the quantified eigenvalue of human body motion posture image pixels at the characteristic space x i . It is usually used to analyze whether the posture pixel value in the human motion area is in u feature spaces. c means the estimated coefficient of probability density, δ is probability function, and h is the pixel value [20,21]. Assuming that the shape of the kernel function extracted from the posture feature is k(·). Since the human body motion posture image will be disturbed by the environment, the human motion posture image pixels close to the center are more stable than the distant pixels. k(·) can provide pixels with larger weights to pixels in the middle position, while pixels at a long distance can only get a small weight [22,23].
Assume p � p u , u � 1 . . . m means the color histogram distribution of human body motion posture image in candidate area. Define p as the antibody, and y is the posture of current frame. en, at the feature value in human body motion posture u, build a probability density estimation model [24], namely, where C P means the normalized coefficient of human body movement posture characteristics, which should meet the condition that n i�1 p u � 1 [25]. Assume that, at moment t, the affinity of the i antibody and antigen a i t can be described with the following equation: When extracting the features of posture, Equation (8) is used to measure the binding strength between antigen and antibody d [25], and the equation is where p means the similarity coefficient, which is between 0 and 1. e larger the value of p, the higher the similarity between the human motion posture feature and candidate area posture feature [26,27]. When p reaches the maximum value, the candidate area of the human motion posture will become the human motion posture feature to be solved in this frame of image ρ.
Mobile Information Systems where q u means the feature area of human body movement posture, and p u means the candidate region for posture feature extraction. When the posture feature is extracted, equation (11) is used to calculate the position of the posture.
where w i means the weight of posture features, and g is the specific posture. e color histogram distribution of the posture is used as the antigen, and search for an area similar to the posture of the human in the image. e candidate area of the human motion posture is used as an antibody, and the posture features are extracted by calculating the affinity between the antigen and the antibody.

e Paper Algorithm.
e deep reinforcement learning network not only has the perception ability of deep learning, but also has the decision-making ability of the reinforcement learning network. Combine the two networks to analyze the change information of the human body's movement posture, and obtain the general direction of the human motion posture. e global threshold segmentation method is used to retain the pixel points of the human body motion posture greater than the threshold, namely, where T refers to threshold value, and f(x, y) means human body motion posture pixels. According to the result of equation (12), the gray value of the human motion posture image is retained to facilitate the detection of the image edge [28].
Assume that, in the posture, the gray level of the original image is L. When the gray value of the image is i, the number of human motion posture images is n i . N is the total pixels, and then the segmentation calculation process of the human motion posture image is as follows: Step 1: By judging the gray level of the human posture image, the normalized histogram of the posture image is calculated, which is expressed as p i � n i /N, and then there is Step 2: In the original image of the human motion posture, calculate the average gray value of all pixels.
Step 3: Calculate the image edge detection threshold T According to the actual situation of the human body motion posture in the detection process, design the edge detection algorithm. It is shown in Figure 1.
According to the human motion posture information obtained from the analysis, the deep learning and reinforcement learning are combined, and the detection process of human motion posture is designed, as shown in Figure 2.
To sum up, according to the edge detection algorithm process, the edge contour of the human motion posture is roughly outlined, and deep learning and reinforcement learning are combined to complete the design of the human motion posture detection algorithm and realize the detection of human motion posture.

Experimental Environment and Data Set.
Since the production process of the posture data set is extremely complicated, multiple cameras need to be used in the collection process, supplemented by the full support of depth cameras and sensors. e data sets selected in this paper are as follows: Human 3.6M data set: the data set is composed of 3.6 million human motion images, all data are provided by 11 experimenters, that is, eleven different characters, each image is composed of different postures, the number of postures is 17, choose S1, S5, S6, S7, and S8 as the training set, and choose S9 and S11 as the test set.
MPI-INF-3DHP data set: this data set is a classic human motion posture data set, including 6 human objects performing 7 motion postures, and the posture data is obtained by motion capture system. CMU Panoptic data set: this data set uses the human posture data collected by 480 VGA cameras.

Experimental Standards.
For detecting the effect of the proposed algorithm, it is necessary to design experimental indicators for specific analysis.
(1) Human motion posture contour detection effect. e posture contour of human motion image is analyzed, and the detection effect of human motion posture contour is compared through the gray level of background area and the contour integrity of human motion posture. e higher the contour integrity, the better the detection effect. On the contrary, the higher the contour integrity, the worse the detection effect. (2) e effective area ratio of the human motion posture image. e effective area ratio is expressed by the ratio of the number of frame subblocks of the detected human motion image to the number of accurately identified frame subblocks. e higher the effective area ratio, the worse the detection effect and, on the contrary, the better the detection effect. (3) Accuracy AP value of human motion posture detection.

Mobile Information Systems
Accuracy of key point detection Pn is shown in the following equation: where TP is the correctly detected target number, and FP is the falsely detected target number. Similarity index of posture key points is shown in the following equation: where d i is the Euclidean distance between the detected key and the actual key. v i is the mark indicating whether the key point is occluded, s is the target size, and k i is the product constant for each key point. en, AP is obtained through the joint calculation of equations (16) and (17)

Results and Discussion.
To verify the performance using deep reinforcement learning in practical applications, this paper compares the algorithm in literature [12], algorithm of literature [13][14][15], and the following experiment was designed. It is shown in Figure 3.
In the three experimental data sets, using human body posture images as research data, four posture detection algorithms are used to detect the posture contour, and the detection comparison results are obtained.
According to Figure 3, in the posture contour of the human motion image, the gray level of the background area has a relatively small contrast, which causes the contour of the human motion posture to appear incomplete, and there is a relatively obvious void phenomenon. After comparison, Literature [13]algorithm Literature [14]algorithm Literature [15]algorithm The proposed algorithm Literature [12]algorithm (a)

Literature [13]algorithm
Literature [14]algorithm Literature [15]algorithm The proposed algorithm Literature [12]algorithm Literature [13]algorithm Literature [14]algorithm Literature [15]algorithm The proposed algorithm Literature [12]algorithm posture detection algorithm of literature [12] is found; the contour detection is obtained by the posture detection algorithm of literature [13]. e result has relatively poor edge integrity; the posture detection algorithm of literature [14] has a better human motion posture contour detection effect: the posture detection algorithm of literature [15] and the posture detection using deep reinforcement learning algorithm, and the completeness of the obtained human motion posture contour is better. e proposed algorithm has higher quality of human motion posture images, which improves the smoothness of the edges of human motion posture images, which further shows that the posture detection proposed algorithm designed has better results. In order to further compare the application performance of the proposed algorithm and the other three kinds of posture detection algorithms, the effective area of the posture contour and many blocks are selected as the measurement standard. It is shown in Table 1.
According to Table 1, this paper selects a human motion image sequence and divides it into 32 frames, 46 frames, and 60 frames, respectively. After processing the posture detection algorithm, when the human motion posture image is in a 32-frame sequence, the number of subblocks is less than the other five human motion posture detection algorithms. As the number of sequence frames increases, the number of subblocks of the proposed algorithm is always the smallest. After they compare that it is found that the human body motion posture detection algorithm based on deep reinforcement learning detects human motion posture, the detected effective area is higher than the other five algorithms. It shows that the proposed algorithm can reduce the amount of information loss in the posture image.
To verify the proposed algorithm effect, literature [12] algorithm and literature [13][14][15] algorithm are selected. And the method in this paper is used to analyze the accuracy of motion posture contour detection under different input resolutions. It is shown in Figure 4.
According to Figure 4, for human motion images at different resolutions, the accuracy values AP of motion posture contour detection are different. When the image resolution is 128 × 96 px, the algorithm in literature [12] has a motion posture contour detection accuracy of 52%, and the algorithm in literature [13] has a motion posture contour detection accuracy of 53.5%, and the algorithm in literature [14] has a motion posture contour detection accuracy of 56%, the algorithm in literature [15] has a motion posture contour detection accuracy of 57.5%, and the algorithm in this paper has a motion posture contour detection accuracy of 67%. When the image resolution is 1 384 × 256 px, the algorithm in literature [12] has a motion posture contour detection accuracy of 58%, and the algorithm in literature [13] has a motion posture contour detection accuracy of 59%, and the algorithm in literature [14] has a motion posture contour detection accuracy of 64%, the algorithm in literature [15] has a motion posture contour detection accuracy of 72%, and the proposed algorithm has the contour detection accuracy of 87%. e method in this paper can retain the original input resolution information in the human motion posture image monitoring, thereby enriching the local features and improving the detection accuracy.
In order to further test the recognition efficiency of this method in motion posture recognition, this problem can be transformed into solving the recognition time problem. On the basis of the above experiment, this paper uses the time interval measuring instrument to obtain the identification time and input the experimental data into the SigmaPlot 14.0 software to obtain the result. It is shown in Figure 5.
According to Figure 5, the size of the human motion image is different, and the time it takes to recognize the human motion posture is different. When the image size is 10 MB, the recognition time of the method in this paper is only 0.3 s, the recognition time of the algorithm in literature [12] is 6 s, the recognition time of the algorithm in literature [13] is 5 s, and the algorithm in literature [14]. e recognition time of the algorithm is 9 s, and the recognition time of the algorithm in literature [15] is 12 s. When the image size is 30 MB, the recognition time of the method in this paper is only 0.8 s, the recognition time of the algorithm in literature [12] is 16 s, the recognition time of the algorithm in literature [13] is 11 s, and the recognition time of the algorithm in literature [14] is 21 s, and the recognition time of the algorithm in literature [15] is 23 s. e method in this paper takes significantly shorter time to detect images, which shows that the recognition efficiency of the method in this paper is higher. is is because this paper introduces a deep reinforcement learning method to detect human motion posture.
is paper uses the perception ability of deep learning to match the feature points of human motion. By locating the feature points of human motion posture, the position and direction of human motion posture features are determined, and the human motion posture features are obtained. is effectively improves the recognition efficiency of motion posture recognition.
In order to verify the human motion posture detailed capture effect of the method in this paper, the human motion posture detailed capture rate is calculated by the algorithms in literature [12] and literature [15] and the method in this paper. e experimental results are shown in Figure 6.
According to Figure 6, it can be seen that the capture rate of human motion posture details is different under different methods. When the number of iterations is 100, the human motion posture detailed capture rate of the algorithm in literature [12] is 85%, the human motion posture detailed capture rate of the algorithm in literature [13] is 74%, the human motion posture detailed capture rate of the algorithm in literature [14] is 77%, the human motion posture detailed capture rate of the algorithm in literature [15] is 71%, and the human motion posture detailed capture rate of the algorithm in this paper is 95%. When the number of iterations is 500, the human motion posture detailed capture rate of the algorithm in literature [12] is 83%, the human motion posture detailed capture rate of the algorithm in literature [13] is 81%, the human motion posture detailed capture rate of the algorithm in literature [14] is 80%, the human motion posture detailed capture rate of the algorithm in literature [15] is 75.5%, and the human motion posture detailed capture rate of the algorithm in this paper is 98.5%. is  The proposed algorithm Literature [12]algorithm Literature [13]algorithm Literature [14]algorithm Literature [15]algorithm shows that the human motion posture detailed capture effect of this method is better. is is because this paper introduces the deep reinforcement learning method, takes the candidate region as the antibody, calculates the affinity between the antigen and the antibody, realizes the capture of human motion posture detail features, and effectively improves the detailed capture rate.

Conclusions
is paper uses the advantages of the deep reinforcement learning network to design a detection algorithm of human motion posture. By analyzing the characteristics of human motion posture, extracting the characteristics of human motion posture, and combining the two capabilities of deep learning and reinforcement learning decision-making, the results show that the proposed algorithm has a good performance. When the human motion posture image is in a 32-frame sequence, the number of subblocks is less than that of the other five human motion posture detection algorithms. As the number of frames in the sequence increases, the number of subblocks in this algorithm is always the smallest. e proposed algorithm can reduce the loss of information details in the human motion posture image. When the image resolution is 384 × 256 px, the motion posture contour detection accuracy of this algorithm is 87%. e method in this paper enriches the local features and improves the detection accuracy. When the image size is 30 MB, the recognition time of the method in this paper is only 0.8 s, which shows that the method in this paper effectively improves the efficiency of motion posture recognition. However, it takes a lot of time to collect images of human motion posture in this paper. erefore, the detection time of this method needs to be further optimized.

Data Availability
e data used to support the findings of this study are included within the article. Readers can access the data supporting the conclusions of the study from Human 3.6M, MPI-INF-3DHP data set.

Conflicts of Interest
e authors declare that there are no conflicts of interest with any financial organizations regarding the material reported in this manuscript. Literature [12]algorithm Literature [13]algorithm Literature [14]algorithm Literature [15]algorithm The proposed algorithm