Automatic Manipulator Tracking Control Based on Moving Target Trajectory Prediction

. The core issue of automatic manipulator tracking control is how to ensure the given moving target follows the expected trajectory and adapts to various uncertain factors. However, the existing moving target trajectory prediction methods rely on highly complex and accurate models, lacking the ability to generalize diﬀerent automatic manipulator tracking scenarios. Therefore, this study tries to ﬁnd a way to realize automatic manipulator tracking control based on moving target trajectory prediction. In particular, a moving target trajectory prediction model was established, and its parameters were optimized. Next, a tracking-training-testing algorithm was proposed for manipulator’s automatic moving target tracking, and the operating ﬂows were detailed for training module, target detection module, and target tracking module. The proposed model and algorithm were proved eﬀective through experiments.


Introduction
With the rapid development of industrial technology, manipulators have been successfully applied to original manual operations, becoming the most widely used manmade tool for industrial production [1][2][3][4][5][6].e application of manipulators makes production more efficient and flexible.e core issue of automatic manipulator tracking control is how to ensure the given moving target follows the expected trajectory and adapts to various uncertain factors [7][8][9][10][11].It is of great practical significance to derive an automatic tracking control strategy for moving targets under uncertainties and external interference.
Target tracking is an important prerequisite for manipulator-assisted services.Zhu et al. [12] improved the near-field computer vision system for intelligent fire robots.
e improved system can predict the falling jet path under the complex light environment and interference during firefighting, identify the jet path based on length and area ratio, and parametrize and extract the features of jet path by superposing radial centroids.Wu et al. [13] adopted a human-following method suitable for a manipulator containing visual sensors with a limited perception range, integrated two physical motion models into an adaptive trajectory prediction algorithm, and improved the prediction accuracy by adaptive adjustment of model parameters.For the trajectory control of Par4 parallel robot, Zhang and Ming [14] designed a type 2 fuzzy predictive compensation proportional-integral-derivative (PID) controller based on the improved dynamic gray wolf optimizer (GWO) based on the mutation operator and the eliminating-reconstructing mechanism (DMR-GWO2).e proposed controller speeds up the response of the parallel robot and improves the adaptability of the entire system.
In actual conditions, two manipulators are often needed to pick up and place moving objects through the planning and execution of collision-free trajectories.Tika et al. [15] put forward a layered control strategy for collaborative picking and placement tasks in a narrow, shared workspace and realized the synchronous execution of task scheduling in top-level design, path planning, and robot tasks.Xia et al. [16] proposed a visual prediction framework based on time granularity.
e core of the framework is an integrated moving target prediction module based on multiple long short-term memory (LSTM) neural network.Compared with the latest prediction algorithms, the framework excels in prediction accuracy, success rate, and robustness.Focusing on the action understanding of mirror neurons, Zhong et al. [17] simulated the walking mode of humanoid robots and predicted the moving direction according to the previous walking trajectory.
Trajectory prediction is the last step in the visual perception of the manipulator.After a series of segmentation, detection, and tracking, the algorithm could determine the type, bounding box, and other information of the object.However, the future movement trend and trajectory of the target must be predicted to realize automatic tracking control.To sum up, the traditional trajectory prediction methods for moving targets mainly rely on features such as color and contour.e recognition effect is very poor, if the target has multiple features.Moreover, the existing moving target trajectory prediction methods rely on highly complex and accurate models, lacking the ability to generalize different automatic manipulator tracking scenarios [18][19][20][21][22]. erefore, this study develops an approach for automatic manipulator tracking control based on moving target trajectory prediction, aiming to improve the manipulator's trajectory prediction accuracy and automatic tracking control effect.Section 2 establishes a moving target trajectory prediction model and optimizes its parameters.e established model can predict the position and pose of irregular moving objects at the same time and boast a strong generalization ability.Section 3 details the principles of the trackingtraining-testing algorithm for manipulator's automatic moving target tracking and explains the operating flows for the training module, target detection module, and target tracking module in the algorithm.e proposed model and algorithm were proved effective through experiments.
is study solves the problems of the manipulator in recognition, positioning, and trajectory prediction of moving objects, models the error in target tracking, and tests the feasibility of the proposed method through tracking experiments.
e internal parameters of the proposed trajectory prediction network for moving objects were all trained on datasets.e training ensures the degree of modularity and generalization ability of the network.However, the prediction precision of our network could be further improved by changing network structure and modifying network parameters, when the network is applied to predict the position and pose of complex and irregular moving targets.

Moving Target Trajectory Prediction Model
e precision of moving target trajectory prediction hinges on the accuracy of motion model.is study establishes a moving target trajectory prediction model based on LSTM, which is known for its good accuracy and generalizability, and further enables the manipulator to recognize, and automatically track and control the moving target.

Model Construction.
To accurately predict moving target trajectory, this study imports the three-dimensional (3D) spatial position of a moving target from time h to time h + K into the trajectory prediction model, which outputs the 3D spatial position of the moving target at time h + K + 1.
Figure 1 shows the overall structure of our moving target trajectory prediction model.e model consists of an input layer, a hidden layer, an output layer, and a training module.In the input layer, a complete sequence of moving target trajectories G � g 1 , g 2 , . . ., g m   is subjected to Z-score normalization: ( To satisfy the input requirements of the hidden layer, the input data were segmented.Let K be the prediction step length of the model.en, the tensor of the input data after the segmentation can be expressed as follows: Batch processing is applied on the input data to fully utilize computer resources and improve the training efficiency of the neural network.at is, A is treated as a tensor composed of a batch of 3D spatial coordinates [r, K, 3], where r is the number of batch processing samples. e training accuracy of the model must ensure that each batch of data is a complete trajectory of the moving target; i.e., the batch size should be defined as (m − K). en, we have the following equation: e theoretical output of the input layer can be expressed as follows: e hidden layer in the trajectory prediction model contains K LSTM nodes, which are connected in chronological order.Each node has F LSTM units.e output of the hidden layer can be expressed as follows: e dimensionality [r, K, F] of O should be consistent with that of model output.Let ω ti be the weight of a fully connected layer, and t be the output of the output layer.Before outputting the predicted position of the moving target, the data must be handled by a fully connected layer: To test the prediction accuracy, the number r of batch processing samples is set to 1. e first K 3D spatial coordinates of a complete trajectory in the test set are imported: Based on the input A g , the model outputs the predicted trajectory: 2 Scientific Programming Let t K+1 be the 3D spatial position predicted for the moving target at time K + 1. is position is merged with the last K − 1 3D spatial positions in A g to obtain the new input for the trajectory prediction model: en, A g+1 is imported to the trajectory prediction model.e model will output the predicted 3D spatial position t K+2 of the moving object at time K + 2. e above steps are iteratively executed, and the final prediction of the 3D spatial position of the moving object can be obtained as follows: e fitting and prediction accuracy of the model can be quantified by the error between input A and output t.
Both the predicted value and theoretical output of the trajectory prediction model are 3D spatial coordinates.e loss of the model is calculated by the Euclidean loss function.Let b be the theoretical output of the model.e error between predicted value and theoretical output can be calculated by the following equation: e model training aims to gradually reduce the value of the loss function.Based on the AdaGrad algorithm, the learning rate δ of our model is updated automatically.Let ξ be the small constant to prevent denominator from being zero; ω be the weight parameter of the model.en, the model can be updated by the following equation: Fullyconnected layer ere are many parameters in our trajectory prediction model.e most critical ones include prediction step length K, the number of hidden nodes F, and the learning rate δ.To weaken their influence on the prediction of moving target trajectory, this study firstly evaluates the prediction accuracy on all test samples and then chooses the optimal combination of K, F, and δ, which leads to the highest prediction accuracy.e objective function can be expressed as follows: e multilayer grid search algorithm is adopted to process K, F, and δ to determine the best values of these crucial parameters.e grid search is carried out from inside to outside in three steps: Step 1. Set the number of batch processing samples r and number of training steps T steps , which are two key parameters, and preset the value ranges of K, F, and δ based on formula (13).
Step 2. Traverse K, F, and δ layer by layer, and implement model training and prediction in the innermost layer.After the training, maintain the three parameters to obtain the fitting and prediction accuracies of the model.
Step 3. Sort the parameter search results in descending order by the prediction accuracy, and select the K, F, and δ in the top-ranking combination for the optimal model.

Automatic Tracking Control Algorithm
3.1.Algorithm Principles.Based on machine vision, manipulator moving target tracking might involve multiple moving targets at a time and needs to consider multiple motion states of each target.e moving targets face changes in moving direction, speed, color, and brightness, and could be occluded by obstacles.erefore, the tracking technology should be able to detect the 3D spatial position of each moving target in real time and judge whether the target is missing or occluded.is study proposes a tracking-training-testing algorithm for manipulator's automatic moving target tracking and combines the algorithm with moving target trajectory prediction to enable manipulators to grasp, as well as automatically track and control targets.
e automatic tracking algorithm can select the moving target from each frame image of the video stream.e architecture of the algorithm is shown in Figure 2.
e training module processes the detection result of the target detection module and the tracking result of the target tracking module.e processing and feedback results from the training module are used to update the target detection module and the target tracking module.
is cyclic optimization process can handle complex situations, such as the appearance changes in the moving target over time and the temporary disappearance of the moving target from the shooting range, thereby ensuring the target identification and tracking effects of the algorithm.
Let GYH be the normalized cross-correlation coefficient.To select the moving target from the video frame, the similarity between two adjacent frames w i and w j must be defined before analyzing the main modules: e matching image set N containing both positive samples w + i and negative samples w − i of moving targets can be expressed as follows: en, n positive sample w + i and n negative samples w − i are sorted in the order of i � 1, 2, 3, . .., n and then added to the matching image set.e similarity between a matching image N G and each frame w can be divided into the similarity with the nearest neighbor of w + i and the similarity with the nearest neighbor of w − i : e similarity between the frame w and the labeled first half of the positive samples can be calculated by the following equation: e cross-correlation of w can be calculated by the following equation:  Formula (18) shows that the value of RE s falls in [0, 1]. e greater the RE s , the more credible that the frame contains a moving target.e conservative similarity of w can be calculated by the following equation: e cross-correlation obtained by formula (18) is the threshold for the nearest neighbor classifier that determines the similarity (RE s , RE d ) between frame w and matching image N G .If RE s (w, N) > ρ MM , w is a positive sample; if RE s (w, N) < ρ MM , w is a negative sample.Here, RE s (w, N)-ρ MM is the classification threshold ensuring the convergence of the classifier.

Target Detection Module.
e variance classifier is the first link of the cascade classifier in the target detection module.Let Q (w) be the expectation of w solved by the integral image method.en, the variance of any frame w can be calculated by the following equation: If the total variance of gray values for all pixels in the frame within the window is smaller than half of the total variance of gray values for all pixels in the moving target box, then the window is invalid and needs to be removed.In this way, half of the image contents, including ground and shadows, can be eliminated.e ensemble classifier is the second link of the cascade classifier in the target detection module.
e frame outputted by the variance classifier is imported to the ensemble classifier composed of m basic classifiers.Here, each basic classifier is a decision tree (DT).e output of classifier i is a posterior probability vector composed of code a: e m classifiers output m posterior probability vectors.e mean of all vectors can be calculated by the following equation: If GV * > 1/2, the window is retained; if GV * > 1/2, the window is eliminated.
As the eigenvalue of the frame, the combined code vector is distributed to all the basic classifiers of the ensemble classifier.Each basic classifier corresponds to a posterior probability.
e ith posterior probability is denoted as GV i (b|a).If the posterior probability of each basic classifier is described by binary code a, then where During initialization, w i (b|a) � 0, and the posterior probability corresponding to each basic classifier characterizes a negative sample.During later training, the ensemble classifier classifies the labeled frames and updates w i (b|a) (b| a) (as shown in Figure 3).
Most unqualified contents are eliminated from the input frame through the filtering by both variance filter and ensemble filter.e filtered results are further processed by the nearest neighbor classifier.If RE s (w, N) > ω MM , the frame content in the scanning window is a positive sample.

Target Tracking Module.
e target tracking module combines the Lucas-Kanade (LK) optical flow method with the forward and backward error tracking theory.e forward and backward directions refer to the positive and negative directions of the sequence of video frames, respectively.If there is a large error between the target tracking results in the two directions, then the predicted trajectory of the moving target must be incorrect and unreliable.e forward-backward error helps to judge whether the moving target is tracked successfully, but cannot identify unobvious errors in trajectory prediction.erefore, this study designs an image frame difference comparison method for slowmoving target tracking points.e frame sequence of slowmoving target can be expressed as follows: FD � J τ , J τ+1 , . . ., J τ+v . (25) Let A τ be the coordinates of the moving target at time τ; v be the times of forward tracking of point A τ .
en, the forward trajectory tracking sequence of the moving target can be given by the following equation: e forward tracking and backward tracking are denoted by subscripts x and y, respectively.en, the pixel coordinates A τ+v are backward tracked to the previous frame.en, the backward trajectory tracking sequence can be given by the following equation: Combining formulae ( 26) and ( 27), the tracking error of the moving object can be obtained by the following equation: To sum up, the forward and backward tracking errors can be obtained by formula (28), as long as a suitable threshold is determined for different image sequences.en, it is possible to judge the success or failure of target tracking.Figure 4 illustrates the flow of tracking error calculation.

Training Module.
e training module contains the classifier to be trained, labeled training set, positive/negative training sample generator, etc. e classifier is trained on the training set to achieve comprehensive integrated learning.
e incorrect detection probability of negative samples can be characterized by the ratio of the number of correctly detected positive samples m + C to the number of incorrectly detected negative samples Φ: e reliability of negative sample labels can be characterized by the ratio of the number of correctly detected negative samples m − C to the sum of the number of correctly detected negative samples and the number of incorrectly detected negative samples m e incorrect detection probability of positive samples can be characterized by the ratio of the number of correctly detected negative samples m − C to the number of incorrectly detected positive samples Ω: e classifier performance evaluation equations (29)-(32) must satisfy the following equation: e number of incorrectly detected negative samples Φ and the number of incorrectly detected positive samples Ω can be, respectively, updated by the following equation: (5)  Scientific Programming . en, a 2 × 2 matrix Q can be defined as follows: After rewriting formulae (25) and (26) as matrices, the recursive formula of _ O(v) can be established as follows: e above formula shows that the recursive system of the manipulator's moving target tracking is both discrete and dynamic.us, the ultimate control goal of our algorithm is to gradually reduce the system error increment to zero, with the growing number of iterations.

Experiments and Result Analysis
e multilayer search algorithm was introduced to optimize the three parameters K, F, and δ of the proposed moving target trajectory model.Firstly, the number of the training steps was set to 120, and the value ranges of the three parameters were preset as follows: K∈{15, 20, 25, 30}, F∈{60, 120, 180, 240}, and δ∈{0.01,0.02, . .., 0.1}.e objective function is to maximize the prediction accuracy of moving target trajectory, i.e., minimize the prediction error.e possible parameter combinations were sorted in descending order of error.Table 1 lists the top five parameter combinations and their errors.It can be seen that the optimization of the three parameters greatly enhanced the accuracy of our moving target trajectory model.e three key parameters of the moving target trajectory prediction model were optimized as K � 30, H � 60, and δ � 0.08.Next, the hidden units in the hidden layer nodes were configured as recurrent neural network (RNN) and gated RNN (GRNN).
e prediction results of these two models were compared with those of our model (Table 2).Our model achieved better training accuracy and test accuracy than RNN and GRNN.
Figure 6 records the loss variations of different prediction models during the training.Overfitting occurs to the RNN when the training lasts too long; i.e., the number of iterations is too large.As shown in Figure 6, the loss of the RNN dropped the fastest, but the loss of our model gradually moved below that of RNN and GRNN, with the growing number of iterations.e prediction error was defined as the distance from the spatial coordinates on the predicted trajectory of the moving target to those on the actual trajectory.Table 3 compares the prediction errors of our model with RNN and GRNN.When too many trajectory points needed to be predicted, RNN had a lower prediction accuracy than GRNN and our model, because it cannot effectively process the historical positions on distant trajectories.Our model surpassed the GRNN by 56.7% in the prediction accuracy of the spatial coordinates on the trajectory of moving targets.
Figures 7 and 8 show the predicted trajectory of moving targets and the predicted grasping position trajectory of the manipulator.Figure 9 presents the prediction error of moving target trajectory.Table 4 lists the prediction error of moving target trajectory.Most errors were within 0.2 cm, which verify the generalizability of the proposed tracking control algorithm.
To verify the learning effect of our training module, the probability density of classification error was calculated.e classification error of the classifier fell in (−0.9142, 0.8747), which basically obeys normal distribution (as shown in Figure 10).

Conclusions
is study explores how to realize automatic manipulator tracking control based on moving target trajectory prediction.Firstly, a moving target trajectory prediction model was established, and its parameters were optimized.Next, a tracking-training-testing algorithm was proposed for manipulator's automatic moving target tracking, and the operating flows were detailed for training module, target detection module, and target tracking module.e experimental results show the effectiveness of the proposed model and algorithm.During the experiments, the parameter combination was optimized, the corresponding errors were obtained, and the values of three key parameters K, F, and δ were optimized.
e prediction results and losses of different models were compared, revealing that our model is more accurate in prediction than other models.Finally, the moving object trajectory and the manipulator's grasping position trajectory were predicted, and the prediction error of moving target trajectory was used to confirm the generalizability of the proposed tracking control algorithm.

Figure 1 :
Figure 1: Overall structure of moving target trajectory prediction model.

Figure 2 :
Figure 2: Architecture of manipulator's automatic moving target tracking algorithm.

Figure 5
explains the flow of the training module.During classifier training, the training quality is closely associated with the absolute number of labeled positive and negative samples.Hence, the training module should be able to quantify the relationship between the classifier performance and the absolute number of samples.e classifier performance can be characterized by the reliability of positive sample labels, the incorrect detection probability of negative samples, the accuracy of negative sample labels, and the incorrect detection probability of positive samples.e reliability of positive sample labels can be characterized by the ratio of the number of correctly detected positive samples m + C to the sum of the number of correctly detected positive samples and the number of incorrectly detected positive samples m + C + m + E :

Figure 3 :Figure 4 :
Figure 3: Generation of binary code.Note: subgraph (1) is the input image + window; subgraph (2) is the grayscale map of the input image + window; subgraph (3) is the fuzzified image-+ window; subgraph (4) is the pixel comparison image + window; and subgraph (5) is the final binary code.

Figure 5 :
Figure 5: Flow of training module.

4 P
it c h ( r a d ) R o l l ( r a d ) Yaw (rad)

Figure 8 :
Figure 8: Grasping position trajectory of the manipulator.

Table 1 :
Top five parameter combinations and their errors.

Table 2 :
Prediction results of different models.

Table 3 :
Prediction errors of different models.

Table 4 :
Prediction error of moving target trajectory.