6DOF Object Positioning and Grasping Approach for Industrial Robots Based on Boundary Point Cloud Features

For the three-dimensional (3D) pose estimation of metal blank casts estimate in industrial production process, a novel 6-degree-offreedom (6DOF) positioning and grasping approach for industrial robots based on boundary features and combining structured light 3D vision and the point cloudmatchingmethod is proposed.(e proposed approach first uses the Gray code plus grating phase-shift algorithm to reconstruct the 3D surface information of the object. Subsequently, an improved point pair feature (PPF) matching location method based on point cloud boundary extraction is proposed to realize the accurate 6DOF pose estimation of the object. In this method, the point cloud boundary feature is used to optimize the PPF algorithm. Finally, by combining with the point cloud preprocessing process and the improved PPF matching location method, the 6DOF pose positioning of metal blank casting by industrial robots is realized. (e proposed approach can accurately complete pose measurement and positioning of objects placed randomly in an open environment. (e experimental results demonstrate that the proposed PPF matching location method significantly improves both matching speed and accuracy compared to the traditional PPF algorithm.


Introduction
In recent years, industrial robot vision positioning in industrial production processes has been extensively researched. e industrial robot positioning and grasping workstation combined with machine vision technology not only saves labor cost but also ensures better safety and reliability compared with traditional manual operation. In the production and processing of metal castings, many problems, such as high work intensity and poor production environment, exist. e advantage of applying industrial robot vision positioning technology to the casting industry is obvious. e 6-degree-of-freedom (6DOF) pose measurement algorithm is the basis of a robotic bin picking task and has been studied by many researchers. Hinterstoisser et al. [1] proposed a line-MOD template matching algorithm that can be used for the 6DOF pose estimation of nontextured objects. On the basis of this work, Ye et al. incorporated the multilevel matching positioning strategy of an image pyramid and realized the real-time detection of 6DOF pose of objects [2]. Le and Len [3] proposed a deep learning-based method to realize a robot's disordered grasping. By establishing the target object database, the model was trained by using a Mask-RCNN detector; thus, the function of robotic bin picking was realized. A special three-finger gripper was used in [4], and its implementation in disordered grasping was studied.
Compared with the common problems in the 6DOF object pose estimation, such as occlusion, cluttered background, and object texture interference [5], in industrial production, the pose estimation of rough metal castings faces more complex problems. First, unprocessed rough metal castings have burrs on the surface, which can introduce errors in the pose estimation. Compared with processed products, the surface profile of unprocessed castings is smoother, which is not conducive to visual recognition. Second, the production environment of metal castings is harsh, which is not conducive to the maintenance of vision system hardware equipment. Finally, metal castings with a large number of repetitive or similar features (large planes) present more complicated problems in the pose estimation [6]. Such features easily conceal the local features of the object surface used for direction recognition, causing the failure of the direction recognition of the positioned object.
A three-dimensional (3D) matching method based on the point pair feature (PPF) is a widely concerned 3D pose measurement method [7]. is method shows successful object recognition performance on different 3D datasets [8]. In terms of recognition performance and recognition accuracy, PPF-based algorithms are comparable to learning class 3D object detection algorithms that gained popularity recently [9][10][11][12]. However, when the object itself has many repetitive features (such as large planes), the matching performance of the traditional PPF method will be reduced [6]. e disordered sorting of industrial applications also faces this problem. To solve this problem, some scholars have improved the PPF algorithm. In [6,13], the edge points of the point cloud are calculated using the depth map of the measured object, and matching calculation is performed to improve the 3D positioning accuracy. However, this method of calculation of the edge points of the point cloud is not accurate in complex environments. In [14], the edge points of the point cloud are calculated from multiple perspectives using CAD technology, and the matching calculation method of the point pairs is improved to realize high-precision 3D point cloud matching; however, this algorithm is complex and not applicable to industrial objects.
In view of the abovementioned problems, a combination of the binocular structured light vision technology and the point cloud registration algorithm with a 6DOF pose estimation approach for robots based on 3D matching is proposed. e proposed approach mainly includes three phases: 3D point cloud reconstruction based on binocular structured light, an improved PPF matching method based on boundary features of point cloud, and an industrial robot positioning strategy based on the PPF algorithm. is approach realizes the positioning and grasping of 6DOF pose of industrial robots for rough casting objects, and it achieves favorable processing speed and positioning accuracy. e main focus of the study is as follows: first, the existing binocular structured light technology is optimized, and fast 3D reconstruction of the measured object is realized. Second, an improved PPF registration algorithm based on principal component analysis (PCA) boundary extraction is proposed. en, through the improved PPF method, point cloud preprocessing, and robotic coordinate translation, the 6DOF pose positioning of metal blank casting by industrial robot is realized. Finally, a comparative experiment is conducted to prove the robustness and accuracy of the proposed method. e rest of the paper is organized as follows: Section 2 introduces the architecture and hardware composition of the proposed 6DOF measurement approach. Section 3 analyzes the key technologies of the proposed strategy. First, the point cloud generation technology based on binocular structured light is introduced; second, an improved 3D reconstruction method based on binocular structured light for metal blank castings is used. en, the traditional PPF 3D matching algorithm is optimized by combining point cloud boundary extraction algorithm with PCA. Finally, the object positioning online measurement process and robot coordinate transformation are introduced. Section 4 introduces the relevant experimental equipment and presents the experimental analysis. Section 5, finally, summarizes the study.

2.
Approach Architecture e proposed grasping system mainly consists of three parts: the binocular structured light vision system, industrial robot grasping system, and control system. e binocular structured light vision system collects image information of the object and realizes 3D reconstruction. e robot can cooperate with the vision system to achieve an execution mechanism for image acquisition and positioning. Based on the 3D information reconstructed from the vision system, the central control system calculates the grasping position and pose of the robot and plans the grasping path.

Operation Process.
e operation process of our approach is divided into three stages: 3D reconstruction, offline calibration, and online measurement. In the 3D reconstruction process, the 3D information of the object is generated. e offline calibration process completes the calibration of the binocular structured light vision system and creates the template required in the 3D matching process. In the online measurement process, the data of the offline calibration process are used to directly calculate the pose of the object and realize grasping. e operation logic of the proposed grasp system is shown in Figure 1.

Test Object.
e object used in this study for testing the grasping system is cast aluminum blank with a size of 100 mm × 100 mm × 5 mm. e surface of the object consists of many burrs and stains as well as many planar features. In addition, there are many similar features on the object surface, which hinders orientation. Figure 2 shows the status of a single object and the actual multiple objects to be grasped randomly.
In the actual production process, the upper part of the object randomly placed must be detected and grasped by the robot accurately.

System Description
To realized high-precision 6DOF positioning of industrial objects, the main technique of the novel approach consists of binocular structured light 3D reconstruction, an improved PPF point cloud registration method based on boundary features, and an industrial robot positioning strategy.

Generation of Structured Light by Gray Code and
Phase-Shift Method. e 3D reconstruction based on binocular structured light avoids the problem that a binocular stereo vision method cannot obtain the detailed 3D information of an object and solves the problem of complex calibration of the monocular structured light vision technology. Compared with the linear structured light technology, this method has the advantages of fast imaging speed and low cost. e projection method of the Gray code and phase-shift method is used for 3D reconstruction. With this method, Gray code coding does not require multiple image reconstruction, the high-frequency code image is not easily affected by noise, and the phase error of the phase-shift method is suppressed. e color projection image is used to improve the efficiency of the system. According to the test, the reflection of blue-green light by the object is good, so the projection image is composed of blue-green light (see Figure 3). By introducing the color projection image, the projection effect of 10 monochromatic light images can be achieved with 5 images. In the 3D reconstruction process, RGB images must be decomposed into single channel images. To simultaneously improve the calculation efficiency, shadow mask calculation is introduced into the method [15].   Mathematical Problems in Engineering e absolute phase diagram is calculated by the decoded code diagram and the dephased phase diagram as follows: where ψ(x, y) and M(x, y) are the absolute phase value and Gray code decoding value of pixel position(x, y), respectively.

3D Point Cloud Generation.
Zhang's calibration method [16] is selected to complete the double target determination and obtains the intrinsic and extrinsic parameters of the left and right cameras and their relative positions. e Bouguer stereo correction method [17] is used to eliminate distortion and aligns the left and right views; therefore, the imaging origin coordinates of the left and right views are consistent, the optical axes of the two cameras are parallel, the left and right imaging planes are coplanar, and the epipolar lines are aligned. Stereo matching aims to find matching points in the left and right absolute phase maps after stereo correction. e effective difference between the left and right phases is (h − n, h + n): where T is the contrast threshold and LG and RG are the phase values of (x, y) points in the left and right absolute phases. e corresponding point RP1 of LP1 in the left image can be obtained in the right image by satisfying Equation (2). From equation (3), the corresponding point LP2 of point RP1 in the left image can be obtained. If the distance between LP1 and LP2 is less than D1, then (LP1 + LP2/2) in the left image and RP1 in the right image are a pair of matching points.
After obtaining the matching points in the left and right images, they are combined with the camera calibration results; then, using the triangulation method, the 3D reconstruction of the measured object surface can be completed. e reconstruction process is shown in Figure 4.

3D Point Cloud Matching and Positioning.
Point cloud matching is one of the most popular methods for 6DOF pose estimation of objects. Commonly used methods include the viewpoint feature histogram (VFH) and the PPF-based 3D surface matching method. However, if these algorithms are directly applied to the object in this study, a small number of hollowed out area features in the plane point cloud will be covered by the dense area in the point cloud during the matching calculation. is leads to the failure of the plane point cloud on the Z-direction rotation axis of the object to be matched ( Figure 5). To solve this problem, this paper proposes a PPF matching location method based on edge extraction and realizes the accurate 6DOF positioning of the object.

Point Cloud Matching Algorithm Based on PPF.
For points m1 and m2 in the point cloud, the corresponding normal vectors are n1 and n2 and d � m2 − m1, and then, a PPF exists as follows: where ∠(a, b) ∈ [0, π] is the angle between two vectors and F is asymmetric. e matching process is divided into offline and online processes. In the offline process, the point cloud of the object to be matched is set as a model, and the feature vector F is calculated for all the point pairs on the template surface. en, a hash table is constructed to record the points with the same PPFs. In the online process, the same PPFs are extracted from the scene point cloud to be matched, and the similar point pairs are found in the template for pose registration. Finally, a voting mechanism is used to calculate the position and pose of the object that falls on most point pairs on the surface of the template.

PPF Matching Location Method Based on Edge
Extraction. To solve the problem that the original PPF matching algorithm cannot estimate the present object pose accurately, a PPF matching location algorithm based on edge extraction is proposed. is algorithm is divided into offline modeling and online matching stages.
In offline modeling, the PPF model file required in the online matching stage is created. e template can be obtained from a CAD file or by directly sampling the object with the 3D vision system.

Point Cloud Edge
Extraction. PCA is a common reduction algorithm. Bazazian et al. [18] proposed a nonclustering point cloud edge extraction method based on PCA, which extracts the boundary points of the point cloud according to the changes in the characteristic value of each point in the point cloud. is statistical method eliminates the sensitivity of normal estimation to sharpening edges, eliminates the clustering steps, and simplifies the process of edge estimation.
PCA-based edge detection method must calculate the covariance of each point in the point cloud. Covariance is a measure of how much each dimension changes relative to each other's average. For 3D datasets (x, y, z), the 3 × 3 covariance matrix C of sample point P (x, y, z) is given by the following formula: where cov (x, y) is the covariance of points x and y and is calculated as follows: e characteristic value of C is λ 0 ≤ λ 1 ≤ λ 2 , with Here, σ n (p) quantitatively describes the change along the surface normal, that is, the degree of deviation of the estimated point from the tangent plane [19]. λ 0 , λ 1 , and λ 2 reflect the position relationship between a point in the point cloud and the surrounding points: When λ 1 is σ ′ ≥ k × λ 1MAX , the point is considered the edge point of the point cloud. k is determined by the shape of the point cloud; for the object shown in Figure 2, let k � 0.8.

Point Cloud Model Generation.
rough the above calculation, the point cloud boundary of the object can be obtained. To improve the computational efficiency, the Euclidean clustering segmentation method is used to segment the point cloud boundary point set. Suppose the query point is p i ∈ P and P is the point cloud set. In addition, suppose there are n points in the neighborhood whose radius is r, that is, p ik , k � 1, 2, 3, . . . , n . In this study, we set r � 0.5. en, the Euclidean distance between the adjacent point and the query point is calculated as follows: where Q is the processed point cloud and d (.) is the distance between two points. rough the Euclidean segmentation, the contour point cloud Ps of the object can be divided into several subpoint clouds Pn; that is, Ps ∈ P1, P2, . . . , Pn { }. e point cloud is filtered according to the number of points in the subpoint cloud: Pu is the newly generated point cloud after filtering, Kn � 300 is the threshold of the number of point clouds, and size (Pe1) represents the number of point clouds in Pe1. e model point cloud is created by Pu (see Figure 6).

Online Matching.
In the online matching, the PPFs of the scene point cloud are calculated, and the matching object pose is calculated by a voting method (see Section 3.3).

Point Cloud Preprocessing.
To reduce external interference and improve the computational speed and robustness of the PPF matching algorithm, it is necessary to preprocess the point cloud acquired by the vision system (see Figure 7).
Due to the existence of sensor error and measurement error, there is inevitably some noise in the point cloud. e statistical filtering method is used to filter outliers in the target point cloud as follows: x n , y n , z n � savepoint1 |p(x, y, z) − μ| < σT outliner else where S is the neighborhood point set of point p in the point cloud, μand σare the mean and variance of the distance between the point and other points in S, and k � 30 is the number of midpoints of S. From equation (11), the filtered target point cloud can be obtained [20]. en, the random sample consensus (RANSAC) algorithm is used to segment the target point cloud to remove the plane background as follows: x n , y n , z n � plant point, d n < τ, outliner, d n < τ, where (a, b, c, d) are the plane parameters to be fitted, dn is the distance from any point cloud to the fitting plane, and τ � 6 is the threshold value of outliers in the control point cloud.
Finally, the pass-through filter is used to further remove the invalid point cloud as follows: outliner, else.
where T s⟶g represents the translation and rotation transformation from the reference point to the coordinate system in the scene, T m⟶g represents the translation and rotation transformation from the reference point to the coordinate system in the model, and R x (α) represents the rotation angle α around the positive half axis of the X axis.
To speed up the calculation, αis decomposed into x (−α s ) � R x (α s ) and combining with equation (15), the following is obtained: In [14], it is pointed out that, in the process of PPF matching, the point pairs whose distance is greater than the maximum size of the model cannot belong to the same target. erefore, the weight screening criteria are often added to the voting mechanism: where d(N) � ������������� dx 2 + dy 2 + dz 2 , where dx, dy, and dz are the maximum length of the object in the X, Y, and Z directions, respectively, and ω is the threshold value to ensure that the object does not exceed the fixed tilt angle. Voting weight selection criteria can improve the matching speed and accuracy of the PPF matching algorithm.
To improve the accuracy of the Z-direction angle recognition of the target object in the matching process, weight calculation is introduced in the calculation of the matching similarity score of the contour point cloud. For the proposed object, the contour point cloud is divided into boundary point cloud Pb and nonboundary point cloud Pnb: where S Main is the similarity score of single object matching, S Pb and S Pnb are the matching scores of the boundary point cloud and the nonboundary point cloud, and w 1 and w 2 are their corresponding weights.

Robot
Grasping. e position of the object relative to the end of the robot flange can be obtained by hand-eye calibration. e point cloud generated by the 3D camera is relative to the left camera coordinate system.
According to equation (16), the grasping mode of the robot for the object with different positions and postures can be calculated. e specific derivation process is shown as follows [21]: where U T 1 T is the posture of the six-axis flange in the robot coordinate system when the robot grasps the object, that is, the robot grasp posture; T T P is the position of the object in the robot grasp coordinate system; U T 2 CT is the posture of the robot when the vision system takes an image; CT T C is the posture of the vision system relative to the end of the robot flange, which can be obtained by hand-eye calibration; and C T P is the posture of the object relative to the vision system, which is directly acquired by the vision system. In this part, an industrial robot positioning and grasping strategy which includes the PPF matching algorithm based on point cloud boundary extraction method and robot coordinate translation method is proposed. e proposed method considers the point cloud boundary as the matching object and introduces weight optimization in the similarity calculation process. Compared with the traditional PPF matching method, our method has a higher matching efficiency and accuracy.

Experimental Platform.
A six-axis industrial robot with 20 kg load was selected for image acquisition and object grasping. A high-speed projector with a resolution of 1028 × 720 was selected for structured light projection, two cameras with a resolution of 1280 × 1024 were used to complete image acquisition, and an industrial computer was used for 3D reconstruction and 3D matching. Figure 8 is the photograph of the pose estimation system. e hand-eyes calibration parameter is shown in Table 1.

Analysis of 3D Reconstruction Data of Binocular Structured
Light. 3D reconstruction is the process of calculating the point cloud image through phase unwrapping and triangulation (see Figure 9). e point cloud information of the object surface is obtained by 3D reconstruction. e color projection can achieve the effect of 10 projection images of the monochromatic projection model with 5 images; thus, the efficiency of 3D reconstruction is improved.
Due to the hardware limitation of the structure light projector and the binocular color camera, only the blue-green light spectrum can be well separated. If better equipment is used, less 3D image reconstruction can be achieved.

Comparative Analysis of Point Cloud Matching Based on PPF.
e improved PPF matching algorithm realizes the disordered grasping of the object, which is a rough metal casting, by the industrial robot through the measurement of the position and orientation of the target object. A method of target recognition using CAD technology was proposed by Ulrich et al. [22] and optimized by Zhu et al. [23]. In this paper, Ulrich's method, the original PPF matching algorithm, and the proposed point cloud boundary PPF matching method are compared.

Comparison of Recognition Robustness.
e three methods were applied to the task of identification of a single object and multiple objects. e comparison the recognition effects of the three methods, shown in Figure 10, indicates that, with a good light source, Ulrich's method has better recognition effect for a single object; however, it has higher requirements for the light source and the recognition effect of disordered objects is poor. Hence, this method is not suitable for positioning randomly placed objects. e traditional PPF algorithm has a good positioning effect, except for the rotation axis in the Z-direction, but its accuracy is still insufficient. In this study,

Robot
Vision system Objects   the edge-based object matching method is better than the other two methods in both single and multiple object recognition.

Recognition Speed
Comparison. Ten objects were tested with the three methods, and the results are shown in Figure 11. Figure 11 indicates that the performance of the proposed method is better than that of the first two methods in terms of the matching time and similarity score for the metal castings tested in this study. Unlike the PPF matching algorithm, which only creates one model for the same object, Ulrich's method requires several models, which increase the matching time. e similarity score is used to measure the similarity between the tested object and the model object.
Because the matching feature of the proposed algorithm can better reflect the position relationship between the model object and the object to be tested, the similarity score of our method is also better. e proposed method was applied to the location test of different shape metal casts with different shapes (see Figure 12). Figure 12 indicates that the proposed improved PPF matching method is advantageous in both precision and speed for conventional metal castings. e PPF matching algorithm based on the boundary can extract the boundary points that are more representative of the object features in the point cloud and reflect the pose information of the object to be measured with fewer points. erefore, compared with the original method, the matching accuracy and matching speed are improved.
To verify the stability of the proposed method, a grasping measurement experiment was conducted as follows: the object was placed in the area for robot grasping pose estimation. e object grasping pose of the robot was obtained by hand-eye calibration, which was used as the pose of the robot grasping model. With the movement of the robot, the vision system measured the selected object from different positions many times, and the robot grasping position of the object was calculated after each measurement. e stability of the system was demonstrated by comparing the recorded values with the position and pose of the grasping model. Fifty measurements were performed, and the results are shown in Table 2.
As shown in Table 2, we can find when the number of experiments increases, the positioning accuracy of Ulrich's method is not high. is is because the interference of external light and other factors in the industrial environment leads to the 2D vision system's inaccurate extraction of the feature information of the object surface, resulting in the decline of matching accuracy and stability. Compared with the pose information of other axes, the original PPF matching method does not extract sufficient feature information for the Z-direction rotation of the object. As a result, the original PPF point cloud matching algorithm has small measurement errors for the A-and B-axis rotation in the X-, Y-, and Z-directions but fails to match the C-axis rotation. By contrast, the C-axis matching error of our proposed algorithm is within ±2°, and the matching accuracy of the other axis is also good, which proves the effectiveness of the proposed method.

Summary
In this study, a novel 6DOF positioning and grasping approach for industrial robots based on 3D matching was proposed. is approach can guide the industrial robot in realizing high-precision grasping of blank castings placed randomly and achieve a high recognition rate for repetitive and similar features, which is not possible with the traditional PPF algorithm. e main contributions of this paper are as follows. First, the 3D reconstruction method based on binocular structured light was used to realize the 3D reconstruction of objects with rough surfaces. en, a PPF matching location method based on edge optimization was proposed.
e proposed method not only overcomes the ineffective matching of planar features by the traditional algorithm but also improves the matching accuracy and speed. Furthermore, the proposed method was used in an industrial robot grasp strategy to realize metal object positioning and grasping. Finally, the stability and accuracy of the proposed method were confirmed by the comparative experiments conducted for various objects.
At present, the proposed algorithm uses only the surface profile features of the object and does not sufficiently use the radian features of the noncontour information of the object surface. is aspect will be investigated in our future work to further improve the pose estimation accuracy of the proposed system.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that they have no conflicts of interest.