Base Detection Research of Drilling Robot System by Using Visual Inspection

This paper expounds the principle and method of calibration and base detection by using the visual measurement system for detection and correction of installation error between workpiece and the robot drilling system. This includes the use of Cognex Insight 5403 high precision industrial camera, a light source, and theKEYENCEcoaxial IL-300 laser displacement sensor.The threebase holes method and two-base holes method are proposed to analyze the transfer relation between the basic coordinate system of the actual hole drilling robot and the basic coordinate system of the theoretical hole drilling robot. The corresponding vision coordinates calibration and the base detection experiments are examined and the data indicate that the result of base detection is close to the correct value.


Introduction
Industrial robotic applications are becoming widely used in the aviation sector [1] because of lower investment, increasing automation, table working performance, and good accessibility.From the beginning of twenty-first Century, American GEMCOR, EI (ElectroimPact), Italian COMAU, and German BROETJE-Automation have been committed to the design and development of drilling robot systems, and their drilling robot system production has been used in aircraft manufacturing enterprises, such as the F-16, F-22 vertical wall, F-35 aircraft wing panel, and A380 wing panel [2][3][4].S H Bi developed a drilling robot system, which uses an industrial camera to establish the relationship between the workpiece and the robot coordinate system [5].During the drilling process of the movable robot, the moving device locks floatingly when the drilling equipment is in the working position; then the robot will arrive at the designated location following the programmed workpiece coordinate system [6].Due to errors caused by the orientation of the motion device or the installation of the workpiece, differences in the relative location of the workpiece and the robot drilling system will occur, which will result in the offset of the actual arrival position [4,7].
To simplify this problem, working pieces are assumed to be in a theoretical position, and all installation errors can be concentrated on the robot coordinate system [8,9].The fundamental theory of base detection can be illustrated, first, by the actual position being checked through measuring the datum holes on the workpiece when the drilling robot system arrives at the designated location and, second, as the matrix between the theoretical robot coordinate system and the actual robot coordinate system being deduced to a generally correct position of the holes involved in programming.

Calibration of Base Detection System
In this paper, we have developed a drilling end effector system and installed it on a Kuka KR-500 robot as the experiment platform, as shown in Figure 1.
The calibration of the visual coordinate system is composed of two parts: the position calibration and the direction calibration [10,11].Position calibration is used to obtain the 2D coordinates of the robot Tool Center Point (TCP) in the visual coordinate system, and the readings of the four laser displacement sensors, where the distance between pressure nose tip and the workpiece surface is 45 mm.Direction calibration is used to obtain the rotation angle between the visual coordinate system and the tool coordinate system around the -axis of tool coordinate system, and the laser emission direction of laser displacement sensor [12,13].Because the vision coordinate system is fixed on base detection system, the calibration of the visual coordinate system described is consistent with the calibration of base detection system.

Position Calibration.
A special calibration board has been designed for position calibration where several holes are finished with certain relative positions.
As shown in Figure 2, the standard cutter mounted on the spindle is sheathed in the upper right hole of calibration plate.This hole is referred to as the spindle center hole.The remaining five holes in the calibration plate can be captured by adjusting the focal length of camera.According to the relative positions of the spindle center hole and the remaining holes (Figure 2(a)), the coordinates of the spindle center hole can be calculated in visual coordinates.
According to the image of spindle center hole taken by industrial cameras, the position in default coordinate system can be calculated by the contour detection and image segmentation algorithm [14,15], as illustrated in Figure 3.
The coordinate of spindle center hole in visual coordinate system is given by where   ⇀ 1 is the vector from  to 1,   ⇀ 14 is vector from 1 to 4,   ⇀ 12 is the vector from 2 to 1,  1 is the vertical distance from TCP to , and  2 is the radial distance from TCP to .
The laser displacement sensor should be installed on the right position to ensure that the laser is projected near the reference hole.Manual teaching robot is used to keep the front end of the pressure nose at a certain distance (programming safety height) from the workpiece surface.The reading of the laser displacement sensor  0 is recorded as the zero position of the laser displacement sensor by this time.

Direction Calibration.
The direction calibration of the vision coordinate system is mainly to obtain the direction of the optical axis of the industrial camera and laser displacement sensor.Calibration method is demonstrated in Figure 4 The calibration plate is fixed to the test bench, and the coordinates of the hole  on the calibration board in the camera coordinate system ( 0 ,  0 ) are recorded.Then, the industrial robot is moved a certain distance of  along the − direction of tool coordinate system and set while the calibration plate is remeasured with the industrial camera.If both the and -axes of the visual coordinate system are in accordance with the + direction and the − direction of the tool coordinate system, then the  coordinate value of hole  will increase  while the  coordinates remain unchanged.
If there is a certain angle  between the +-axis of visual coordinate system and +-axis of tool coordinate system (positive for clockwise, negative for counterclockwise), the coordinates of remeasured hole  will be replaced with ( 0 +  × cos(),  0 −  × sin()).Then angle  can be calculated by the coordinate values (, ) obtained from the actual measurements.
To calibrate the direction of laser displacement sensor, a series of planes that have a certain distance paralleled to the pressure foot plane should be measured, so that the plane angle between laser and pressure foot plane   can be easily obtained according to the distance between two adjacent planes and the reading of the laser displacement sensor, as shown in Figure 5.

Base Detection Theory
Two methods are used for base detection in this paper, including three-base holes detection and two-base holes detection.
Three-base holes detection is based on both their actual and theoretical locations to analyze the transfer relation between basic coordinate system of the actual hole drilling robot and the basic coordinate system of the theoretical hole drilling robot.
On account of the actual and theoretical position of one base hole, the actual position, actual normal, theoretical position, and theoretical normal of the other base hole, the two-base hole detection can be used to calculate the transfer relation.
As shown in Figure 6, the theoretical coordinates of threebase holes  1 ,  2 , and  3 in robot base coordinate system are given by

Theoretical position
Actual position Due to the error of workpiece installation and drilling robot orientation, the actual coordinates of three datum holes can be described as 3.1.Three-Base Hole Theorem.Direct calculation and least square matching are used to serve three-base hole theorem.
Direct Calculation.Coordinate system 1- is established by taking  1 for origin, the direction of   ⇀  1  2 for -axis, and the normal plane that contains  1 ,  2 , and  3 as the -axis (as shown in Figure 6).Theoretically, the transfer matrix from the robot-based coordinate to coordinate  1 - is The transfer matrix from the actual robot-based coordinate   -      to actual workpiece coordinate is written as where We can obtain  2 by substituting  1  by  1 ,  2  by  2 , and  3  by  3 .

Two-Base Hole Theorem.
As shown in Figure 7, twobase point detection is used to obtain the transfer matrix by knowing the actual position of point   1 , the actual normal direction   1 ,   1 , theoretical position  1 , theoretical normal direction  1 ,  1 , actual position   2 , and theoretical position of another point  2 .
The principle is shown in Figure 7.The workpiece coordinate system is established by the method shown in Figure 7, where  1 is the origin, the direction of   ⇀  1  2 is represented by the -axis, the normal vector of the plane that contains  1  2  3 is the -axis, and the -axis is determined by the right hand rule.The definitions of  1 and  2 are those as demonstrated in ( 5) and (6).
where  1 ,  1 are Euler angle of  that can be detected in normal direction.

Calibration Experiment of the Base Detection System.
First, the position calibration of the industrial camera and the laser displacement sensor is carried out.According to the installation method of the calibration plate shown in Figure 2, the image collected by Insight Explorer software is shown in Figure 8. Coordinates in the default camera coordinate system (units: pixels) are shown in Table 1.
The relation between the pixels and the distance (1 mm = 50.11pixel) is obtained by using the distance between the five calibration holes,  15 ,  14 ,  13 ,  12 = 10 ± 0.01 mm.After moving the default camera coordinate system to the central point of view, the coordinates of the calibration hole in the camera coordinate system are obtained and shown in Table 2.
According to (1), the coordinate of the spindle hole in the visual coordinate system (unit: mm) is given as (−186.5838,106.1815).
Figure 9 illustrates the direction calibration of the industrial camera.The calibration board is adjusted to the normal level after it is fixed to the test fixture in two different positions, making sure that the robot tool axis is perpendicular to the test fixture.Then the coordinates of 1, 2, and 3 are measured in the default camera coordinate (pixels), as shown in Table 3.
When the calibration plate is placed in position 1, the coordinate system in the robot base coordinates is (1354.51,−2622.831494.66,−90.08, −0.55, −1.57), and when in position the tool coordinate position in the robot base coordinate system is (1545.86,−2665.15,1461.93,−89.4,0.3, −1.56); when the drilling robot moves a certain distance in the + direction of the tool coordinate system, the coordinates of holes 1, 2, and 3 (pixel) are remeasured.The results are shown in Table 4.
After moving, the tool coordinate in the robot base coordinate system is (1354.According to the relative position of the calibration holes 1, 2, and 3 on the calibration board, the corresponding relationship between pixels and the distance can be obtained: 1 mm = 47.741pixel in position 1 and 1 mm = 50.2792pixel in position 2.
It can be seen from Table 5 that when the moving distances of robot are 4.493 mm and 11.5047 mm, the error between the actual value and the theoretical value is less than 0.06 mm, and the rotation angle of the tool coordinate system in  direction can be neglected.
The calibration method of laser displacement sensor is used to make the end effector tool perpendicular to the calibration plate with the normal adjustment function.Readings of sensors in  direction are shown in Table 6 with different height gauge added onto the calibration plate.The plane angle of the laser displacement sensor and pressure foot is given as arcsin(10/10.4).
The sensor reading is  0 = −34.8mm where the height between the front face of pressure nose and calibration plate  = 45 mm.

Base Detection Experiment of Robot Drilling
System.This study has determined that the three-point detection theorem is best suited to verify the correctness of the benchmark detection principle.The experiment field is shown in Figure 10.The calibration board is fixed on the test fixture and in order to reduce the coordinate acquisition errors, the normal adjustment function is used to control the verticality of tool and calibration plate under 0.1 degrees before industrial the camera takes images.
The drilling robot is taught manually to make the industrial camera lens center aligned to the base hole; then the laser tracker is used to verify whether the end effector spindle is perpendicular to the calibration plate.The three holes on the calibration board are measured and the coordinates of the corresponding cameras and the laser displacement sensors recorded (unit: pixel).The readings in the robot teaching device (unit: mm) are shown in Table 7.
The manual work piece coordinate system is set to [12, −5, −12, −0.3, −0.05, −0.01], the robot pose is controlled to repeat positioning to the gesture in Table 8 (in work piece  coodinates), and the board holes are recalibrated to achieve the pixel coordinates shown in Table 9.
The three coordinates of 1, 2, and 3 are converted to the robot base coordinate system, and the coordinates of three points are obtained, as shown in Table 10.
By comparison, it can be seen that both two methods can find accurate solutions, but the least square matching method is closer to the true value.This proves that the detection principle is feasible.Deviation caused by the robot's positioning error and various conversion errors can be reduced by robot positioning accuracy compensation.

Conclusion
Improving the positioning accuracy of the drilling robot is significant in improving the quality hole drilling.The deviation of the relative position between the workpiece and the robot will affect the positioning accuracy of the drilling holes.In order to identify the actual relative position of the workpiece and the robot, the base detection should be done before robot drilling preparation.This study has discussed calibration using the base detection system, and the base detection technology principle, and then we designed the calibration experiment and base detection experiment to verify base detection methods.The data indicate that the result of base detection is close to the correct value and that the principle is feasible.

Figure 1 :Figure 2 :
Figure 1: The specific structure of drilling end effector and robot system.

Figure 5 :
Figure 5: Direction calibration of laser displacement sensor.

Figure 6 :
Figure 6: Base detection theorem of three-base holes.

Figure 7 :
Figure 7: Base detection theorem of two-base holes.

Figure 9 :
Figure 9: Direction calibration of industrial camera.

Figure 10 :
Figure 10: Base detection experiment of robot drilling system.

Table 1 :
Industrial camera's raw data of position calibration.

Table 2 :
Base hole's coordinate in camera coordinate system.

Table 3 :
Data before direction calibration.

Table 4 :
Data after direction calibration.

Table 5 :
The difference of coordinates before and after moving.

Table 6 :
Readings of sensors in  direction.

Table 8 :
Coordinates of base holes in robot coordinate system before changing coordinate system.

Table 9 :
Values after changing coordinate system.

Table 10 :
Coordinates of base holes in robot coordinate system after changing coordinate system.