Journal of Robotics
Volume 2018 (2018), Article ID 8767531, 9 pages
https://doi.org/10.1155/2018/8767531

Research Article

Base Detection Research of Drilling Robot System by Using Visual Inspection

Zhanxi Wang,1,2 Jing Bai,¹ Xiaoyu Zhang,¹ Xiansheng Qin,¹ Xiaoqun Tan,¹ and Yali Zhao²

¹School of Mechanical Engineering, Northwestern Polytechnical University, Xi’an 710072, China
²Xinlian College, Henan Normal University, Zhengzhou 451400, China

Correspondence should be addressed to Zhanxi Wang; zxwang@nwpu.edu.cn

Received 8 November 2017; Revised 11 December 2017; Accepted 21 December 2017; Published 31 January 2018

Academic Editor: L. Fortuna

Copyright © 2018 Zhanxi Wang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

This paper expounds the principle and method of calibration and base detection by using the visual measurement system for detection and correction of installation error between workpiece and the robot drilling system. This includes the use of Cognex Insight 5403 high precision industrial camera, a light source, and the KEYENCE coaxial IL-300 laser displacement sensor. The three-base holes method and two-base holes method are proposed to analyze the transfer relation between the basic coordinate system of the actual hole drilling robot and the basic coordinate system of the theoretical hole drilling robot. The corresponding vision coordinates calibration and the base detection experiments are examined and the data indicate that the result of base detection is close to the correct value.

1. Introduction

Industrial robotic applications are becoming widely used in the aviation sector [1] because of lower investment, increasing automation, table working performance, and good accessibility. From the beginning of twenty-first Century, American GEMCOR, EI (ElectroimPact), Italian COMAU, and German BROETJE-Automation have been committed to the design and development of drilling robot systems, and their drilling robot system production has been used in aircraft manufacturing enterprises, such as the F-16, F-22 vertical wall, F-35 aircraft wing panel, and A380 wing panel [2–4]. S H Bi developed a drilling robot system, which uses an industrial camera to establish the relationship between the workpiece and the robot coordinate system [5]. During the drilling process of the movable robot, the moving device locks floatingly when the drilling equipment is in the working position; then the robot will arrive at the designated location following the programmed workpiece coordinate system [6]. Due to errors caused by the orientation of the motion device or the installation of the workpiece, differences in the relative location of the workpiece and the robot drilling system will occur, which will result in the offset of the actual arrival position [4, 7].

To simplify this problem, working pieces are assumed to be in a theoretical position, and all installation errors can be concentrated on the robot coordinate system [8, 9]. The fundamental theory of base detection can be illustrated, first, by the actual position being checked through measuring the datum holes on the workpiece when the drilling robot system arrives at the designated location and, second, as the matrix between the theoretical robot coordinate system and the actual robot coordinate system being deduced to a generally correct position of the holes involved in programming.

2. Calibration of Base Detection System

In this paper, we have developed a drilling end effector system and installed it on a Kuka KR-500 robot as the experiment platform, as shown in Figure 1.

Figure 1: The specific structure of drilling end effector and robot system.

The calibration of the visual coordinate system is composed of two parts: the position calibration and the direction calibration [10, 11]. Position calibration is used to obtain the 2D coordinates of the robot Tool Center Point (TCP) in the visual coordinate system, and the readings of the four laser displacement sensors, where the distance between pressure nose tip and the workpiece surface is 45 mm.

Direction calibration is used to obtain the rotation angle between the visual coordinate system and the tool coordinate system around the -axis of tool coordinate system, and the laser emission direction of laser displacement sensor [12, 13]. Because the vision coordinate system is fixed on base detection system, the calibration of the visual coordinate system described is consistent with the calibration of base detection system.

2.1. Position Calibration

A special calibration board has been designed for position calibration where several holes are finished with certain relative positions.

As shown in Figure 2, the standard cutter mounted on the spindle is sheathed in the upper right hole of calibration plate. This hole is referred to as the spindle center hole. The remaining five holes in the calibration plate can be captured by adjusting the focal length of camera. According to the relative positions of the spindle center hole and the remaining holes (Figure 2(a)), the coordinates of the spindle center hole can be calculated in visual coordinates.

(a)

(b)

Figure 2: Visual system calibration plate.

According to the image of spindle center hole taken by industrial cameras, the position in default coordinate system can be calculated by the contour detection and image segmentation algorithm [14, 15], as illustrated in Figure 3.

Figure 3: Position of spindle center hole (TCP).

The coordinate of spindle center hole in visual coordinate system is given by

where is the vector from to , is vector from to , is the vector from to , is the vertical distance from TCP to , and is the radial distance from TCP to .

As shown in Figure 3, can be obtained by substituting = 185 mm and = 109 mm into (1).

The laser displacement sensor should be installed on the right position to ensure that the laser is projected near the reference hole. Manual teaching robot is used to keep the front end of the pressure nose at a certain distance (programming safety height) from the workpiece surface. The reading of the laser displacement sensor is recorded as the zero position of the laser displacement sensor by this time.

2.2. Direction Calibration

The direction calibration of the vision coordinate system is mainly to obtain the direction of the optical axis of the industrial camera and laser displacement sensor. Calibration method is demonstrated in Figure 4

Figure 4: Direction calibration of industrial camera.

The calibration plate is fixed to the test bench, and the coordinates of the hole on the calibration board in the camera coordinate system are recorded. Then, the industrial robot is moved a certain distance of along the direction of tool coordinate system and set while the calibration plate is remeasured with the industrial camera. If both the - and -axes of the visual coordinate system are in accordance with the + direction and the direction of the tool coordinate system, then the coordinate value of hole will increase while the coordinates remain unchanged.

If there is a certain angle between the +-axis of visual coordinate system and +-axis of tool coordinate system (positive for clockwise, negative for counterclockwise), the coordinates of remeasured hole will be replaced with . Then angle can be calculated by the coordinate values obtained from the actual measurements.

To calibrate the direction of laser displacement sensor, a series of planes that have a certain distance paralleled to the pressure foot plane should be measured, so that the plane angle between laser and pressure foot plane can be easily obtained according to the distance between two adjacent planes and the reading of the laser displacement sensor, as shown in Figure 5.

Figure 5: Direction calibration of laser displacement sensor.

3. Base Detection Theory

Two methods are used for base detection in this paper, including three-base holes detection and two-base holes detection. Three-base holes detection is based on both their actual and theoretical locations to analyze the transfer relation between basic coordinate system of the actual hole drilling robot and the basic coordinate system of the theoretical hole drilling robot.

On account of the actual and theoretical position of one base hole, the actual position, actual normal, theoretical position, and theoretical normal of the other base hole, the two-base hole detection can be used to calculate the transfer relation.

As shown in Figure 6, the theoretical coordinates of three-base holes , , and in robot base coordinate system are given by

Figure 6: Base detection theorem of three-base holes.

Due to the error of workpiece installation and drilling robot orientation, the actual coordinates of three datum holes can be described as

3.1. Three-Base Hole Theorem

Direct calculation and least square matching are used to serve three-base hole theorem.

Direct Calculation. Coordinate system is established by taking for origin, the direction of for -axis, and the normal plane that contains , , and as the -axis (as shown in Figure 6). Theoretically, the transfer matrix from the robot-based coordinate to coordinate is

The transfer matrix from the actual robot-based coordinate to actual workpiece coordinate is written as

where

We can obtain by substituting by , by , and by .

The least square solution of transformation matrix can be calculated according to and [14, 15].

3.2. Two-Base Hole Theorem

As shown in Figure 7, two-base point detection is used to obtain the transfer matrix by knowing the actual position of point , the actual normal direction , theoretical position , theoretical normal direction , actual position , and theoretical position of another point .

Figure 7: Base detection theorem of two-base holes.

The principle is shown in Figure 7.

The workpiece coordinate system is established by the method shown in Figure 7, where is the origin, the direction of is represented by the -axis, the normal vector of the plane that contains is the -axis, and the -axis is determined by the right hand rule. The definitions of and are those as demonstrated in (5) and (6).

where , are Euler angle of that can be detected in normal direction.

We can obtain by substituting by , by , and by .

4. Base Detection Experiment

4.1. Calibration Experiment of the Base Detection System

First, the position calibration of the industrial camera and the laser displacement sensor is carried out. According to the installation method of the calibration plate shown in Figure 2, the image collected by Insight Explorer software is shown in Figure 8. Coordinates in the default camera coordinate system (units: pixels) are shown in Table 1.

Table 1: Industrial camera’s raw data of position calibration.

(872.5, 594.5)
(437.25, 344.5)
(1120.5, 158.25)
(623.5, 1028.5)
(1307.25, 842)

Figure 8: Image of calibration plate.

The relation between the pixels and the distance (1 mm = 50.11 pixel) is obtained by using the distance between the five calibration holes, , , , =  mm. After moving the default camera coordinate system to the central point of view, the coordinates of the calibration hole in the camera coordinate system are obtained and shown in Table 2.

Table 2: Base hole’s coordinate in camera coordinate system.

(1.4468, )
(−7.2392, 5.0988)
(6.396, −8.8157)
(−3.5223, 0.5513)
(10.1229, 4.8294)

According to (1), the coordinate of the spindle hole in the visual coordinate system (unit: mm) is given as (−186.5838, 106.1815).

Figure 9 illustrates the direction calibration of the industrial camera. The calibration board is adjusted to the normal level after it is fixed to the test fixture in two different positions, making sure that the robot tool axis is perpendicular to the test fixture. Then the coordinates of , , and are measured in the default camera coordinate (pixels), as shown in Table 3.

Table 3: Data before direction calibration.

Position 1
(947.25, 700.25)
(470.25, 686.75)
(959, 224.75)

Position 2
(1350.75, 878.25)
(847.75, 866.25)
(1365.5, 375.5)

Figure 9: Direction calibration of industrial camera.

When the calibration plate is placed in position 1, the coordinate system in the robot base coordinates is (1354.51, −2622.831494.66, −90.08, −0.55, −1.57), and when in position 2 the tool coordinate position in the robot base coordinate system is (1545.86, −2665.15, 1461.93, −89.4, 0.3, −1.56); when the drilling robot moves a certain distance in the + direction of the tool coordinate system, the coordinates of holes , , and (pixel) are remeasured. The results are shown in Table 4.

Table 4: Data after direction calibration.

Position 1
(949.75, 915.5)
(473.5, 902.5)
(961.25, 439.25)

Position 2
(771.25, 875.25)
(268.75, 863.75)
(785.5, 372.5)

After moving, the tool coordinate in the robot base coordinate system is (1354.68, −2622.78, 1499.15, −90.07, −0.55, −1.57) in position 1 and (1534.36, −2665.26, 1462.24, −89.4, 0.3, −1.56) in position 2.

According to the relative position of the calibration holes , , and on the calibration board, the corresponding relationship between pixels and the distance can be obtained:

1 mm = 47.741 pixel in position 1 and 1 mm = 50.2792 pixel in position 2.

It can be seen from Table 5 that when the moving distances of robot are 4.493 mm and 11.5047 mm, the error between the actual value and the theoretical value is less than 0.06 mm, and the rotation angle of the tool coordinate system in direction can be neglected.

Table 5: The difference of coordinates before and after moving.

Displacement of robot

Position 1
(0.0524, 4.5087)
(0.0576, 4.5191)
(0.0471, 4.493)
+: 4.493 mm

Position 2
(−11.5320, −0.0596)
(−11.5221, −0.0498)
(−11.5420, −0.0543)
+: 11.5047 mm

The calibration method of laser displacement sensor is used to make the end effector tool perpendicular to the calibration plate with the normal adjustment function. Readings of sensors in direction are shown in Table 6 with different height gauge added onto the calibration plate.

Table 6: Readings of sensors in direction.

Height of gauge (mm)
0
20
30
40
50
60
70

Readings of sensors in direction (mm)

The plane angle of the laser displacement sensor and pressure foot is given as .

The sensor reading is = −34.8 mm where the height between the front face of pressure nose and calibration plate = 45 mm.

4.2. Base Detection Experiment of Robot Drilling System

This study has determined that the three-point detection theorem is best suited to verify the correctness of the benchmark detection principle. The experiment field is shown in Figure 10. The calibration board is fixed on the test fixture and in order to reduce the coordinate acquisition errors, the normal adjustment function is used to control the verticality of tool and calibration plate under 0.1 degrees before industrial the camera takes images.

Figure 10: Base detection experiment of robot drilling system.

The drilling robot is taught manually to make the industrial camera lens center aligned to the base hole; then the laser tracker is used to verify whether the end effector spindle is perpendicular to the calibration plate. The three holes on the calibration board are measured and the coordinates of the corresponding cameras and the laser displacement sensors recorded (unit: pixel). The readings in the robot teaching device (unit: mm) are shown in Table 7.

Table 7: Values before changing coordinate system.

Coordinate of industrial camera
Value of laser displacement Sensor
Gesture of robot
(, , , , , )
/mm, /mm, /mm, /°, /°, /°

(855.5, 573.5)

(185.09, , 1689.50, , 0.026, 0.0259)

(881.75, 585.00)

(215.49, , 1689.62, , 0.021, 0.020)

(889.5, 590.75)

(185.57, , 1659.88, , 0.021, 0.024)

The manual work piece coordinate system is set to , −5, −12, −0.3, −0.05, , the robot pose is controlled to repeat positioning to the gesture in Table 8 (in work piece coodinates), and the board holes are recalibrated to achieve the pixel coordinates shown in Table 9.

Table 8: Coordinates of base holes in robot coordinate system before changing coordinate system.

183.894
213.73
183.64

−2608.07
−2608.31
−2608.34

1690.07
1689.94
1660.08

Table 9: Values after changing coordinate system.

Actual value
Coordinate of industrial camera
Value of laser displacement sensor

(620.75, 396.75)
−39.1

(637.75, 396.5)
−39.2

(648.25, 406.75)
−39.0

The three coordinates of , , and are converted to the robot base coordinate system, and the coordinates of three points are obtained, as shown in Table 10.

Table 10: Coordinates of base holes in robot coordinate system after changing coordinate system.

189.10
219.124
188.82

−2605.55
−2605.49
−2605.72

1694.05
1694.18
1664.03

According to the three-base hole detection principle, compensation matrix is given as , −6.99429, −12.833, −0.36, −0.08, by using direct method, and the compensation matrix is given as , , , , , by using least square matching method.

By comparison, it can be seen that both two methods can find accurate solutions, but the least square matching method is closer to the true value. This proves that the detection principle is feasible. Deviation caused by the robot’s positioning error and various conversion errors can be reduced by robot positioning accuracy compensation.

5. Conclusion

Improving the positioning accuracy of the drilling robot is significant in improving the quality hole drilling. The deviation of the relative position between the workpiece and the robot will affect the positioning accuracy of the drilling holes. In order to identify the actual relative position of the workpiece and the robot, the base detection should be done before robot drilling preparation. This study has discussed calibration using the base detection system, and the base detection technology principle, and then we designed the calibration experiment and base detection experiment to verify base detection methods. The data indicate that the result of base detection is close to the correct value and that the principle is feasible.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work is partially supported by the National Natural Science Foundation of China (Grant no. 51505380), Shaanxi Science and Technology Innovation Project (Grant nos. 2016KTZDGY06-01, 2016KTZDGY4-12), and the “111 Project” of China (Grant no. B13044).

References

A. Olabi, R. Béarée, O. Gibaru, and M. Damak, “Feedrate planning for machining with industrial six-axis robots,” Control Engineering Practice, vol. 18, no. 5, pp. 471–482, 2010.

J. Atkinson, J. Hartmann, S. Jones, and P. Gleeson, “Robotic drilling system for 737 aileron,” in Proceedings of the SAE 2007 AeroTech Congress and Exhibition, pp. 1–3821, SAE Technical Papers, Los Angeles, Calif, USA, 2007.

R. DeVlieg and E. Feikert, “One-up assembly with robots,” Training, 2008. p. 09-30, 2013.

R. Devlieg, K. Sitton, E. Feikert, and J. Inman, “ONCE (ONe-sided Cell End effector) robotic drilling system,” SAE Technical Papers, 2002.

S. Bi and J. Liang, “Robotic drilling system for titanium structures,” The International Journal of Advanced Manufacturing Technology, vol. 54, no. 5-8, pp. 767–774, 2011.

Y. Gao, D. Wu, Y. Dong, X. Ma, and K. Chen, “The method of aiming towards the normal direction for robotic drilling,” International Journal of Precision Engineering and Manufacturing, vol. 18, no. 6, pp. 787–794, 2017.

H. Chen, T. Fuhlbrigge, S. Choi, J. Wang, and X. Li, “Practical industrial robot zero offset calibration,” in Proceedings of the 4th IEEE Conference on Automation Science and Engineering, pp. 516–521, Washington, DC, USA, August 2008.

A. Nubiola and I. A. Bonev, “Absolute calibration of an ABB IRB 1600 robot using a laser tracker,” Robotics and Computer-Integrated Manufacturing, vol. 29, no. 1, pp. 236–245, 2013.

J. Wang, H. Zhang, and T. Fuhlbrigge, “Improving machining accuracy with robot deformation compensation,” in Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2009, pp. 3826–3831, usa, October 2009.

S. Garnier, K. Subrin, and K. Waiyagan, “Modelling of Robotic Drilling,” in Proceedings of the 16th CIRP Conference on Modelling of Machining Operations, CIRP CMMO 2017, pp. 416–421, France, June 2017.

B. Mei, W. Zhu, K. Yuan, and Y. Ke, “Robot base frame calibration with a 2D vision system for mobile robotic drilling,” The International Journal of Advanced Manufacturing Technology, vol. 80, no. 9-12, pp. 1903–1917, 2015.

A. Frommknecht, J. Kuehnle, I. Effenberger, and S. Pidan, “Multi-sensor measurement system for robotic drilling,” Robotics and Computer-Integrated Manufacturing, vol. 47, pp. 4–10, 2017.

Y. Liu, Z. Shi, P. Yuan, D. Chen, M. Lin, and Z. Li, “A visual positioning and measurement system for robotic drilling,” in Proceedings of the 14th IEEE International Workshop on Advanced Motion Control, AMC 2016, pp. 461–466, New Zealand, April 2016.

P. Arbeláez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 5, pp. 898–916, 2011.

P. Arena, A. Basile, M. Bucolo, and L. Fortuna, “An object oriented segmentation on analog cnn chip,” IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, vol. 50, no. 7, pp. 837–846, 2003.