Ultrasound Calibration for Dual-Armed Surgical Navigation System

Ultrasound (US) imaging system is widely used in robotic systems for precision positioning in clinical applications. The US calibration is critical to minimize the difference of spatial coordinates between instruments, for minimally invasive surgery (MIS) in navigation systems. In this study, we propose a dual robotic arm system that combines US imaging with one arm for path planning and monitoring and accurate positioning with the other arm for instrument placement via the preplanning procedures. A phantom with N-wire and N-wedge was designed for US calibration. The US calibration showed a mean error of 0.76 mm; the mean dual-arm calibration error is 0.31 mm. The positioning error of the system was verified with a mean error of 1.48 mm. In addition, we used two abdominal phantoms with computed tomography scan validation, with an averaged position error of 1.867 ± 0.436 mm and an orientation error of 2.190 ± 0.764°. The proposed system is aimed to perform clinical operations, such as abdominal MIS, with real-time image monitoring of the organ tissues and instrument positions, which meet the requirements for medical application.


Introduction
Surgical navigation is a critical system making the existing treatment procedures more accurate or approaching new possible procedures [1]. e procedures of minimally invasive surgery (MIS) require the targeted insertion of instruments for diagnostic or therapeutic purposes, such as biopsy, surgery, and radiotherapy [2]. e advanced image navigation system is assisted to provide patients with highprecision radiotherapy or surgical treatment and improvement of treatment effectiveness. Ultrasound (US) imaging is widely used in the clinic with its high temporal resolution, noninvasion, and high controllability. e US-guided navigation system can provide real-time monitoring, nidus targeting, and more accurate results [3]. Several integrated US-guided robotic systems have been proposed for precision positioning in clinical applications. For example, a mounted needle insertion with a US-guided robot was designed for percutaneous puncture which can improve the positioning and orientation accuracies than free-hand puncture [4]. In the integrated system, the US arm calibration is essential to minimize the difference of spatial coordinates between US images and instruments.
For the calibration between the US image and robotic arm with instruments, the localization of identified objects from the US image to system coordinate is based on the transducer position. e localization data of the robotic arm and the US transducer including position and orientation are required [5]. Trackers or mechanical arms are commonly used for collecting such data for calibration. Many trackerbased systems utilizing various tracking technologies have been proposed, providing the most maneuverability combined with US real-time imaging property [6]. In contrast, the mechanic arm can locate the positioning more accurately and has the ability to focus on a specific area or target by fixing the location with a US transducer [7]. To perform the US calibration, the phantom is commonly used to obtain the transformation from tracked coordinates to image coordinates. Various types of phantom models have been demonstrated, such as single or multiple point targets, crosswire, 2D shape alignment, Z-fiducial (or N-fiducial), and wall phantoms [8][9][10]. ese phantoms are designed with specific features to determine the world coordinates for the tracking system, and therefore, the objects observed in the US image can be located. However, for a US-guided navigation system, the treatment procedures are highly dependent on the operators [11]. Many free-hand US systems or US-instrument integrated system have been proposed to track the operations, but the localization and the operations with these systems depend on the medical experts' experiences that largely affect the outcome of the treatment [8].
To minimize the invasion and maximize the outcome of the treatment, in this paper, we propose a dual robotic arm system for MIS. e system combines US image acquisition with one arm for path planning and the accurate positioning with the other arm for instrument placement via the preplanning path. e calibration for the US image and the two arms was based on an in-house designed multiwire and N-wedge phantom. Finally, it is important to analyze not only positioning errors but also the orientation of the proposed system, and therefore, the system was verified by using a verification phantom and two abdominal phantoms for positioning and orientation accuracy test. e designed system is aimed to perform clinical operations with real-time image monitoring of the organ tissues and instrument positions. e organization of this paper is listed as follows: (1) e first part introduces the background, motivation, and the aim of this study. (2) e second part describes the materials and methods of the dual robotic arm system in detail, including the instruments, spatial transformation, calibration, and integration methods. e calibration and verification methods with the designed experiments are introduced. (3) e third part analyses the results of calibration experiments for US image, US to arm transformation, dual-arm calibration, and the results of the verification for the US imaging-guided dual-arm system. e accuracy of the system is described by the spatial deviation and orientation. (4) Finally, we summarize the system and the results of the experiments and point out the importance of the system for clinical requirements.

Materials and Methods
e procedures include four phases: the hardware calibration, the USarm operation, the TLarm operation, and the system validation ( Figure 1). e calibration procedures included the calibration between US image and arm, dual robotic arm calibration, the operations of two arms (the robotic arm with US transducer, USarm, and the robotic arm with tool, TLarm) with US image, and the validation. e dual robotic arms were calibrated by recording a set of the positions of robotic arms, and transformation between both was calculated. e coordinate system transformation from the US image to two arms was applied from hardware calibration. e system was validated by the difference between the target point and the standard stylus point. e symbols for the calibration system and spatial information used in this study are listed in Table 1.

Hardware Configuration.
e architecture of the system is comprised of four items, including two six-axis robotic arms, an ultrasound imaging system, and the control system. In this study, the calibration between US image and arm is performed by one robotic arm to acquire the US image of the phantom. e spatial information from the US image was identified based on the manufacturer and experimental data. e distance features between objects from the US image were used to compare the designed values for transformation calculation. With a combination of the dual-arm and image-arm spatial transformations, the compartments of the system were aligned and then validated by measuring the positions of two robotic arms by assigning the points from the US image. e calibrated system is designed for imaging, planning, and operating the preplanned procedures for surgical navigation. Two collaborative robotic arms of Universal robots UR5e (Universal Robots, Odense, Denmark) with 6 degrees of freedom were used in this system, which provide flexible control components for system setup.
e movement repeatability of UR5e is ±0.03 mm, and the working range is within 850 mm. One arm was designed to hold the US transducer and the other one was designed to hold the instruments, which were fixed on the trolley. e arm which held the US transducer was defined as USarm, and the other which equipped the tools (e g., stylus or instruments) was defined as TLarm. e US system of BenQ T3300 (BenQ Corporation, Taipei, Taiwan) was used, which is equipped with the transducer of C62 B sector-type linear array, 128 channels of the phased array, and the frequency of 2.0 ∼ 6.0 MHz. e output image resolution of the capture card is 1920 * 1080. A calibration phantom was designed in combination with N-wire and N-wedge. e wires provide precise spatial positions, and the planes have geometric features. It can improve the system accuracy from image processing. e sheet features are used to confirm the spatial positioning and to reduce errors caused by ultrasound image resolution. In addition, we designed three feature points p o , p x , p y P on the calibration phantom surface, which were used for the robotic arm to get the coordinates on the phantom. e calibration phantom was placed in a tank filled with water. e calibration phantom with the designed features is shown in Figure 2. For system verification, a homemade pyramid-shaped phantom and two abdominal phantoms, CIRS-057A and CIRS-071B (Computerized Imaging Reference Systems, Inc. (CIRS), Norfolk, USA), were used. e pyramid sharp points were designed for ultrasound imaging detection; the abdominal phantoms included several features such as organs or tumors that were used as target points. A coordinate-measuring machine (CMM), HEXAGON Romer absolute arm 7312 (HEXA-GON, Stockholm, Sweden) with point repeatability of ±0.014 mm, was used for measuring three-dimensional coordinates, providing the surface shape and coordinates of the object with a contact probe. CMM was fixed on the same platform with two robots and was able to probe the coordinate of the tip of the robots and the markers of phantoms. e symbols for the coordinate system and transformation between each coordinate system in the system are listed in Table 1.

Rigid-Body Transformation.
A rigid body refers to an object whose shape and size remain unchanged during movement and after being subjected to a force, and the relative position of the internal points of the object (e g., arms, instruments, or phantom) does not change. Rigid transformation of the plane preserves length, involving reflection, translation, and rotation. In this study, the leastsquares approximation method was used to transform the spatial coordinates of the two known sets of corresponding points. In order to transform an arbitrary set of points P I and P P from the W I to W P , the optimized translation T and rotation R need to be calculated. e minimum of the sum of squares was used for optimization, as follows: e control points P I and P P were known as corresponding points, and the optimal solution for transformation could be calculated based on a closed-form solution and quaternion method [12,13]. e cross-covariance matrix can be calculated by the following equation: where μ → I and μ → P are the center of mass. e cyclic components of the antisymmetric matrix A ij � (Σ IP − Σ T IP ) ij were used to form the column vector Δ � [A 23 A 31 A 12 ] T , and this vector was then used to form the symmetric 4×4 matrix Q(Σ IS ) as where I 3 is the 3×3 identity matrix. e unit eigenvector q ⇀ v � [q 0 q 1 q 2 q 3 ] t corresponding to the maximum eigenvalue of the matrix Q(Σ IP ) was selected as the optimal rotation. e rotation matrix is described as follows: e optimal translation vector T is given by    Figure 3 illustrates the transformations during ultrasound image calibration. e transformation T USarm I was calculated by the following equation: At the first step, we calculated the transformation between the USarm and the phantom (T P USarm ). e phantom coordinate system was preidentified by CMM. We operated the robotic arm with a standard stylus to each feature position of the phantom and recorded the robotic coordinates p o , p x , p y TLarm . e unit vectors v e eigenvectors v In the second step, we established the transformation between the ultrasound image and the calibration phantom (T I P ). e ultrasound transducer was used to scan the calibration phantom perpendicular to the base from above. Image denoising was performed by Gaussian binomial filter [14]. e edges of the objects in the US image were detected by Canny edge detection, and the circles and lines were segmented [15,16]. e circles on the image were corresponding to the lines of the calibration phantom, and the lines on the image were corresponding to the planes of the calibration phantom. After feature detection, six-point set P I � p 1 , p 2 , . . . , p 6 I and three-line set L I � l 1 , l 2 , l 3 I on the images were obtained. From P I and L I , the feature point set from the image P I � p 1 , p 2 , . . . , p 12 I was identified, which was the corresponding phantom point set P P � p 1 , p 2 , . . . , p 12 P , as shown in Figure 2(a). e image pixel size T scale was calculated by P I and P P , and S x and S y represented the scale of the X-axis and Y-axis in the US image by following equations: P P is only defined with the X-and Y-axis values. In order to obtain the scan position in the three-dimensional space, the Z-axis value, denoted as P P−z , is needed to be calculated. Based on the design of the calibration phantom, the coordinates m 1 , m 2 , m 3 in the coordinate system of the calibration phantom are shown in Figure 2(b). e points p 10 p , P P p 12 p , m 1 , m 2 , and m 3 were used to calculate P P−z based on similar triangles, as shown in Figure 2(b). e equation is as follows:  Journal of Healthcare Engineering e transformation T I P between the ultrasound image and the calibration phantom was obtained from the image featured point set P I and corresponding phantom point set P P . Finally, the coordinate systems of the US image and USarm were aligned by the transformation T USarm I � T P USarm * T I P .

Dual Robotic Arm Calibration.
In this section, we describe the dual robotic arm calibration method and verification by CMM. To integrate the dual robotic arm workspace for collaborative working, the transformation T TLarm USarm between both the robotic arms needed to be obtained. Two robotic arms, CMM, and the phantom were mounted on the stable platform and not moved during the measurement procedure. Before measuring the position of the dual robotic arm, each robotic arm's tool center point (TCP) was defined at the original flange. We predefined a set of the robotic arm's TCP position P def � p, p 2 , p 3 , . . . , p n def and operated the robotic arm to the target pose for each robot arm. is defined pose was within a specific area that CMM was able to measure it. e CMM was used to locate the featured mechanism on the robotic arm's flange and record the three-dimensional coordinate in each predefined position. e set of measured points was defined as P USarm � p, p 2 , p 3 , . . . , p n USarm and P TLarm � p 1 , p 2 , p 3 , . . . , p n } TLarm . According to the rigid transformation method stated above, we were able to compute the transformation T CMM USarm and T CMM TLarm between the robotic arm and the CMM coordinate system by two known sets of corresponding points. e transformation T TLarm USarm between two robotic arms was defined as follows:

System
Integration. Figure 4 illustrates the system architecture with two arms, the US transducer, the standard stylus, and the phantom. e positioning of the proposed dual robotic arm system was aligned from the US image coordinate system (W I ), two arms coordinate systems (W USarm and W TLarm ), calibration phantom coordinate system (W P ), and stylus coordinate system (W s ). e coordinate transformations of US image to USarm (T USarm I ), USarm to TLarm (T TLarm USarm ), and TLarm to standard stylus (T S TLarm ) were computed for the alignment of coordinate systems. e equation of the transformation between the target points p target in the coordinate systems of US image and stylus from W I to W s is described as 2.6. System Verification. e verification of the dual robotic arm system was tested by various phantoms in two phases. First, the homemade pyramid-shaped calibration phantom was used ( Figure 5). e shape point of the pyramid was set as the target point. e TLarm with a stylus was driven to the target point by setting an appropriate path. e CMM was used to measure the coordinates of the target point and the stylus tip for distance errors. e procedure was repeated 30 times. Second, the RFA Journal of Healthcare Engineering experiments with abdominal phantoms were performed to simulate the ablation in surgery (Figure 6). e TLarm with radiofrequency ablation (RFA) needle (CelonPro-Surge 150-T40) was driven to the target point by setting an appropriate path from the skin surface via US imageguided planning (Figure 6(a)) and 6(b)). e CT scans of two abdominal phantoms were performed before and after the needle insertion experiments (Figure 6(c)). e CT images were analyzed to extract the needle positioning and orientation errors for verification ( Figure 6(d)). e verification procedure with abdominal phantoms was repeated 14 times, 7 times for each. e RFA needle was implemented on TLarm for precise positioning. Two models of standard image guide abdominal biopsy phantom.
ese simplified abdominal phantoms are suitable for training and demonstrating image-guided needle biopsy navigation tools or procedures that require a constant visual reference for needle placement.

US Image Calibration.
Based on the N-wedge phantom proposed by Lin et al. (2018) [17], we designed a phantom with six wires and the planes for US calibration. e scanned features in the US image are shown in Figure 7. e pixel size of the US image was 0.16 ± 0.001 mm of the x-axis and 0.16 ± 0.002 mm of the y-axis, representing the width and the depth, respectively. Figure 7(a) is the original US image obtained by scanning the calibration model. After image processing and feature extraction (Figures 7(b) and 7(c)), US image with 12 feature points was identified (Figure 7(d)). From these featured points, US image calibration is to calculate transformation between US images and the calibration model. To verify the accuracy of the ultrasound image calibration method, we tested a set of independent verification experiments. After US calibration via calibration phantom, the ultrasound probe has performed the scan of the verification phantom and identified the feature points p target I on the ultrasound image. e feature position of the phantom was measured by CMM and denoted as p phantom CMM . e top feature of the pyramid was selected for verification ( Figure 5). We transformed p target I and p phantom CMM to W USarm and calculated the deviation. e verification was performed with the pyramid-shaped verification model. e registration error of the tip point between the US image and the verification model was calculated (Figure 8(a)). e mean x-axis, y-axis, and z-axis errors were 0.475 ± 0.464 mm, 0.193 ± 0.289 mm, and 0.311 ± 0.312 mm, respectively. Our US calibration showed a mean error of 0.759 ± 0.296 mm. e results of the US calibration were highly influenced by US image quality and resolution. Besides, the image artifact of phantom reflection, US beamwidth, and focal zone were the major factors for error accumulation in image processing.

Dual-Arm Calibration.
To verify the repeatability of the dual robotic arm after calibration, we first set three groups of position movements for USarm. We set the same coordinate movement and multiply the movement by T TLarm USarm for TLarm. e transformations between two robotic arms included translation and rotation. CMM was used to measure these positions of two robotic arms. e distribution of error distance is shown in Figure 8   Journal of Healthcare Engineering

Experimental Analysis with the Verification Phantom.
A standard stylus was used to perform the positioning for system verification. Target position p target I was transformed to robot base coordinate system p target S by predefined transformation (1). e robotic arm TLarm with the stylus was driven to the target pose and recorded the tool position p target CMM by CMM. e position deviations between p phantom CMM and p target CMM are shown in Figure 8(c). e mean system error in 10 experiments was 1.477 ± 0.405 mm. Our system indicated that the error distribution of the x-axis was relatively large, which might be the unstable positioning caused by the US beamwidth in the US calibration. Our results showed that the accuracy of US positioning can still be maintained at the same level compared to other studies [8,18,19].

Experimental Analysis of the RFA Insertion with Abdominal Phantoms.
e results of positioning and orientation error in 7 experiments for CIRS-057A were 1.828 ± 0.240 mm and 2.239 ± 0.947 degrees (Figure 9(a)) and in 7 experiments for CIRS-071B were 1.907 ± 0.593 mm and 2.141 ± 0.600 degrees (Figure 9(b)). e results of positioning and orientation error of these 14 experiments were 1.867 ± 0.436 mm and 2.190 ± 0.764°. In the postoperative CT images, the RFA needle showed some deviation during the process of entering the phantom. It might be attributed to two potential factors, the rigidity of the probe and the insertion angle. e rigidity of the needle might be led to the deviation caused by penetrating the tissues of the phantom while the needle might be bent by the surface elasticity. e positioning error could have resulted from the length of the needle and the elastic characteristic parameter of its materials. In addition, as shown in Figure 9(c), the positioning error  x-axis y-axis z-axis distance would be larger as the insertion depth increased due to the orientation error. Taken together, depending on the scenarios of surgery, the characteristics of instruments would be considered, and the trend estimation provides the reference for depth optimization while performing surgery. e proposed system showed the precise positioning and orientation by testing the error compared by the RFA practice and planning path with abdominal phantoms.
e dual robotic arm with the US imageguided system is aimed to perform clinical operations, such as abdominal MIS.

Conclusion
In this study, the operation of the system was designed for surgical procedures. We focused on the accuracy of the overall system and established a high accuracy robotic arm surgical guided system. e pros and cons of each hardware calibration and registration would greatly affect the accuracy of the proposed system. e dual robotic arm integration, US image spatial calibration, and system integration are critical factors for highly precise surgical robot system design. However, the collaborative robots UR5e were used for system implementation, but a certificated robot for use in surgery is needed in the next stage. In this work, the system is simply integrated with the coordinating transformation; however, the treatment is highly dependent on the accuracy of the positioning to perform the presurgical planning strategy. Our current results of the hardware calibration and system can provide the precision positioning to meet the clinical requirement for MIS. Further improvement for the clinical environment and workflow as well as adapting it to patient and operation environment for clinical applicability is needed.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that there are no conflicts of interest. Journal of Healthcare Engineering 9