Image-Guided Abdominal Surgery and Therapy Delivery

Image-Guided Surgery has become the standard of care in intracranial neurosurgery providing more exact resections while minimizing damage to healthy tissue. Moving that process to abdominal organs presents additional challenges in the form of image segmentation, image to physical space registration, organ motion and deformation. In this paper, we present methodologies and results for addressing these challenges in two specific organs: the liver and the kidney.


INTRODUCTION
Over the past 20 years, Image-Guided Neurosurgery has moved from the laboratory to the operating rooms (OR), and from concept to standard of care.It has brought better patient outcomes, reduced morbidities and shortened OR times [1][2][3].Yet looking only at renal cell carcinoma (RCC), hepatocellular carcinoma (HCC) and metastatic disease from colorectal tumors, then the occurrence of tumors in the abdomen is twice that of the brain [4][5].Even more of a motivation is that surgical resection in the liver has a 5 year-progression free survival of greater than 55% [6] and a clear margin in a partial nephrectomy has a similar cancer specific outcome as a radical nephrectomy but with improved patient outcomes and health impact [7].If improvements in surgery can be enabled, there is a good probability of excellent clinical outcomes.
In order to bring the advantages seen in image-guided neurosurgery to abdominal surgery, the five parts of image guided surgery must be addressed: (a) tomographic images with the proper contrast, spatial resolution and orthography, (b) a threedimensional localization and tracking system, (c) an image-space to physical-space registration methodology, (d) display of surgical position with respect to surgical targets and sensitive healthy structures, and (e) a correction for peri-operative changes [8].While image-guided neurosurgery has established this formalism, in order to properly address therapeutic guidance in the abdomen, specific challenges must be addressed.

METHODS 2.1. Image Information
Modern tomography has been around since the 1970's in the form of Computed Tomography (CT) and Positron Emission Tomography (PET).These technologies were of importance to guided surgery in that previous imaging methodologies had a third dimensional spatial uncertainty of the object being scanned.The development of MRI in the late 70s and early 80's provided better soft tissue contrast and became the dominant imaging modality for brain scans.
A tomogram is a four dimensional sampling process.It samples an imaging plane into unit quanta, generally resolved as square picture elements known as pixels.In the third dimension, there is a second spatial dimension describing the cross plane resolution, generally known as slice spacing or slice thickness.The pixel area times the slice thickness describe a volume of a tissue known as voxel and it is the interaction of the imaging process with that volume which gives rise to imaging gray scale value of a pixel.The fourth dimension of sampling is time.Since a person being scanned is moving due to cardiac and respiratory motions as well as voluntary and involuntary musculo-skeleton motion, data acquisition from a voxel is subject to blur if the acquisition time is long relative to motion rates.
Modern MRI sets generally have an in-plane size of 256 × 256 pixels, and it is not unusual to have a slice spacing such that the voxels formed are roughly cubic.New techniques such as low tip angle imaging [9] and compressed sensing [10] have significantly sped up image acquisition.
In CT, while other image sizes are possible, 512 × 512 reconstructions have been the norm for the past 15 years.As detectors have become more sensitive, it has been possible to reduce slice thicknesses thus reducing the voxel size while retaining image's signalto-noise ratio.However, it is in the temporal sampling that the modern CT has dramatically improved.Two technologies have led to this increase in speed.Helical scanning, where the gantry holding the patient is not stepped through the imager but moves continuously, reduces lost time due to the starting and stopping of the gantry.The second advance, that of simultaneous multiplane scanning, means that multiple slices are acquired in parallel [11].At the time of this writing, multislice CT is ubiquitous in US hospitals with some medical centers having 256-slice CT scanners.By acquiring images as quickly as these scanners can, complete abdominal image sets can be acquired in a single breathhold thereby reducing the image blur.This was cited by Walter et al. as being the source of improved imaging of the kidney by CT versus fast MRI [12].
While CT lacks the inherent soft-tissue contrast of MRI, in the application of liver and kidney images, it provides sufficient delineation of the shape of the organ.CT images a combination of density and atomic number as increasing either causes the CT pixel to brighten.The liver (Figure 1) with its packed hepatocytes has a higher density than the surrounding anatomic structures, with the exception of the heart.The kidney (Figure 2) is not as dense as the liver, but it is surrounded by a perirenal fat layer which is much less dense than the kidney and hence provides easy contrast.

Figure 1.
Abdominal CT scan showing the liver and other abdominal structures.

Figure 2.
Abdominal CT scan showing the kidneys in abdominal section.
The other major structures of interest in these organs are the blood vessels.They can be highlighted by the use of a contrast agent which temporarily raises the atomic number of the vessel.Effect of an iodine-based contrast agent can be seen in the CT scan in Figure 1 where the white dots in the liver are contrast-agent-filled blood vessels.
Ultrasound (US) has also been used in the guidance of procedures in both the kidney and the liver [13,14].While volumetric US imaging systems exist, B-mode US is inherently a two-spatial-dimension imaging modality.Therefore, it is difficult to determine the relative position of structures of interest and the objects encountered on the path to the target.That lack of three-dimensional image information coupled with the problem of a significant fraction of tumors being isoechoic on US [15,16] make US use in the OR more a matter of convenience than optimum choice.
A three-dimensional rendered surface allows for easier understanding of the spatial relations between surgical target (tumor) and healthy structures the surgeons may wish to avoid (blood vessels).This facilitates surgical pre-planning where the insertion of tools and placement of resection planes can be considered prior to the time-critical period in the OR.

Three Dimensional Localization and Tracking Systems
The first two image-guided surgery systems [23,24] used different classes of localizers: triangulation and articulated arms.Both have been used in abdominal surgery.An articulated arm generally uses fixed length members mounted on revolute joints.By knowing the length of the members and measuring the angles, the end tip position and orientation can be geometrically calculated.The articulated arm most in use in surgery CT image slice with segmentation line.
is the daVinci arm (Figure 5).Because its primary US application has been for prostate surgery, urologists use the system and have migrated its use into kidney surgery (e.g., [25]).While later developing, more applications for liver surgery are emerging [26].
The advantages of the daVinci are a long reach with little tremor in a laparoscopic application and a well-integrated display and haptic interface.The disadvantages are high purchase and operating costs as well as the limitations of any minimally invasive approach.Those limitations are due to the nature of the visualization.The user cannot see beyond the imaging cone of the laparoscope and cannot see into a solid organ to  The daVinci surgical guidance system.locate internal structures.Integration of image-guidance has begun in daVinci surgery [27,28].The dominant choice for localizer systems has been optical triangulation systems such as those from Northern Digital Inc. (NDI, Waterloo, Ontario, CA) and Image Guided Technologies (now owned by Styker, Kalamazoo, MI).These devices locate either optical sources or optical reflectors placed on the proximal end of a rigid tool.By measuring the tool's position and orientation, the location of the tip of a rigid tool can be calculated.Optical localizers have been used in both liver surgery [29,30] and kidney surgery [31].
Electromagnetic triangulation systems, such as the Aurora (NDI) and the systems from Ascension, have an appeal of not requiring line of sight between localizer and tool and thus is amenable to use with flexible tools.The challenge with such systems is that their accuracy falls off quickly as tool moves from the ideal center of localization [32].However, for applications not needing millimeter-scale accuracy, the advantages may outweigh the difficulties.Magnetically tracked applications have been tried in the liver [33] and applications for kidney have been discussed and may soon emerge.

Image Space -Physical Space Registration
Central to any image-guided surgery system is the process of registration, i.e., determining the mathematical relationship between objects in the tomograms and their physical locations in the operating room.This may be done based on points, surfaces or volumes.Point based registration has the critical advantage of known correspondence; i.e., each point in one space is matched to its location in the other space.This allows both the use of closed form least square error solutions [34,35] and the ability to assess the 208 Image-Guided Abdominal Surgery and Therapy Delivery An intraoperatively obtained surface (red) fit via ICP to a surface extracted from a preoperative CT scan [38].
quality of the registration by designating selected points as targets.The points used in the creation of the transformation matrix are known as fiducials.If the fiducials are native to the anatomy, they are intrinsic fiducials; if they are attached to the patient, they are extrinsic fiducials.A target is a point with known locations in both spaces, which is not used in the creation of the transformation matrix.The difference between the transformed location of a target into the second space and its actual location in that space is the Target Registration Error (TRE) [36] and is a true assessment of registration quality.
The second form of image space to physical space registration is to designate a surface in one space (generally physical space) and then match it to a second surface which has been extracted from preoperative images.While a number of techniques have been used for establishing the mathematical match, they are almost all based on the Iterative Closest Point (ICP) method put forth by Besl and McKay [37].As the title suggests, this is an iterative process and the focus of the majority of the development of this algorithm has been to speed the convergence and avoid falling into local minima.
Because the external surface of the head is available to the surgeon prior to bringing the patient to the OR, point based fiducials have become the norm.This is not possible in abdominal organs although a group of German researchers have tried using tracked US imaging systems to determine the location of vascular features such as vessel bifurcations as fiducials [39,40].Therefore most of the image-guided liver and kidney surgery work has been based on surface registrations [41][42][43].
Considerable challenges remain in the use of the surface registrations on abdominal organ.These challenges cluster around the concepts of exposure, deformation and validation.Intuitively, the more complete the surface description, the greater confidence one would have in the quality of the expected registration.However, even in open liver surgery, it is difficult to get access to the whole anterior surface of the liver especially if one uses a laser-range scanner (LRS) [44] in the attempt to get a regular, high density spatial sample.Because surface registrations lack point correspondence, they use a metric such as the distance between a point on one surface to the closest point on the other surface.This makes them vulnerable to surfaces with high rotational symmetries; the fit can "slide" between locations which generate similar metrics.Given that the biology of organs produces rounded structures, this is an ongoing challenge.One approach is to use some a priori knowledge about the surfaces to provide additional metrics to help "lock in" the registration.One approach for this has been the designation of structures which can be guaranteed to be visible in surgery as "salient features" [45].This methodology either by itself or in conjunction with a surface can provide similar quality registrations while reducing the chances for error.
In the kidney, there are two exposure problems.The kidney is veiled with perirenal fat which must be removed to measure the kidney surface in physical space.This is true in either open or minimally invasive cases.The second exposure problem is rapid movement of kidney surgeons to adopt minimally invasive approaches.Benincasa et al. [46] explored the relationship between the registration accuracy and the percentage of exposed surface in kidney registration.
The second challenge to surface based registration is the deformation of organs during the surgical process.While these deformations may arise from patient pose and organ motion due to breathing, most are caused by the surgical intervention.In liver surgery, the liver is generally mobilized prior to the surgical intervention into the liver parenchyma.The act of mobilization is the removal of ligament attachments between the liver and supporting structures.The intent of this process is to free the liver for both ease of surgical approach and in case of major bleeding, allow access to the major vessels underneath the liver.After mobilization, the liver is often packed with surgical sponges to support the tissue for surgery.This process changes the shape of the liver offering a challenge for a surface-based registration process.Possible solutions to this are discussed in section 2.5 below.
The kidney is significantly stiffer than the liver.However, at the start of a surgical procedure in the kidney, the renal artery and renal vein are clamped to minimize blood loss during the surgery.This results in a loss of turgor in the kidney and a small shape change [47].
The final challenge in surface-based organ registration is that of validation.As mentioned earlier, surface-based registrations lack one-to-one correspondence and are sensitive to rotationally symmetric surfaces.Therefore, the value on which the ICP regresses, mean closest point distance, provides no information on the quality of the registration.Additionally, the lack of a one-to-one correspondence means that a TRE cannot be calculated.Some ongoing work centers on finding a single intrinsic point to be used as a target for assessment.Other work includes statistical analyses to provide a confidence interval based on surface characteristics such as changes in surface curvature.

Intrasurgical Display
One of the most under-appreciated challenges of surgical guidance is the display of surgical position and orientation during the case.The display is attempting to provide seven dimensional information (X, Y, Z, Yaw, Pitch, Roll and Time) on a 2D, temporally active display.If the data are presented as a binocular pair, some idea of the relative positions of objects can be realized but at the cost of the information bandwidth.An additional hurdle is the conveyance of essential information to the surgeon in a manner easily absorbed by a distracted viewer.
Two of the earliest image-guided surgery systems [48,49] used a two over two display of the three cardinal image slice directions (transverse, sagittal and coronal) linked to the tracked surgical tool tip.This allowed surgeons to see where motion in one of those planes would take them.By using cardinal planes, the images were presented in a manner familiar to the surgeon, allowing for rapid acclimation.
Other techniques included reslicing the tomographic images along the plane perpendicular to the point tool or providing the information as cut planes in a rendered image.Having multiple image sets (e.g., CT, MRI, PET, SPECT) increases the complexity.Various fusion techniques have been tried but are rarely adopted by surgeons over concerns regarding intraoperative confusion.
Another source of surgical imaging of growing importance is intraoperative imaging.These can range from intraoperative magnetic resonance imaging (iMRI) [50], fluoroscopy [51], cone-beam CT [52] to US [53] and endoscopy [54].For these imaging types, the challenge is to integrate the new information into the surgical scenario.Especially difficult is the case when the preoperative and intraoperative data disagree such as showing a critical structure in two different locations.
None of the currently available intraoperative imaging has proven to be the answer.IMRI is both a major intrusion into the surgical field and prohibitively expensive.Fluoroscopy and CT are radiographic and involve cost/benefit analysis on the dose to the patient and OR staff.US is natively a two-dimensional imaging format and often structures are either iso-echoic to their surroundings or the low signal-to-noise nature of US obscures their presences.Endoscopy provides real-time color information but only in a cone directed from the tip of the 'scope.In addition, 'scopes cannot "see" inside solid organs and can be obscured by a bloody field.
By combining real-time intraoperative imaging with high resolution, high signalto-noise ratio, three-dimensional preoperative tomographic imaging, the strengths of both can be preserved and the weaknesses mitigated.As minimally invasive approaches to abdominal surgery increase in popularity, such techniques will be critical.
There is a final form of surgical display which does not require a new form of image; rather, it uses a derived type of image that is a rendered display.In such a display, the structures of surgical interest, e.g., organ outlines, tumor margins and vascular structures, are localized in the grey scale tomograms, and their image values are replaced with structure labels.Standard computer graphics routines can then take these mapped labels and display them as interactive structures.Visual clues can be provided by changing the color and opacity of the display.Figure 8 shows a rendered liver with tumors, vascular structures and a preoperatively planned surgical resection plane.Such displays are compelling; they are easy to understand, can be rotated to the surgeon's best viewpoint, and structures can be displayed or not as required by the case.In addition, they not only provide location but orientation information and depth information via shading.However, such displays are dependent on the validity of the For both sensitivity and accuracy concerns, one commercial organization, MEVIS, uses a combination of algorithmic and human segmentations.Once the segmentations are as close as they can be, preoperative analysis such as Residual Liver Volume, Residual Liver Percentage and Residual Functional Volume can be determined.

Correcting Positional Display Errors Due to Perioperative Deformation
In the past, the translation of image-guided surgery techniques to the abdominal environment has been limited due to the presence of perioperative deformation.As a result, the most widely used guidance approaches have been active imaging with the use of US or laparoscopic imaging.The integration of preoperative imaging and planning data for active intraoperative guidance use is not commonplace [45,[55][56][57] and only until recently has begun to be commercialized.As described above, the difficulty that arises when attempting full scale integration is due to the presence of soft-tissue deformation.In recent reports, soft tissue deformation during liver resection has been documented with intraoperative computed tomography (iCT) and has demonstrated significant effects [58].While iMRI and iCT are available, these approaches are cumbersome, incur radiation dose in the latter, and are not economically scalable to Image-Guided Abdominal Surgery and Therapy Delivery mid-level medical centers.The work by Lange et al. [57,59,60] attempted a CT-to-US vessel-based non-rigid registration system for providing the link between image and physical space.While the handful of cases reported performed well, the likelihood of this approach working within the confines of OR workflow is a challenge.In addition, it requires the identification of as many vascular targets as possible with tracked US and then determination of corresponding targets within the CT.While the subsurface information would be valuable for non-rigid deformation correction, there is a significant likelihood of misidentification in highly vascularized organs such as the liver, and the encumbrance of the technique may challenge adoption.
Given the nature of abdominal procedures, the need to compensate for deformation is evident, but as the nature of resection is unlike its neurosurgical counterpart, the requirements for compensation need to be balanced with workflow and accuracy needs.As an example, presentation for open liver surgery (and even laparoscopic to a degree) involves significant organ distortion prior to the ability to resect or even collect geometric data.However, considerable exposure of the organ (as opposed to brain) is afforded for understanding surgical presentation.Additionally, because the liver regenerates, the margins for surgical resection are more liberal, but understanding the local vascular environment during resection is critical.In contrast, partial nephrectomy margins cannot be so liberal and the accuracy requirements are considerably elevated.The exposure can be difficult and the causes of deformation are quite different.In the kidney presentation, the renal artery and vein are clamped to prevent excessive blood loss during resection.This creates a state of turgor within the organ that is different than the preoperative image counterpart.Upon resection, significant drainage from the cortex and medulla regions can ensue and cause significant shape changes.In both of these examples, the surgical characteristics serve as constraints to data acquisition and guidance procedure execution.As the field of image-guidance moves forward, it will be continually evolving.As the presentation, application of surgical technique, and mechanism of therapy evolves for candidate organs in image-guided procedures, the ability to align preoperative information will also have to continue to evolve to solve new challenges in non-rigid registration.
Over the past several years, approaches to deformation correction for abdominal procedures have been achieved that use sparse intraoperative surface data followed by controlled extrapolative predictions based on computer models [61][62][63][64][65].These methods have largely reflected either the acquisition of the intraoperative organ surface using a tracked intraoperative stylus swabbed over the organ surface [41,67,68] or the use of non-contact methods such as LRS [69][70][71].
The basic approaches begin with an initial rigid body registration usually performed using traditional [68] or weighted surface registration methods [44,45].Once achieved, early correction methods focused at calculating a correspondence between surfaces acquired intraoperatively and their CT/MRI segmented counterparts.Once determined, the distances would be used as boundary conditions in a finite element model derived from the segmented organ in image space [44,69].The result could then be used to modify the shape of the CT/MRI organ to match the intraoperative state.While this does provide some improvement, in the regions that immediately flank the intraoperative surface, the deformations often look somewhat distorted.In addition, these early techniques did not have a real strategy for handling boundary conditions in non-visible regions of the organ.In more recent works, efforts to generate extrapolative methods such that more natural flanking deformation fields are produced have been forthcoming [62,63,65].
More specifically, three extrapolative methods recently investigated involved iteratively fitting an average shape model to the intraoperatively deformed organ [62], and two intraoperative model-based computing approaches.The shape-based method was called the iterative closest atlas (ICAt) technique and systematically fit a constructed shape by extracting a weighted combination of pre-computed shapes [62].This method had the advantage of pre-computing the shapes associated with deformation using a finite element model which allowed for rapid registration intraoperatively.While preliminary results were encouraging, the atlas shape models were challenging to generate for surgical data.There are still powerful aspects to this work and investigation is continuing.
To compare and complement that work, the remaining two sparse data solutions involve intraoperative computing of biomechanical models during surgery.Figure 9 is the general form of these methods.In one method [63], the correspondence function between LRS data (shown as green points in Figure 9) is used to guide the application of boundary conditions similar to work by [56,61,69].However, the difficulty in the direct approaches of [56,61,69] was that sufficient information regarding posterior surfaces needed to be specified.These surfaces were difficult to approximate, and 214 Image-Guided Abdominal Surgery and Therapy Delivery  General steps with non-rigid deformation correction from surface data (mesh surface is segmented CT liver surface, and green are points from an LRS liver surface).
flanking regions around the laser-range scanned partial surface were left unmodified leading to unnatural looking deformations (i.e., a plug-like effect).In [63], the general approach was to generate a radial capture region that was initially quite large (large enough to propagate to the posterior region of the liver).This region would serve as an averaging kernel to distribute the closest-point-based boundary conditions.With the large kernels, the averaging would result in a small increment of deformation to be applied.As the kernel size is reduced, the liver shape would approach that of the shape as acquired by LRS.These computations were all compatible with OR timing.It should also be noted that the radial spatial filter was modified by a norm-sensing procedure such that as it distributed boundary conditions to the posterior side of the liver, the boundary conditions would change direction.One advantage of this 'filter' approach is that it introduced sufficient boundary conditions such that a priori assumptions regarding the posterior regions of the liver need not be specified; instead, sufficient enhancement to the condition number of matrices associated with Finite Element Model calculations allowed for rapid solutions with standard sparse matrix techniques.While this method was comparable to ICAt, the results were of limited success.In [63,65], the solution to Laplace's equation along the organ surface was generated to extrapolate boundary conditions into the flanking and posterior regions.Components of this approach have been used to assist in non-rigid surface registration of the breast where fiducials were not present [72].Upon completion of the solution of Laplace's equation, the boundary conditions assigned to the posterior regions also undergo a norm-sensed application direction change.Once computed, a deformation transform can be provided to the guidance system for subsequent correction process.
One important note about these methods is that each method is considerably restricted by the use of sparse data and could be considerably improved with additional data.Conversely, the methods have been thoughtfully designed with respect to surgical workflow and speed of translation.This emphasizes that solutions that do not accommodate workflow and are overly encumbered and requiring too much attention from the surgeon are not easily adopted, if at all.These types of surgically related conflicting design problems were recently classified in [65], "the problem of extrapolating cost-effective relevant information from distinctly finite or sparse data, while balancing the competing goals between workflow and engineering design, and between application and accuracy is the sparse data extrapolation problem".In the above work, computational tools are advocated as a valuable tool to enable more effective therapeutic delivery in light of the sparse data extrapolation problem.As the domains of surgery and engineering continue to require integration, it follows that novel algorithmic developments will entail not only sophisticated mathematics and instrumentation, but equally critical, knowledge of surgical practice.This merging of expertise is an exciting development for the future of surgery.

Clinical Works Accomplished
We have experience with both image-guided liver and kidney surgeries.In both cases, the idea is more specific resections, that is the complete removal of the tumor with minimum damage to the remaining healthy tissue.In both cases, the surgical progression is toward smaller, less radical resections.
In the liver, removal of an entire lobe of the liver was the most common procedure as shown by a study by the Edinburgh Liver Surgery and Transplantation Experimental Research Group and seen in Figure 10 [73].This radical approach was driven by an issue of vascular control.Given the size and the pressure in the arteries in the liver, unplanned cutting of a vessel leads to rapid blood loss which would quickly proceed to exsanguination if not controlled.The vessels which feed each lobe of the liver are available on the posterior side of the liver and thus could be clamped before the lobe is resected.If the disease neither progresses nor recurs then the liver will regenerate the lost volume to resection.However, such approaches are not useful with bilateral disease (a common occurrence in metastatic cancers) and place the patient at risk if the patient has a cirrhotic liver (often a precursor to hepatocellular carcinoma).It would be best if only the liver segments containing the tumor need be removed.But the vascular supply to the segments is buried in the liver parenchyma making control difficult unless they can be localized.
In our liver surgery work, our initial approach has been for open resections.The predominant imaging modality has been CT in which we gather multi-phase contrasted images allowing for the preoperative determination of arteries and veins.The images are segmented for liver surface and vascular structures.In the OR, the patient's abdomen is opened and the anterior surface exposed.We use optical trackers to localize both the surgical tools and LRS for acquiring an intraoperative surface for registration [41,68,216 Image-Guided Abdominal Surgery and Therapy Delivery 71].With that surface, our mean distance between the physical surface and the extracted CT surface was 2.9 mm [68].While such a metric does not fully quantify the quality of the registration, in the absence of true targets, it is the only available quantitation.
One of the noted drawbacks to an ICP driven surface registration is that they are sensitive to rotational symmetries and biology tends to create smoothly curved surfaces.Thus an ICP driven surface registration can "slide" to incorrect location reducing the robustness of the registration methodology.To address this issue, we elected to capture rough designations of surface features (inferior rim and falciform ligament) in both the image set and the LRS images.These "salient features" were weighted with the surface to add information to the registration process [45].In six sets of clinical data, the localization of observed features had a mean error of 24.15 mm with a standard deviation of 23.7 mm and a median of 18.65 mm.When the "salient feature" registration was applied, the mean error dropped to 3.6 mm, the standard deviation to 1.0 mm and the median was 3.6 mm.

Deformation Work
In [65], the comparison among the three algorithms presented above for liver phantom deformation correction indicated a reduction in targeting error of 40-55% over rigid body registration alone.The surface Laplacian method was found to be the most robust and consistently performed better across varying initial registration poses.It was also found that the initial alignment of the liver phantom could vary results 4-10%.In addition, it was found that more complete surface information could reduce target error by as much as 70%. Figure 11 demonstrates the changes in error using rigid registration alone as well as adding correction.The (*) demonstrates the solution when 0-2.5 2.5-3.5 3.5-4.5 5.5-6.5 4.5-5.5 >6.5 Target error (mm) more complete surface data are known.This figure shows that the combined rigidand-correction in both the sparse and more complete data sets results in a clear shifting of targeting errors to smaller values.It is also clear that the more complete data yields better results than the sparse; however, more complete data would require intraoperative imaging either via CT/MRI or optically tracked US imaging.While these are quite possible technologically, the impact to workflow relative to the benefit to surgical application technique is considerable and it is not clear if it is necessary at this time.
With respect to clinical data, a great deal of retrospective application has been achieved.However, quantitative validation in significantly sized patient studies has not been reported.Figure 12 shows an example of a clinical case before and after correction.Figure 12a and b clearly show misaligned surfaces.The guidance display resulting from this usually renders a probing stylus either within the image volume or hovering outside the volume when the surgeon is, in reality, touching the organ surface.While not quantified, this perceptive difference is difficult to rectify for the purposes of navigation.As Figure 12 c, d demonstrate, the surfaces are more closely aligned with the addition of correction which would rectify stylus-position effects within the guidance system and provide better guidance fidelity.
Figure 13 demonstrates the effect on a candidate image-guided display.For this particular example, an optically tracked LRS liver surface was acquired in the OR.The surface was then rigidly registered using a weighted surface registration [45].Once completed, the data were used within the correction framework reflected in [63,65].Once complete, the optically tracked stylus was dragged across the physical liver and a transverse image display was generated without and with the correction.It should be Image-Guided Abdominal Surgery and Therapy Delivery noted that this particular analysis was performed retrospectively.Figure 13a illustrates the image display when no correction is used.The image clearly shows the location of the stylus considerably off the liver surface due to deformation.Figure 13b illustrates the image display when correction is enabled.It clearly demonstrates an improvement.As in the liver, the surgical goal for kidney is towards more complicated but less damaging surgical procedures.Recent studies [74][75][76][77] have demonstrated that a partial nephrectomy, either open or laparoscopic, is an effective procedure for select renal cell carcinoma and is especially applicable for tumors less than 4 cm [75][76][77].In addition to providing equivalent oncologic outcomes, improved patient morbidity and mortality, as compared to complete kidney removal, has been noted.Nephron-sparing procedures are imperative when the contralateral kidney is functionally impaired or has been surgically removed [75,77].
Bringing image-guidance to the kidney poses some specific challenges.Unlike any previous image-guided surgery target, the kidney is covered in perirenal fat, which  hinders access to the surface of the kidney and to localization of specific anatomic surface targets.The fat also results in mechanical coupling to the abdomen and diaphragm, resulting in significant movement with respiration.Since a significant fraction of kidney tumors protrude from surface of the kidney, 'locating' the tumor is often not the challenge.Rather, it is the localization of the internal resection margin of the tumor while maintaining a clear surgical margin that is the surgical challenge.In addition, by interactively showing the surgeon the location of his or her tools during the surgery, image-guidance would minimize both excess nephron removal and unintended damage to vascular structures and the collection system.

Preliminary Animal Studies
We have conducted a number of preliminary studies using a porcine animal model.Swine were chosen as their kidneys closely approximate the size and structure of human kidneys.The first experiment assessed the effect of the loss of perfusion associated with standard kidney vessel clamping and a body force similar to the insufflation pressure in an MIS application.Kidneys were obtained from anesthetized or newly euthanized pigs under an IACUC-approved protocol.Heparin was administered intravenously to prevent blood clotting, and the renal artery and vein were closed to retain turgor before resection.Between 15 and 20 glass beads with 2-mm radii and holes through the center were sutured onto the kidney surface in a roughly even distribution over the entire kidney.CT scans of the kidney (160 or 300 mAs, 90 keV, 0.8 mm slice spacing, Phillips human CT scanner) were taken before and after the renal artery and vein were cut and the kidney decompressed.We can assess the kidney changes due to fluid loss by comparison of the CT scans and by tracked fiducial location.Figure 15 shows a subtraction image from the CT.
We performed similar experiments incorporating the effect of turgor loss due to an incision.Figure 16 displays the incision, while Figure 17 shows the fiducial displacements.Fiducial motion ranges from near zero to 1.1 cm.After applying a correction algorithm, the errors drop dramatically showing a maximum of 0.8 cm at the site of incision but a mean of less than 2 mm.It should be noted at this point that the model is not designed to deal with incisions; therefore, the residual error at the opening is to be expected.

Preliminary Human Studies
We have experience in human studies for kidney guidance.In the human, while the perirenal fat poses an additional challenge, we gain an advantage in that a significant number of human kidney tumors are exophtic, that is they protrude from the kidney.This provides a change in surface curvature ideal for surface registrations.In our preliminary human studies, we obtained LRS surfaces of a kidney during an open procedure.Since we cannot place extrinsic objects in the kidney prior to surgery, we implemented a different approach.Once the kidney was exposed, six dots were placed on the kidney using a surgical marker.We then performed an LRS of the kidney to obtain a color image which can be mapped onto the 3D surface and hence the dots can be localized as virtual fiducials.The kidney was clamped and iced as a standard procedure, and after 10 minutes, a second LRS was obtained.Finally, the surgery proceeded and the tumor was resected before a final LRS was performed.Our results showed that after clamping and icing, the mean TRE for the virtual fiducials was 0.95 mm (max = 1.33mm).After the resection, the surface had a significant resection crater which impeded the surface registration (mean TRE 7.33 mm and 9.53 mm max).However, we know that an accurate registration could still be performed even after the resection due to the use of the virtual fiducials in a point-based registration (see Figure 18).

DISCUSSION
The concept of personalized medicine is one of the driving forces behind advances in medical care.The treatment of the individual patients can be significantly enhanced if it is based on their particular anatomy, physiology or even genetic structure.An example of personalized medicine is the use of modern medical imaging to provide three-dimensional maps of an individual patient's anatomy to differentiate diseased and healthy tissue during a real time surgical intervention.This can be critical in advancing abdominal organ surgery as it has been in neurosurgical interventions.By using radiological data not only for diagnosis but for guidance, we can achieve more complete resections with minimum damage to healthy tissue.
In image-guided abdominal surgery, excellent tomographic image sets can be obtained in combination with either optical or magnetic tracking systems.In addition, modern visualization tools such as the Visualization Tool Kit (Kitware, Clifton Park, NY) make guidance displays sensitive only to surgical preference and the validity of the segmentations.Continued research is needed in physical space to image registration and deformation correction.These are entangled problems requiring careful consideration of tissue properties, algorithm behavior and surgical workflow.Techniques need to be robust and well understood.The worst outcome is when a registration fails in a way which misleads the surgeon.Similarly in deformation correction, the algorithm is effectively modifying the presurgical data.In doing so, the algorithm is assuming some responsibility for patient care.Such techniques require careful consideration, slow implementation and extensive testing.

CONCLUSION
Depending on the study, about 60% to 80% of all liver tumors are considered "unresectable".Far too many kidneys with localized tumors are removed because a radical nephrectomy is technically easier with a reduced chance of short term complications, although it places the patient at a greater long term risk with chronic kidney disease.Neither of these is acceptable.By creating teams of surgeons and engineers, we can develop, test, validate and translate tools to patient care.In this paper, we show registration, localization and deformation-correction tools that are changing existing surgeries and enabling previously impossible surgeries.These tools should have clear advantages in terms of long-term outcome and reduced short term risk for the patient.But the techniques to design such systems require that the surgeons and engineers understand each other's vocabulary, task structure, and mindset.There needs to be almost constant feedback and careful validation because of the nature of the task.This is "mission-critical" engineering and enhanced healthcare.

Figure 3 .
Figure 3.CT image slice with segmentation line.
Figure 5.The daVinci surgical guidance system.

Figure 6 .
Figure 6.An intraoperatively obtained surface (red) fit via ICP to a surface extracted from a preoperative CT scan[38].

Figure 7 .
Figure 7.An example of a 2 over 2 display showing the three cardinal planes.A wireframe display is also shown.

Figure 8 .
Figure 8. Rendered liver in intraoperative display.Arteries are shown in red, veins in blue.The tumors are in brown, the preoperatively-planned surgical plane is yellow and the surgical tool is green (courtesy of Pathfinder Therapeutics Inc., Nashville, TN).

Figure 9 .
Figure 9.General steps with non-rigid deformation correction from surface data (mesh surface is segmented CT liver surface, and green are points from an LRS liver surface).

7 SDFigure 10 .
Figure 10.Residual Liver Volume (RLV) versus case type.Notice the high number of cases (n) in the last two types [55].

Figure 11 .
Figure 11.Target errors in liver phantom experiment subject to rigid registration alone, and rigid registration with correction.The (*) solutions used complete surface description.

Figure 12 .
Figure 12. (a, b) Results of laser-range scanned liver and computed tomographic surface-based rigid registration.(c, d) Results of non-rigid correction.

Figure 13 .
Figure 13.Transverse image of liver with guidance cursor shown (a) without, and (b) with correction.Tracked stylus was on physical surface of liver.

Figure 14 .
Figure 14.(a) Extracted kidney surface from CT scan.(b) Kidney surface showing outline tumor margin (green) and distal point of tumor from the surface.

Figure 15 .
Figure 15.Subtraction of images in turgid and relaxed forms.Large intensity differences indicate the areas of greatest kidney displacement due to loss of kidney perfusion and pressure.

Figure 16 .
Figure 16.Incised kidney.(a) shows the kidney and the fiducial positions (red beads).(b) shows a CT scan showing the depth of the incision.(c) is a rendering of the full CT scan.

Figure 17 .
Figure 17.Change in fiducial locations due to turgor loss.The left image is uncorrected distance and the right image is after a first-order correction.

Figure 18 .
Figure 18.Post resection display showing former tumor location (green), the laserrange scanned surface (red) and the preoperative CT (gray).