Tool Sequence Trends in Minimally Invasive Surgery: Statistical Analysis and Implications for Predictive Control of Multifunction Instruments

This paper presents an analysis of 67 minimally invasive surgical procedures covering 11 different procedure types to determine patterns of tool use. A new graph-theoretic approach was taken to organize and analyze the data. Through grouping surgeries by type, trends of common tool changes were identified. Using the concept of signal/noise ratio, these trends were found to be statistically strong. The tool-use trends were used to generate tool placement patterns for modular (multi-tool, cartridge-type) surgical tool systems, and the same 67 surgeries were numerically simulated to determine the optimality of these tool arrangements. The results indicate that aggregated tool-use data (by procedure type) can be employed to predict tool-use sequences with good accuracy, and also indicate the potential for artificial intelligence as a means of preoperative and/or intraoperative planning. Furthermore, this suggests that the use of multifunction surgical tools can be optimized to streamline surgical workflow.


INTRODUCTION
Minimally invasive surgical (MIS) techniques are favored for the associated decrease in patient trauma and resulting improvement in outcomes. To some extent, however, MIS techniques carry with them increased time requirements. This has prompted the design of special tools to facilitate MIS procedures (e.g., [1][2][3]). The process of changing tools has been identified as a key contributor to slowing the surgical workflow, due to the differences in tool removal and insertion vs. open surgery [4][5][6]. This has led to the design of modular, multifunction tools for MIS [7][8][9][10][11]. In order for these tools to be used successfully for streamlining surgical workflow, an understanding is needed regarding ways in which different tools are used together, in particular the sequences or patterns of tool use.
Although numerous researchers have studied surgical or medical workflow (e.g., [12][13][14]), few have studied surgical tool use in particular. Mehta et al. [9] used time motion analysis and state transition diagrams to identify elements of multifunctionality in common laparoscopic tools and to document the sequences of tool use in 29 MIS procedures. Based on the recorded tool-use sequences, they were able to identify several common tool interchanges in each of 4 procedure types. Although many other studies have used time motion analysis, the focus on sequences in [9] was among the most important. Bouarfa et al. [15] observed the significant effect of uncertainty on surgical procedures and presented methods for training Hidden Markov Models to represent surgical workflow. Although they focused on general surgical processes rather than the particulars of surgical tool use, Neumuth et al. [16] established the concept of generic surgical process models constructed using the "mean" of a set of aggregated surgical procedure models. Rosen et al. [17] and Padoy et al. [18] both used variations of the Hidden Markov Models approach to represent surgical tool motions, although they focused on the temporal kinematic events rather than the discrete tool-interchange events that we are concerned with here.
In this paper, we present a new method for analyzing tool-use trends in surgical procedures, and we study tool-sequence trends in MIS procedures and investigate the relative importance of case-by-case statistical variation in these trends. In so doing, we improve upon the work of [9] while taking into account the presence of uncertainty recognized in [15]. An aggregation technique which takes into account multiple procedures in the same way as [16] is used, but the treatment of uncertainty using a novel approach adds to the uniqueness of the work. We also present a method of validating the effects of these variations on presurgical planning with multifunctional instrument systems, and show the potential for real-time intraoperative computer-assisted tool selection as an aid in streamlining surgical procedures. As suggested in [7,8], such computer-assisted, predictive tool selection could be of benefit to individuals whose role is to assist surgeons in the operating room, help to decrease operating time, and thereby decrease risk and/or improve patient outcomes. Furthermore, the method could be applied perhaps even more effectively in robot-assisted surgery.

Matrix Method of Data Representation
Videos of 67 laparoscopic procedures were analyzed to identify patterns in tool use sequences, as recorded in Table 1. These videos were taken from a publicly accessible database [19], and included a variety of common MIS cases with varying numbers of surgeons per procedure type. Since this was a retrospective study using available data, patient selection and surgeon inclusion were not directly considered. Data were manually coded by watching the videos and noting time-stamps for each tool-use event, and graph theory was employed to model the data collected. As shown in Figure 1, the individual tools used in a given procedure were represented as vertices in the graph. The lines connecting those vertices, called edges, carried numerical values that represented the quantity of tool changes observed between the respective tools. The numeric data captured in each graph are easily represented using the graph's adjacency matrix (A). In this matrix, the rows and columns represent the set of surgical tools, and the entries are the numerical values attributed to each edge, called edge weights, in the graph, or the tool-change quantities. In other words, entry a pq in matrix A is the number of tool changes between tools p and q. In this study, we do not distinguish between changes Graph representing a single adrenalectomy procedure, and its associated adjacency matrix. Tools: 1 = cautery shears, 2 = cautery hook, 3 = slender dissector, 4 = stapler, 5 = blunt dissector, 6 = cautery dissector. Edge weights in parentheses represent tool changes from higher to lower vertices (a higher number tool is exchanged for a lower number tool), and vice versa; entries in the adjacency matrix are the sum of weights on a given edge since directionality of tool changes is not important.
from p to q and from q to p, so A is symmetric and a triangular form could be used; however, we keep the full matrix form here for the sake of generality in case a directed graph representation retaining this additional information were preferred. The complete tool list for each procedure type was compiled, and an arbitrary standard numbering for these tools was established. This provided a framework for recording all the tool-use data into a single adjacency matrix A i0 for each procedure type i. This A i0 matrix contained the mean adjacency matrix values over the entire set of procedures for the respective procedure type (averaged over the number of those procedures in which each particular tool was used): (1) where n is the number of procedures of type i. Likewise, aggregate standard deviation matrices S i were also calculated for each procedure type: (2) where the (.*) operator indicates element-wise multiplication as opposed to matrix multiplication, and the square root is also performed element-wise on the matrix. Using the concept of signal-to-noise ratio (SNR), the statistical strength of the tool-use trends indicated in the adjacency matrices was calculated as where the (./) operator indicates element-by-element division as opposed to matrix division. By taking into account both the mean of the matrix values and their spread across the data sets, this SNR metric represents a "confidence" in the indicated tool interchanges occurring during surgical procedures. The matrices A i0 and S i give more weight to longer surgical procedures (those with more tool changes) since the entries in their respective A ij matrices are larger in general. Although this seems a reasonable approach, it also makes sense to consider each individual procedure as having the same weight. Therefore, a set of normalized matrices was also calculated: Tool Sequence Trends in Minimally Invasive Surgery: Statistical Analysis and Implications for Predictive Control of Multifunction Instruments where all − A ij are normalized (scaled) such that their entries sum to 2 (or the entries in either the upper or lower triangular part sum to 1). SNR can then be calculated as in eqn. 3, but using the normalized quantities from eqns. 4-5. To investigate the effects of these two different methods of computing signal to noise ratios, a weighting function was employed to generate a spectrum of SNR matrices: (6) with the weights w 1 and w 2 taking on values as given in Table 2. This is simply a linear weighting which favors the normalized and non-normalized SNR expressions of eqns. 3 and 4-5, respectively, at its two extremes, to enable observance of any dependency on procedure length within the statistical outcomes.

Optimal Tool Sequences
To simulate a given surgical procedure, the procedure can be represented as a sequence of tool uses, with tool interchanges between sequentially used tools. If tools are arranged in a rotating cartridge, the "distance" between tools and the "cost" of the tool interchange can be thought of as the number of slots in the cartridge separating the respective tools. The overall tool-change cost in a given surgical procedure is the sum of individual tool-change costs incurred while traversing its tool-use sequence. Each of the 67 analyzed surgical procedures was thus simulated using MATLAB software, assuming a modular cartridgetype tool as in [7], using 7 different tool-cartridge arrangements with arrangement 1 based on raw data from the adjacency matrix for that procedure only (the respective A ij ), arrangement 2 based on aggregated raw data from all adjacency matrices of the respective procedure type (based on A i0 ), and the remaining 5 arrangements from cases 1-5 from Table 2, respectively, applied to all adjacency matrices of the particular procedure type.
The tool-cartridge arrangements were generated using an optimization technique presented in [20], in which the tool-arrangement problem is treated as the classical "traveling salesman" optimization problem (TSP) in graph theory. In this approach, the tool placement (ordering) in the cartridge is determined by the path through the toolchange graph which visits each vertex once and only once, and traverses the maximal sum of edge weights. (Edges not shown on the graph do exist but have zero weight.) This favors placing pairs of tools with frequent interchanges next to each other in the cartridge. The question being addressed is then which of the possible tool-use representations to input in this algorithm. The 7 optimized arrangements, as determined from either single-procedure, aggregated, or weighted data (using Table 2 and eqn. 6), Journal of Healthcare Engineering · Vol. 3 · No. 2 · 2012 265 are a subset of the unique tool-cartridge arrangements that exist for an m-bay modular cartridge, where m is the number of tools used in a given procedure [20]. Each surgery was also simulated under all the remaining possible tool arrangements, and the outcomes of the 7 arrangements of interest were ranked against all possible arrangements by the amount of cartridge indexing required, with the goal being to minimize the amount of cartridge indexing (representing time and energy wasted during surgery). The number of possible tool-cartridge arrangements, z, for a procedure using m tools is the number of circular permutations without consideration of direction; i.e., 3. RESULTS

Adrenalectomy
The general trends in tool interchange for the 12 adrenalectomy procedures are shown in Figure 2. In this and subsequent figures (4-13), the SNR data are presented as mean divided by normalized standard deviation ("Mean/NSD"), i.e., using eqn. 6 with w 1 = 0 and w 2 = 1. The most important (highest) values are shaded for emphasis, with darker shading indicating higher values. It can be noted that the transition from cautery shears to cautery hook is one of the top three interchanges regardless of whether one uses the mean A i0 or the signal to noise ratio SNR as the metric. The data in Figure 2 were used to simulate the process of performing the respective surgical procedures using a multifunction surgical instrument, based on a set of near-optimal tool cartridge arrangements as calculated in [20]. In these simulations, each tool arrangement is determined using the TSP approach previously described, and the number of tool cartridge indexing steps is calculated based on the tool-use sequence for the respective surgical procedure. This quantity is desired to be  Tool (matrix column/row) labeling, matrix of means A i0 , and signal to noise ratio matrix SNR for adrenalectomy (n = 12 minimized. For all possible tool arrangements (permutations as in eqn. 7), the proportion which outperform the optimized arrangement (involving fewer overall indexing steps during tool changes) is indicated in green (top of each bar), the proportion having equal performance is shown in red (middle section of each bar), and those outperformed by the optimized arrangement are indicated in blue (bottom section of each bar) in Figure 3. Thus a perfect or ideal tool arrangement would be indicated by an all-blue plot in Figure 3, meaning that there are no possible tool arrangements having fewer indexing steps for the surgical procedure. These results are aggregated over the set of 12 adrenalectomies, first giving equal weight to each procedure ( Figure 3a) and then scaling the contribution of each procedure by the size of the set of possible cartridge arrangements (Figure 3b, to account for procedures using fewer tools having smaller sets of possible solutions and therefore a higher likelihood of achieving good outcomes). Each bar in the figure represents the performance of a tool arrangement determined based on a matrix of data as in Figure  1, either raw tool-sequence data from individual surgical procedures, aggregated (summed) data over multiple procedures of the same type, or SNR' values weighted using eqn. 6 and Table 2.  Figure 3.
Quality of tool arrangements calculated as in [20] for adrenalectomy (n = 12) based on various combinations of raw (a) and normalized (b) SNR (overall results summed across all procedures; overall results summed using scaled procedure weights based on size of solution set). The span of the y-axis represents comparison against the total set of possible tool arrangements.
In Figure 3, the outcome in the first bar (labeled "Raw Data") is expected to be the best, since the actual tool sequences of the procedures (not aggregated data from multiple procedures) are used to optimize the tool arrangement. This is equivalent to knowing the exact tool-use sequence ahead of time and basing the tool arrangement on that prior knowledge. As can be seen in Figure 3, using the SNR data to determine tool arrangements for optimizing tool interchanges results in markedly better performance than using the aggregated data alone (A i0 , second bar from the left). In fact, the performance of this technique nearly approaches a priori knowledge of the casespecific tool sequences (leftmost bar). (It should be noted that the leftmost bar represents a set of n distinct tool arrangements, each optimized for a specific surgical procedure or tool-use sequence, and all the other bars represent a single "universal" tool arrangement.)

Cholecystectomy
The general trends in tool interchange for the 5 cholecystectomy procedures are shown in Figure 4. The most important values are shaded for emphasis (5 different entries in the mean data due to a 3-way tie). It can be noted that two of the top three interchanges are common across the two metrics (mean A i0 or SNR).
Using simulation similar to that in Figure 3 for cholecystectomies and all subsequent procedure types, some interesting observations can be made. For the cholecystectomy simulations, the tool arrangement based on the aggregated data actually gave a better result than using the raw case-specific tool-use data. This is a result of the optimization algorithm giving only near-optimal results as a tradeoff for computational speed. Despite this phenomenon, it was observed that the tool arrangements based on weighting the normalized SNR' at least 50% (cases 3-5 of Table 2) produced very competitive results compared to either the raw case-specific data or the aggregated data.

268
Tool Sequence Trends in Minimally Invasive Surgery: Statistical Analysis and Implications for Predictive Control of Multifunction Instruments

Colectomy
The general trends in tool interchange for the 4 colectomy procedures are shown in Figure 5. The most important values are shaded for emphasis. It can be noted that two of the top three interchanges are similar regardless of whether one uses the mean A i0 or SNR as the metric. Simulation showed that all the cases using SNR resulted in better tool arrangements than simply using the aggregated tool-change data. This was a rare situation (2 procedure types out of the 11 considered) in which the normalized SNR' approach clearly gave worse results in terms of optimizing tool-change time than the nonnormalized SNR. For the two cases with highest weight values on the non-normalized SNR, the performance far exceeded that of using the raw aggregated data, coming very close to the optimal case of using the case-specific raw tool-change data directly.

Gastric Bypass
The general trends in tool interchange for the 9 gastric bypass procedures are shown in Figure 6. The most important values are shaded for emphasis. The transition between cautery shears and cautery dissector appears in the top three interchanges independent of the choice of metric. In this procedure type, there appear to be many different tool interchanges used (the matrices are more populous, with fewer zeros). Because of this, there are fewer clear "winners" in terms of strength of trend across the two metrics; there seem to be more tool-change trends that have appreciable strength relative to each other.
Another way of describing the above phenomenon was better observed through simulation. Because of the variations from case to case, which are reflected in Figure 6, there does not appear to be any universally excellent tool arrangement. Although most of the SNR-based results are about as good as or better than using aggregated data, none has superior performance like the case-specific optimal arrangements.   Tool (matrix column/row) labeling, matrix of means A i0 , and signal to noise ratio matrix SNR for colectomy (n = 4). The top tool pairings are {(cautery shears, blunt dissector); (cautery shears, stapler); (cautery dissector, cautery shears)} for the mean and {(cautery shears, blunt dissector); (cautery shears, irrigator); (cautery shears, cautery dissector)} for SNR. Total tool changes per procedure: µ = 13.8, σ = 5.9.

Splenectomy
The general trends in tool interchange for the 4 splenectomy procedures are shown in Figure 7. The most important values are shaded for emphasis. It can be noted that two of the top three interchanges are common across the two metrics.

Appendectomy
The general trends in tool interchange for the 5 appendectomy procedures are shown in Figure 8. As in the adrenalectomy and colectomy data, it can be noted that one transition (the change from slender dissector to irrigator in this case) appears in the top three interchanges regardless of whether one uses the mean A i0 or SNR as the metric.

Appendicular Peritonitis
The tool interchange data for the 2 appendicular peritonitis procedures are shown in Figure 9. Note that one of the two most important tool transitions is common independent of the choice of metric, even with such a small sample size.  Cautery hook 5 2 Figure 9.

Sigmoidectomy
The general trends in tool interchange for the 5 sigmoidectomy procedures are shown in Figure 10. Again, one transition (the change between cautery shears and curved dissector) appears in the top three interchanges independent of the choice of metric.

Nephrectomy
The general trends in tool interchange for the 5 nephrectomy procedures are shown in Figure 11. In this case, the top two tool transitions are common and are even similarly scaled regardless of whether one uses the mean A i0 or SNR as the metric.

Genitourinary Prolapse
The general trends in tool interchange for the 5 genitourinary prolapse procedures are shown in Figure 12. Again, one of the top two tool interchanges is common to both metrics.

Nissen Fundoplication
The general trends in tool interchange for the 11 Nissen fundoplication procedures are shown in Figure 13. In this case, all the tool transitions indicated in Figure 13 may be important, both those highlighted using the A i0 metric and using the SNR metric. SNR values were not calculated at matrix entries for which the standard deviation was zero (the result would be undefined). This is the case for the 2 nd and 3 rd ranked tool  interchanges as measured using the A i0 metric. Therefore, all 5 of the indicated tool interchanges are significant. However, the top-ranked interchange (between cautery shears and curved dissector) is common to both metrics and should be considered the most important. (This issue with undefined SNR values occurs occasionally in some other procedure types, but not usually for entries having high mean significance; therefore, we limit mention of this to the Nissen procedure.)

Summary of Results
Using the SNR matrix data, common tool interchanges were identified for each procedure type analyzed. Using the sum of each row in the matrices as a base value, "likelihood percentages" were calculated similar to the technique used in [9], reflecting the ratio of use of the specific tool pair relative to all use of the first tool in the pair. In other words, the likelihood that the next tool is Y given that the current tool is X can be expressed as the ratio of entry (X,Y) to the sum of entries in row X. This illustrates the predictive power of the data analysis technique used in this study. (The likelihood percentage is tool-order sensitive due to the way it is calculated, as opposed to the general tool-pair trends identified above, which are not order-sensitive.) A select set of these interchanges having both high occurrence and low variance (i.e., high SNR values) are shown in Table 3, with comparison to their likelihood values using only the mean occurrence data. Only 3 common procedure types are shown here for clarity. One can notice that many of the predictions using SNR data are significantly stronger than those arrived at using just the mean tool-use data, with only a few true exceptions where the role is reversed. This reflects the more statistically robust nature of the SNR data,  since it takes into account the variance of the data from the mean. This also suggests that the indicated trends in tool sequence are actual and can be used for decision-making in surgical workflow.

DISCUSSION
When compared against the earlier results in [9] regarding tool interchange patterns, the results in Table 3 are surprisingly different. Even after attempting to match the different tool terminologies used, it was clear that different tool sets were observed in this study compared to the previous work. This may reflect an evolution of technique in MIS procedures during the time interval separating the two studies, changes in tool technology, effects of small sample size per procedure type, and/or significant effects of surgeon preference. One limitation of this study is the small number of surgeons represented per procedure type, which could have an effect of narrowing the variances of the trends. In particular, the available data only included a single surgeon for the appendicular peritonitis and Nissen fundoplication procedure types, with only two procedures in the appendicular peritonitis category. The small number of surgeons would make the trends appear stronger than they generally are across multiple surgeons, and an expanded study might shed additional light on whether some or all trends remain strong across a broader set of surgeons. Nevertheless, surgeon-specific tool-use trends are also of interest. Despite the differences between the results of this study and previous work, the data appear to have strong trends enabling some level of automatic prediction of tool-use sequencing in real time using computational intelligence. Notably, many of the most important tool interchanges remain important, whether measured using mean data across a set of procedures or the SNR metric. However, each procedure type tends to have its own set of important tool interchanges which are more predictable than others; this is logical based on the fundamental differences in the associated sub-tasks of the various surgery types. The new metric serves to accentuate the value of statistical predictability, or confidence in the prediction of tool order. As seen in Figure 3, one outcome of this is that modular, multifunctional tools can be used to achieve highly efficient sequencing of tool tips (optimal tool arrangements) for streamlining surgical workflow specific to tool interchanges; since a small number of strongest trends often represent a significant portion of the total tool interchanges, identification of just these strong trends leads to the majority of time savings achievable. Furthermore, the identification of high-use tool pairs is important for guiding design of new multifunctional instruments [21]. These outcomes could ultimately lead to reductions in operative time, with corresponding decreased patient risk and improved surgical results.
This technique can also be used to reduce surgeons' cognitive load by not only queuing up anticipated tools but providing suggested options from which the surgeon can select, or which the surgical assistant or robotic surgical system should have at the ready. Based on data presented in Figures 2-13 and Table 3, an automated system using this type of analysis could suggest a ranked list of 2 "next tool" options which would include the surgeon's choice of next tool roughly 80% of the time based on either SNR or mean tool-use data. In addition to presenting tool options to less experienced surgeons and/or surgical assistants, the added benefit of time savings in tool interchange as discussed previously is also inherent in this possible approach. Tool presentation using this technique can be equally applied to either manual or robot-assisted procedures, with perhaps somewhat more time savings realizable in robot-assisted surgical settings having automated tool-change capabilities [22]. This is an important step forward in improving technology for computer-assisted surgery.

CONCLUSIONS
In this paper, we have identified common surgical tool interchange pairings through analysis of 11 different procedure types in minimally invasive surgery. In addition to these findings, we introduced a new statistically based method for determining the strength of such tool-use trends and showed that tool-use data can be aggregated by procedure type for accurate prediction of tool-use sequences. This suggests that various artificial intelligence implementations leveraging data from previous surgeries could be used for preoperative and/or real-time adaptive intraoperative planning to streamline the procedures, increasing throughput while decreasing operative risk to patients. The findings also suggest that the use of multifunction surgical tools can be optimized to streamline surgical workflow by integration of common tool pairings in a single tool.