Retrospective Validation of New Simulation Software to Prevent Perioperative Hypothermia inMajor Urologic Abdominal Surgery

Background. Perioperative hypothermia is still a major problem. Therefore, it would be useful to predict which warming method would prevent perioperative hypothermia in a particular patient. Methods. The simulation software “ThermoSim Vers. 5.07” was validated using the data of two prospective, randomized, and controlled studies. The diagnostic effectiveness was assessed by calculating sensitivity and specificity, positive predictive value (PPV), and negative predictive value (NPV) in the detection of perioperative hypothermia. Results. Sensitivity of the software was 100% (Study 1) and 94% (Study 2), specificity was 67% (Study 1) and 38% (Study 2), PPV was 60% (Study 1) and 50% (Study 2), and NPV was 100% (Study 1) and 90% (Study 2). Conclusions. The software is helpful in predicting which warming method is necessary to prevent perioperative hypothermia. Thermal treatment based on the prediction of the software will lead to some overtreatment, but will prevent perioperative hypothermia reliably.


Introduction
Perioperative hypothermia is one of the oldest known side effect of general anesthesia [1] and is associated with numerous adverse outcomes such as longer duration of neuromuscular blocking agents, increased intraoperative blood loss with consecutive higher transfusion requirements, shivering, thermal discomfort, longer duration of postanesthetic recovery, morbid cardiac events, surgical wound infections, and a longer duration of hospitalization [2][3][4][5].
The ideal patient warming system would be able to prevent perioperative hypothermia in every clinical situation and should be safe, easy to use, and cheap.However, no currently available system has all of these characteristics.The most powerful systems, such as water warming garments [6] or self-adhesive water conditioned and highly conductive hydrogel pads [7], are much too expensive to be used for every patient, and are therefore only used in special clinical situations like off-pump coronary artery bypass surgery or during liver transplants.In contrast, forced-air warming is a relatively inexpensive, common, and wellaccepted method for preventing hypothermia in surgical patients [8].However, forced-air warming alone is not sufficient to prevent hypothermia in every operative procedure, especially when it is used without prewarming [9] or infusion warming [10].Other methods, such as infusion warming [11], conductive warming of the back [12], insulation [13], or airway heat and moisture exchanging filters [14], have only limited efficacy, but are often used in combination with forced-air warming.It would be extremely helpful to predict exactly which warming method or combination of warming methods would work best to prevent perioperative hypothermia in a patient in order to choose the appropriate and most cost-effective method before starting the case.The present study was conducted to determine the accuracy and diagnostic effectiveness (sensitivity and specificity, positive predictive value, and negative predictive value) of a new computer software designed for the prediction of perioperative hypothermia during major urologic abdominal surgery.

2.1.
Description of the Software.The simulation software "ThermoSim Vers.5.07" was developed by Draeger and programmed in the graphical programming environment LabVIEW 7. The software includes equations for heat gain or heat loss to the environment by convection, radiation, and evaporation [15].The equation for convection includes air temperature and air humidity as well as air velocity between the patient and environment or warming therapy device (e.g., a forced-air warming system).The equation for calculating radiation recognizes the skin temperature of the patient and the temperature of the walls of the room.The equation of conduction includes the heat transfer to the insulated or actively heated mattress.The evaporation rate is described by an empirical equation.
The model uses only two compartments; one represents the core organs (head, heart, liver, kidney, etc.) and the other the periphery (extremities).The simulated core and peripheral temperatures represent the mean temperature of these body's compartments.The blood flow between the central organs is thought to be unobstructed, but vasomotion controls blood flow from the core to the periphery.When the patient's core temperature drops, the vasomotion decreases the blood flow from a "normal" range, of approximately 20 mL kg tissue −1 min −1 down to 1 mL kg tissue −1 min −1 .When the patient becomes hyperthermic, the blood flow can increase to 50 mL kg tissue −1 min −1 .Blood flow and the metabolic rate are controlled by the temperature difference between the actual core and the set temperature.
The metabolic rate is dependent on the gender, age, and weight of the patient.The metabolic rate is decreased during anesthesia.The thermal conductivity of the tissue (bone, fatty tissue, and skin) can be estimated.The insulation of the tissue is dependent on the body mass index, which is normally related to the patient's body-fat percentage.Both influences are included in the simulation software.
Heat loss by respiration is also included in the simulation software, although it does not play a significant role perioperatively.Heat loss or gain by cold or warm infusion, respectively, can have an important influence on the heat balance of the patient.This influence is also included in the simulation software.

Validation of the Software with Validation Study 1.
In the first validation step, we tested the ability of the software to simulate the time course of core temperature change if all relevant data for the simulation software were available.For this purpose, we used the dataset of a previous prospective, randomized, controlled study with 36 patients during major urologic abdominal surgery (open radical cystectomy, open radical prostatectomy, or open radical nephrectomy) [16].This study was approved by the local research ethics committee and written informed consent was obtained from all patients.The study was entitled validation study 1.
The time course of core temperature was simulated using the following data: (i) sex, age, weight, and height of the patient, (ii) room temperature of the ward, the induction room, and the operating room, (iii) duration of the induction of anesthesia, (iv) duration of the operation, (v) amount of infused fluids and packed red cells (PRCs), (vi) core temperature before induction of anesthesia, (vii) thermal management during the procedure (infusion warming (Hotline HL-90 with System L-70, Level 1 Technologies Inc., Marshfield, USA), forcedair warming (WarmTouch, Mallinckrodt Medical, Hennef/Sieg, Germany, with upper body blanket), and insulation (Thermadrape, O.R. Concepts Inc., Roanoke, USA), either alone or in combination).
The resulting core temperatures at the beginning of surgery and two hours after the beginning of surgery were compared with the core temperatures measured at the tympanic membrane.The time course of core temperature was simulated using the following data:

Valdiation of the
(i) sex, age, weight, and height of the patient, (ii) estimated duration of the operation.
Thermal management during the procedure (forced-air warming (WarmTouch, Mallinckrodt Medical, Hennef/Sieg, Germany with upper body blanket or Bair Hugger 505, Augustine Medical, Eden Prairie, Minn, USA, with upper body blanket), insulation (Cotton surgical drape), or electric heating mattress (ASTOPAD OPT 120, Stihler Electronic GmbH, Stuttgart, Germany), used either alone or in combination) Data recording core temperature before the induction of anesthesia, duration of induction of anesthesia, the room temperature in the operating room, and the amount of infused fluids and PRC, were not available, so they were estimated.As an estimate for core temperature before induction of anesthesia, we used 36.4• C because this was the median value given in a large study performed by Mitchell and Kennedy [17].For the duration of induction of anesthesia, we used the mean value of the first study (78 min).The operating room temperature was estimated to be 21 • C and the amount of infused fluids was estimated using the mean value from the first study (5277 mL).
The calculated core temperatures at the beginning of surgery and two hours after the start of surgery were compared with the original core temperatures measured in the distal esophagus.

Valdiation of the Software against the Estimates of Anesthetists.
In a third validation step, we tested the ability of the software to predict perioperative hypothermia against the ability of anesthetists to predict perioperative hypothermia using the data of validation study 2.
Twelve residents in anesthesia with experience of less than one year and 12 anesthetic specialists with more than six years of experience received a questionnaire with the following data: sex, age, weight, and height of the patients, the planned operation, and the thermal management (insulation, heating mattress, forced-air warming).Given these data, the anesthetists had to estimate whether the patient would be hypothermic with a core temperature below 36 • C after 2 hours of surgery.

Statistical Analysis.
Analysis of agreement between the core temperatures calculated with the software and measured core temperatures in validation studies 1 and 2 was performed as suggested by Bland and Altman [18] using the mean difference between the methods and the limits of agreement (≈ two standard deviations around the mean difference).
Diagnostic effectiveness of the software was assessed by calculating the sensitivity and specificity, positive predictive value, and negative predictive value in the detection of perioperative hypothermia, defined as a core temperature below 36 • C two hours after the start of surgery (see the appendix).
Diagnostic effectiveness of the residents and specialists in anesthesia was assessed by the mean sensitivity and specificity, positive predictive value, and negative predictive value for the prediction of perioperative hypothermia.

Results
Patient characteristics and thermal management of the two previous conducted prospective, randomized, controlled studies are shown in Table 1.
The core temperature in validation study 1 ranged between 34.3 • C and 36.9 • C. The mean bias between the measured core temperatures at the beginning of surgery and the calculated core temperatures was -0.06 • C; the limits of agreement were from −1.20 • C to 1.08 • C. Two hours after the start of surgery, the mean bias between the measured core temperatures and the calculated core temperatures was -0.49 • C; the limits of agreement were −1.78 • C to 0.8 • C. The sensitivity of the software in detecting perioperative hypothermia two hours after the start of surgery was 100%, the specificity was 67%, the positive predictive value was 60%, and the negative predictive value was 100%.
The core temperature in the validation study 2 ranged between 34.6 • C and 37.9 • C. The mean bias between the measured core temperatures at the beginning of surgery and the calculated core temperatures was -0.38 • C; the limits of agreement were −1.54 • C to 0.78 • C. Two hours after the start of surgery, the mean bias between the measured core temperatures and the calculated core temperatures was -0.77 • C and the limits of agreement were −2.30 • C to 0.76 • C. The sensitivity of the software in detecting perioperative hypothermia two hours after the start of surgery was 94%, the specificity was 38%, the positive predictive value was 50%, and the negative predictive value was 90%.
Of the questionnaires sent out to 12 residents a total of 8 could be analyzed (67%) and of the questionnaires sent out to 12 specialists, all 12 could be analyzed (100%).The mean

Discussion
The new simulation software "ThermoSim Vers.5.07" overestimates the drop in core temperature in patients undergoing major urologic abdominal surgery.When all relevant data are known, as in validation study 1, the software very accurately predicts the drop in core temperature during induction of anesthesia, with a mean bias of -0.06 • C.However, two hours after start of surgery the mean bias was -0.49 • C, indicating that the prediction of core temperature gets less accurate with a longer duration of simulation.This is not surprising because small differences between the measured data and the simulation will add up during time, thereby decreasing accuracy.The accuracy of the simulation software is better when more exact data like the peripheral temperature are entered into the software.Kellner et al. [19] reported a mean bias of 0.03 • C and limits of agreement of ±0.64 • C in patients treated with warm blankets and a mean bias of 0.04 • C and limits of agreement of ±0.48 • C in patients treated with forced-air warming during short procedures.
If the simulation is started with the reduced set of data that is available from the day before surgery (sex, age, weight, and height of the patient) and the planned thermal management during the procedure, the accuracy of the software is lower.The mean bias of the core temperature at the beginning of surgery was acceptable at -0.38 • C and increased to -0.77 • C two hours after beginning of surgery, thus demonstrating again that the prediction of core temperature gets less accurate with a longer simulation.This effect could also be observed with increasing limits of agreement during the time course.
The sensitivity of the software in detecting perioperative hypothermia two hours after the beginning of surgery was very high.This means that 100% (validation study 1) or 94% (validation study 2) of the patients that were predicted to become hypothermic developed perioperative hypothermia and only 6% of the patients in validation study 2 were missed by this test because they had a false-negative result (normothermia).
However, this high sensitivity was accompanied by low specificity in both validation studies (67% in validation study 1 and 38% in validation study 2).This means that only 67% (validation study 1) or 38% (validation study 2) of the patients without perioperative hypothermia were predicted to maintain normothermia, whereas 33% (validation study 1) or 62% (validation study 2) of the normothermic patients were predicted to get hypothermic.
The positive predictive value was also low.This means that only 60% of the patients in validation study 1 and 50% of the patients in validation study 2 with predicted perioperative hypothermia actually developed hypothermia, whereas 40% (validation study 1) and 50% (validation study 2) of the patients maintained normothermia despite the prediction of perioperative hypothermia.
The negative predictive value was very high: 100% (validation study 1) or 90% (validation study 2) of the patients that were predicted to maintain normothermia were normothermic in reality.Only 10% of the patients in validation study 2 developed perioperative hypothermia although the prediction was that these patients would maintain normothermia.
To assess these values, we compared the results of the software with the results of a questionnaire with the data of the patients in validation study 2. The results demonstrated that the estimates of anaesthetists showed a much lower sensitivity and negative predictive value but a higher specificity and positive predictive value.As expected, the specialists with more than six years of experience had slightly better values than the residents, which had less than one year of experience.
The question, then, is if the software is helpful in predicting which warming method, or combination of warming methods, is necessary to prevent perioperative hypothermia in patients during major abdominal surgery.To enable reliable prevention of perioperative hypothermia the software has to have high sensitivity and high negative, predictive value so that a sufficient thermal management can be chosen.Both values (sensitivity and negative predictive value) of the software are very high (≥90%) and therefore the software fulfills these demands.Will this be always cost effective?Therefore, the software has to have high specificity and high positive predictive value.This is not the case and therefore choosing the thermal management method for patients undergoing major abdominal urologic surgery with help on the actual version of the software will lead to some overtreatment.However, this is rational and it can be assumed that this is cheaper than underestimating the risk of hypothermia because the development of perioperative hypothermia is associated with high costs [2,20,21].
In contrast to the software, the anaesthetists underestimated the risk of perioperative hypothermia in some patients and a thermal management based on the estimates of anaesthetists would be associated with more perioperative hypothermia.
Several limitations of our study should be mentioned.First, there were six patients in validation study 1 who showed an increase in tympanic membrane temperature during induction of anesthesia.This is very unlikely, so that we have to assume measurement errors of core temperature in these six patients.If these six patients are excluded, the mean bias between the measured core temperatures at the beginning of surgery and the calculated core temperatures is 0.14 • C and limits of agreement are −0.47 • C to 0.75 • C. Two hours after the beginning of surgery, the mean bias between the measured core temperatures and the calculated core temperatures is -0.34 • C, and the limits of agreement are from −1.48 • C to 0.8 • C.These results are remarkably better and show that the software can simulate the changes in temperature quite well.However, in validation study two, esophageal temperature measurements were used and it has to be assumed that these results are valid.This underscores the impact of the preanesthetic core temperature to simulate the time course core temperature during anesthesia and surgery.
A second limitation of the study is that we had to assume that perioperative fluid therapy was evenly distributed.This is an oversimplification and it is possible that it has negatively influenced the results.
A third possible limitation is that we estimated the evaporative heat loss via the peritoneum to be 5 W. It is possible that evaporative heat loss via the peritoneum are higher or even lower, however, data about evaporative heat losses via the peritoneum are sparse [22,23].
Forth, the two validation studies were both performed during open abdominal surgery.Therefore, the results of this study cannot be generalized to other patient populations with different types of surgery.
Fifth, the residents and specialists in anesthesia that received the questionnaire were trained in a department that is dealing with perioperative hypothermia for more than fifteen years.This may have increased the awareness for perioperative hypothermia and may have positively influenced the results.
A last limitation of every software program based on physiological equations is the fact that the results will fit the mean of the clinical data much better than the data of an individual patient.On the other hand, it has to be respected that even invasive measurements of the core temperature like bladder temperature can result in a bias of −0.46 • C and limits of agreement up to ±0.9 • C [24] when compared to esophageal temperature by the Bland and Altman method.It cannot be expected that simulation software could have better precision for the prediction of core temperature than different invasive core temperature methods.
Several thermodynamic models have been developed in the last several years.However, these models are very specific, for example, for procedures with pneumoperitoneum [25], for trauma patients [26,27], for damage control surgery in trauma patients [28], or during cardiac surgery with cardiopulmonary bypass [29].In principle, the more specific such a thermodynamic model is and the more detailed variables are included in the algorithms, the better the precision.However, this increases the complexity of the handling of the software and decreases the usability.Further work on the Thermosim software is desirable and should decrease the complexity of the handling to make it easier to use.
The usefulness of the software could be assessed in a prospective study comparing a clinical protocol versus the thermal management that was planned with the software to prevent hypothermia.The software could also be used as a test tool for quickly evaluating the influences of different thermal management strategies (e.g., change in room temperature, adding a fluid warmer to the forced-air warming system) on perioperative core temperature.This is also a very interesting possibility for the further education and training of residents.

Conclusions
Thus, it can be concluded that the software is helpful in predicting which warming method or combination of warming methods is necessary to prevent perioperative hypothermia in patients during major abdominal surgery, because it has a high sensitivity and a high negative predictive value.Thermal treatment of these patients based on the prediction of the software will lead to some overtreatment, but will ensure perioperative normothermia in almost every patient.In order to validate the clinical usefulness of the software, a prospective study including multiple type of surgery is necessary.

Appendix
Diagnostic tests are used to detect diseases.In this study, a positive test result (the software predicts perioperative hypothermia) indicates a prediction of perioperative hypothermia, as defined by a core temperature below 36 • C (Table 3).
(1) Sensitivity (=a/a + c) describes the proportion of patients with perioperative hypothermia that have a positive test.For example, a sensitivity of 90% indicates that 90% of patients with perioperative hypothermia will have a positive test and that 10% are missed by this test because they have a false negative result.(2) Specificity (=d/b + d) describes the proportion of patients without perioperative hypothermia that have a negative test (the software predicts normothermia).For example, a specificity of 60% indicates that 60% of patients without perioperative hypothermia will have a negative test and that 40% are missed by this test because they have a false positive result.(3) Positive predictive value (=a/a + b) describes the proportion of patients with a positive test that have perioperative hypothermia.For example, a positive predictive value of 60% indicates that 60% of patients with a positive test will have perioperative hypothermia, but 40% will not.(4) Negative predictive value (=d/c + d) describes the proportion of patients with a negative test without perioperative hypothermia.For example, a negative predictive value of 95% indicates that 95% of patients with a negative test will maintain perioperative normothermia, but 5% will not.
Software with Validation Study 2. In the second validation step, we tested the ability of the software to simulate the time course of core temperature only with data that were available from the day prior to surgery.
For this purpose, we used the data of another previous prospective, randomized, controlled study with 40 patients during major urologic abdominal surgery (open radical cystectomy, open radical prostatectomy, or open radical nephrectomy) (data only published in an abstract (Weyland W, Fritz U, Landmann H, Schuhmacher I, English M, Kettler D: Effectivity of intraoperative warming techniques [abstract].Can J Anaesth.1996, 43:A21)).This study was approved by the local research ethics committee and written informed consents was obtained from all patients.The study was entitled validation study 2.

Table 1 :
Patient characteristics and thermal management.Data are presented as mean ± SD or number of patients (%).

Table 2 :
Mean sensitivity and specificity, positive predictive value, and negative predictive value for the prediction of perioperative hypothermia of 8 residents in anesthesia with an experience of less than one year and 12 specialists in anesthesia with an experience of more than six years.

Table 3 :
Calculation of predictive indices for perioperative hypothermia.