Performance Assessment of the Mortality in Emergency Department Sepsis Score, Modified Early Warning Score, Rapid Emergency Medicine Score, and Rapid Acute Physiology Score in Predicting Survival Outcomes of Adult Renal Abscess Patients in the Emergency Department

Background Renal abscess is a relatively uncommon yet debilitating and potentially fatal disease. There is no clearly defined, objective risk stratification tool available for emergency physicians' and surgeons' use in the emergency department (ED) to quickly determine the appropriate management strategy for these patients, despite early intervention having a beneficial impact on survival outcomes. Objective This case control study evaluates the performance of Mortality in Emergency Department Sepsis Score (MEDS), Modified Early Warning Score (MEWS), Rapid Emergency Medicine Score (REMS), and Rapid Acute Physiology Score (RAPS) in predicting risk of mortality in ED adult patients with renal abscess. This will help emergency physicians, surgeons, and intensivists expedite the time-sensitive decision-making process. Methods Data from 152 adult patients admitted to the EDs of two training and research hospitals who had undergone a contrast-enhanced computed tomography scan of the abdomen and was diagnosed with renal abscess from January 2011 to December 2015 were analyzed, with the corresponding MEDS, MEWS, REMS, RAPS, and mortality risks calculated. Ability to predict patient mortality was assessed via receiver operating curve analysis and calibration analysis. Results MEDS was found to be the best performing physiologic scoring system, with sensitivity, specificity, and accuracy of 87.50%, 88.89%, and 88.82%, respectively. Area under receiver operating characteristic curve (AUROC) value was 0.9440, and negative predictive value was 99.22% with a cutoff of 9 points. Conclusion Our study is the largest of its kind in examining ED patients with renal abscess. MEDS has been demonstrated to be superior to MEWS, REMS, and RAPS in predicting mortality for this patient population. We recommend its use for evaluation of disease severity and risk stratification in these patients, to expedite identification of critically ill patients requiring urgent intervention.


Introduction
Renal abscess, while being relatively uncommon at an incidence of 1 to 10 in 10,000 hospital admissions [1][2][3], is a debilitating and potentially fatal disease with mortality rates historically ranging from 12% to 50% [2,4]. With the advent of ultrasonography and computed tomography imaging techniques enabling the early identification of patients with renal abscess and subsequent prompt intervention, survival rates have improved; nevertheless, mortality still reaches up to 8.3% currently [3,5,6].
Management of renal abscess comprises antimicrobial therapy, percutaneous drainage, surgical drainage, and nephrectomy [2,4,7]. Though surgical drainage and nephrectomy are often avoided as first-line interventions, Meng et al. (2002) demonstrated that 35% of patients ultimately required exploratory laparotomy and nephrectomy while Coelho et al. (2007) found that 18.5% and 23.1% of renal abscess patients had received surgical drainage and nephrectomy, respectively [4,5]. Apart from being the last-ditch intervention after failure of medical therapy, nephrectomy is also indicated for renal abscesses which develop in small, scarred, poorly functioning kidneys [8]; however, it is almost impossible to establish this history in the emergency department (ED) setting as patients with such a condition usually present in a critical state, complicating the time-sensitive decisionmaking process for emergency physicians and surgeons.
Early appropriate intervention improves the survival outcome of each renal abscess patient; therefore, it is imperative that they are initiated on the right therapy in the ED immediately after accurate assessment of disease severity and the corresponding mortality risk. There is however yet a clearly defined, objective risk stratification tool that lays down the criteria which justifies the decision of choosing medical therapy over surgical intervention and vice versa.
Various studies have suggested that the prognosis of renal abscess patients is related to certain variables such as the presence of lethargy, elevated blood urea nitrogen, thrombocytopenia, abscess size, and serum C-reactive protein levels [6,9,10]. These variables nevertheless do not constitute a complete risk stratification tool, and thus we employed four already-established ED scoring systems and studied their performance in predicting the mortality risk of renal abscess patients in the ED.
The four chosen physiologic scoring systems are the more commonly used ones: Rapid Acute Physiology Score (RAPS) [11], Rapid Emergency Medicine Score (REMS) [12,13], Modified Early Warning Score (MEWS) [14], and Mortality in Emergency Department Sepsis Score (MEDS) [15,16]. These systems are comprised of easily and rapidly obtainable parameters that can be calculated by the bedside, allowing emergency physicians, surgeons, and intensivists to quickly and accurately identify critically ill patients in whom urgent intervention is necessitated.
This study is a continuation in our series of investigations into the use of scoring systems for risk stratification of patients with abdominal infections to improve patient outcomes. Our first study evaluated the performance of Simplified Acute Physiology Score II (SAPS II), the Acute Physiology and Chronic Health Evaluation II (APACHE II) score, and the Sequential Organ Failure Assessment (SOFA) score in predicting mortality of adult hepatic portal venous gas (HPVG) patients presenting to the ED and found that SAPS II performed the best [17]. We however realised that SAPS II, a score devised for intensive care units (ICUs), was not optimised for all ED environments. This was because some parameters of SAPS II required results of various investigations, readily available in the ICU, but cumbersome or not possible in some ED settings.
As such, we undertook a second study using ED scoring systems which incorporated parameters based on rapidly obtainable vital signs, while concurrently expanding our recruitment of adult ED HPVG patients to span 16.5 years. REMS was shown to outperform RAPS and MEWS [18].
Our third study was embarked upon to determine if there was any value in consideration of patient characteristics on top of clinical parameters, and if the results derived from the HPVG patient population could be similarly applied to splenic abscess ED patients. Statistical analysis results demonstrated that MEDS was superior to MEWS, REMS, and RAPS, thus giving merit to the hypothesis that patient characteristics should also be considered in patients with abdominal infections. We were however surprised to note that the performance of REMS in predicting mortality of splenic abscess patients was ranked last [19].
This current study was conceived with three aims: first, to evaluate the suitability of these four aforementioned ED scoring systems in patients with renal abscess, an abdominal infection with differing aetiology and pathogenesis from HPVG and splenic abscess; second, to ascertain the superiority of MEDS in the ED renal abscess patient population as a risk stratification tool which incorporates clinical parameters with patient characteristics; third, to assess the next-best score amongst RAPS, REMS, and MEWS to employ. MEDS's strength of taking into account patient characteristics is a potential weakness if past history cannot be adequately established quickly. This could be due to neurological deficits of the patient, lack of accompanying family members, or language barriers, amongst other issues, Plan B is required in such cases.

Study Design.
This retrospective analysis was conducted at the EDs of two training and research hospitals, Linkou Chang Gung Memorial Hospital (3406 beds with approximately 17000 monthly ED visits in 2017) and Taipei Chang Gung Memorial Hospital (252 beds with approximately 4200 monthly ED visits in 2017). The Chang Gung Memorial Hospital Institutional Review Board approved this study (IRB: 201701502B0C501), waiving the need for consent from study participants. Data was accessed anonymously.

Settings and Subjects.
All adult patients above the age of 18 years admitted to the ED who had undergone a contrastenhanced computed tomography scan of the abdomen and   [20]. Patient mortality or survival upon discharge were taken as study endpoints.

Statistical Analysis.
Median and interquartiles of numerical variables, as well as frequencies and corresponding percentages (%) of categorial variables, were recorded. Univariate analyses were conducted to establish the relationship between predictors and mortality rates; Mann-Whitney U tests and Fisher's exact tests were selected for such analyses for numerical and categorical variables, respectively, in view of the small nonsurvivor group size. Univariate logistic regression analysis was done to develop predictive models between scoring systems and mortality, with Hosmer-Lemeshow statistical analysis used to determine model goodness of fit. Receiver operating characteristic (ROC) curve analysis was performed to evaluate the performance of scoring systems in predicting mortality. Following that, optimal cutoff point for each scoring system was identified by maximizing Youden's index. Last but not least, the corresponding accuracies, sensitivities, specificities, positive predictive values, and negative predictive values of the respective optimal cutoff points were calculated.

Patient Involvement.
No patients were directly involved in this study. Hospitalization records were accessed anonymously, with only the relevant required information extracted for the study.    (Table 5). Univariate logistic regression analysis of the scoring systems with respect to probability of death found the following odds ratios: 1.571 for MEDS (p = 0.0001), 1.529 for MEWS (p = 0.0035), 1.619 for RAPS (p = 0.0009), and 1.528 for REMS (p = 0.0009) ( Table 6). Hosmer-Lemeshow statistical tests found all four models to be a good fit (MEDS 0.8249, MEWS 0.3598, RAPS 0.2430, and REMS 0.3111).

Discussion
This study is the largest yet examining renal abscess patients in the ED and also the first to the best of our knowledge to use ED physiologic scoring systems for risk stratification of these patients. We found MEDS to be the most ideal tool in predicting mortality rates of ED patients with renal abscess.
MEDS was designed and validated by Shapiro et al. in 2003 to risk stratify ED patients suspected to have infections   according to risk of mortality. It comprises 9 parameters: age > 65 years, nursing home resident, rapid terminal comorbid illness, lower respiratory tract infection, bands >5% on a white blood cell count differential, tachypnea or hypoxemia, septic shock, platelet count < 150 x 10 9 /L, and altered mental status. The calculated score is then directly proportional to the patient's mortality risk [15]. Its original intended target population of ED patients with infections sets it apart from the other 3 scoring systems, which were designed with different patient populations in mind. This might explain why MEDS is the most accurate tool in predicting mortality of renal abscess patients, as these patients usually succumb to sepsis and septic shock.
MEDS has also been demonstrated to be a good predictor of prognosis in patients with other intra-abdominal infections [19,21,22] and even performs better than other more established risk assessment tools including APACHE II and quick Sepsis-related Organ Failure Assessment in determining mortality risk of ED patients in infection and/or sepsis [23][24][25][26][27][28][29][30]. Its superiority in this study provides further evidence that both clinical presentation and patient characteristics are significant in assessing mortality risk of patients with abdominal infections as mentioned in our introduction, almost half of the 9 criteria in MEDS (age, nursing home resident, terminal illness, altered mental status) are dependent on the patient's medical history that is easily and quickly ascertainable in the ED.
Univariate analysis in this study found that pulse rate, respiratory rate, tachypnea or hypoxia, SpO 2 , mean arterial pressure, Glasgow Coma Scale, altered mental status, estimated glomerular filtration rate, presence of septic shock, and positive history of terminal illness were significant in predicting the mortality of a renal abscess patient. These findings further reinforce the relationship between septic shock and mortality risk; other than presence of septic shock, most of the other significant parameters are classical findings in septic patients: tachycardia, tachypnea, hypotension, altered mental status, bandemia, and acute kidney injury. Patients with terminal illnesses are also more prone to developing sepsis. Interestingly, the number of abscesses did not have a significant impact on patient mortality, suggesting that preventing, limiting, and resolving sepsis are more important in mortality reduction than abscess characteristics.
The success of MEDS thus lies in its more suitable physiological parameters and incorporation of 5 significant variables (terminal illness, tachypnea or hypoxia, septic shock, bandemia, and altered mental status) into its scoring, enabling it to stratify patients with the highest accuracy rate of 88.82% and sensitivity of 87.50% amongst the 4 scoring systems studied. The high negative predictive value of 99.22% further enables emergency physicians and surgeons to decisively exclude renal abscess patients with a MEDS score of less than 9 from the high-risk group, and as such conservative management of antimicrobial therapy and/or percutaneous drainage can be justified, thus avoiding unnecessary emergency nephrectomies. The AUROC value of 0.9440 for MEDS also demonstrates that it is ideal for predicting mortality in patients with renal abscess. Though MEDS comprises more parameters than the other 3 physiologic scores, the extra information required is easily established through history taking and biochemical investigations routinely done during patient evaluation in the ED.
We nevertheless recognize that there will be certain situations in which patient history cannot be adequately established quickly in the ED, rendering emergency physicians unable to calculate MEDS. In these cases, our findings show that REMS is the next-best risk stratification tool to predict mortality in renal abscess patients, similar to study conclusions of our investigation into HPVG patients [18]. Future efforts could be directed into further determining the superiority of REMS in relation to RAPS and MEWS for ED populations with other abdominal infections, so as to identify the optimal score to employ if only patient's vital signs are available.
Despite being the largest study of renal abscess patients in the ED, this study is still limited by the small number of participants. There were only 2 patients who fulfilled each criteria of "nursing home resident" and "lower respiratory tract infection", two important factors in the MEDS scoring system; the utility of MEDS in our study may therefore be an underestimate. Larger studies which include more patients who fulfil these criteria would aid in providing an even more accurate assessment of MEDS. Further studies to confirm these findings and prospectively validate the use of MEDS in this patient population is also required.

Conclusion
MEDS score is the best performing physiologic scoring system amongst the four studied systems in predicting the mortality of renal abscess patients. We recommend its employment in the ED for rapid risk stratification to promptly ascertain which patients require urgent intervention, thus ensuring timely and early intervention and subsequently improved patient outcomes.

Data Availability
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Ethical Approval
The Chang Gung Memorial Hospital Institutional Review Board approved this study (IRB: 201701502B0C501), waiving the need for consent from study participants.

Consent
Obtaining a consent form was not applicable in this work as data was accessed anonymously.

Disclosure
Su-Han Chang and Chiao-Hsuan Hsieh are the first authors. Both funders had no role in study design, data collection and analysis, decision to publish, or preparation in the manuscript