Comparison of General and Liver-Specific Prognostic Scores in Their Ability to Predict Mortality in Cirrhotic Patients Admitted to the Intensive Care Unit

Introduction Acute Physiology and Chronic Health Evaluation (APACHE) II and III and Sequential Organ Failure Assessment (SOFA) are prognostic scores commonly used in the intensive care unit (ICU). Their accuracy in predicting mortality has not been adequately evaluated in comparison to prognostic scores commonly used in critically ill cirrhotic patients with acute decompensation (AD) or acute-on-chronic liver failure (ACLF). Aims This study was conducted to evaluate the performance of prognostic scores, including APACHE II, SOFA, Chronic Liver Failure Consortium (CLIF-C) SOFA, Child–Turcotte–Pugh (CPS), Model for End-Stage Liver Disease (MELD), MELD-Na, MELD to serum sodium ratio (MESO) index, CLIF-C organ failure (CLIF-C OF), CLIF-C ACLF, and CLIF-C AD scores, in predicting mortality of cirrhotic patients admitted to the ICU. Patients and Methods. A total of 382 patients (280 males, mean age 67.3 ± 10.6 years) with cirrhosis were retrospectively evaluated. All prognostic scores were calculated in the first 24 hours of ICU admission. Their ability to predict mortality was measured using the analysis of the area under the receiver operating characteristic curve (AUC). Results Mortality was observed in 31% of the patients. Analysis of AUC revealed that CLIF-C OF (0.807) and CLIF-SOFA (0.776) had the best ability to predict mortality in all patients, but CLIF-C OF (0.749) had higher prognostic accuracy in patients with ACLF. CLIF-SOFA, SOFA, and CLIF-C AD had the highest AUC values in patients with AD, with no statistical difference (p=0.971). Conclusions When compared to other general or liver-specific prognostic scores, CLIF-C OF, CLIF-SOFA, SOFA, and CLIF-C AD have good accuracy to predict mortality in critically ill patients with cirrhosis and patients with AD. According to the clinical scenario, different scores should be used to provide prognosis to patients with cirrhosis in the ICU.


Introduction
End-stage liver disease, particularly due to hepatitis B and C, alcoholic liver disease, and nonalcoholic steatohepatitis, accounts roughly for 1.16 million deaths worldwide [1]. It usually evolves over several years from compensated to decompensated cirrhosis, which is identified by the onset of acute decompensation (AD) with the development of hepatic encephalopathy (HE), ascites, and variceal hemorrhage from 39% to 83%, depending on the reason for ICU admission, presence of organ failure, and sepsis [7].When compared to patients without cirrhosis, critically ill patients with the disease have more infections at ICU admission, increased overall mortality, and increased mortality due to sepsis or septic shock [8].
Several ICU and liver-specific scores have been used to predict outcomes of critically ill patients with cirrhosis [7,[9][10][11][12][13][14], as well as futility rules to withhold intensive care support [15,16]. e most often used ICU scores are Acute Physiology and Chronic Health Evaluation (APACHE) II and III and Sequential Organ Failure Assessment (SOFA) scores [17], whereas liver-specific scores routinely applied to patients with cirrhosis are Child-Turcotte-Pugh (CTP) and Model for End-Stage Liver Disease (MELD) scores [18,19]. Both were designed to predict mortality in patients with cirrhosis, respectively, after surgery [18] and transjugular intrahepatic portosystemic shunt (TIPS) placement [19]. While CTP is commonly used in clinical practice to assess disease severity, MELD is also currently used for indication and prioritization for liver transplantation [20].
Due to its better assessment of organ failure, ICU scores usually have a better accuracy to predict mortality, when compared to CTP and MELD scores [12][13][14]. Recently, the concept of ACLF was introduced in the literature by the Chronic Liver Failure Consortium (CLIF-C) to describe a syndrome characterized by advanced chronic liver disease associated with organ failure and a 28-day mortality higher than 15% [25,26]. e authors have employed a prospectively validated modified SOFA score (CLIF-SOFA) to characterize CLIF-C OF and proposed two new prognostic scores: CLIF-C ACLF [27] for patients with ACLF and CLIF-C AD [28] for patients with AD of cirrhosis, without ACLF.
It is important to note that the CLIF-C criteria for ACLF have not been endorsed worldwide, and different definitions have been proposed by the Asian Pacific Association for the Study of the Liver-ACLF Research Consortium and the North American Consortium for the Study of End-Stage Liver Disease (NACSELD) [29]. e purpose of the present study was to evaluate the accuracy of liver-specific prognostic scores, such as CTP, MELD, and its variants MELD-Na, iMELD, and MESO index, as well as ICU scores, such as APACHE II and SOFA, in their ability to predict in-hospital mortality of cirrhotic patients admitted to the ICU with either AD of cirrhosis or ACLF and also to assess the performance of CLIF-C AD and CLIF-C ACLF, respectively, in those patients with either AD or ACLF.

Patients and Methods
All patients admitted to the Gastroenterology and Hepatology Unit of Hospital Português, from January 2012 to June 2018, with AD of cirrhosis or ACLF, were retrospectively reviewed. is ICU is a referral unit for critically ill patients with cirrhosis in Salvador, Brazil. e diagnosis of cirrhosis was based on clinical, biochemical, and echographic findings, as well as liver histology, whenever liver biopsy results were available. e etiology of cirrhosis and clinical features responsible for ICU admission were recorded in all patients. All cirrhotic patients admitted in the postoperative period of abdominal surgery, including liver transplantation, intra-arterial chemoembolization for hepatocellular carcinoma, and patients with HIV coinfection or advanced liver cancer were excluded from the study. Data regarding demographics, presence of comorbidities, cause of cirrhosis, clinical features, and baseline laboratory parameters including leucocyte counts, platelets, international normalized ratio (INR), total bilirubin, serum sodium, albumin, and creatinine levels were collected. General ICU and liver-specific prognostic scores were calculated within 24 hours of ICU admission, including updated Charlson Comorbidity Index (CCI), SOFA, APACHE II, CTP, MELD, MELD-Na, MESO-index, and iMELD, as previously described [17-19, 21, 23, 24, 30]. Patients were categorized into two groups, according to the presence of ACLF or AD without ACLF [31]. In addition, CLIF-C ACLF [27] and CLIF-C AD [29] scores were calculated on day one of ICU admission in patients, respectively, with ACLF and AD of cirrhosis. Acute kidney injury was diagnosed according to International Club of Ascites definition [32]. ACLF and AD of cirrhosis were evaluated in the first 24 hours of admission and graded based on CLIF-C criteria [26,31]. NACSELD definition of ACLF [33] was also used based on parameters obtained within the first 24 hours in the ICU for better characterization of the patients. e presence of organ failures was assessed based on definitions of either CLIF-C or NACSELD [26,27,31,33]. Patients were followed up until death, liver transplantation, and 28-day survival. e primary outcome was in-hospital mortality or transplant-free survival. is study was approved by the Ethics Committee in Research of Hospital Português, Salvador, Bahia.

Statistical Analysis.
Dichotomous variables are presented in text and tables as numbers and percentage and continuous variables were expressed as mean ± standard deviation (SD) or as median and interquartile range, respectively, whether the distribution was normal or skewed. Demographic, clinical, and laboratorial variables were comparing between survivors and nonsurvivors using the chi-square test or Fisher's test for categorical variables or Student's t-test or the Mann-Whitney U test for continuous variables when appropriate. All scores were compared using nonparametric receiver operator characteristic (ROC) curves with respective 95% confidential interval (95% CI).
e areas under the curve (AUC) provided the discriminative ability of the score and were compared as previously described [34]. Additionally, the prognostic score with the highest AUC obtained was considered a gold standard ROC curve. e other scores were compared to the gold standard using the Bonferroni-adjusted significance probability. In this analysis, models with an AUC equal to or greater than 0.7 were considered clinically significant. e Youden index was used to identify the optimal cut-off point for each score [35], and the corresponding sensitivity, specificity, positive, and negative predictive value (PPV and NPV) with respective 95% CI and likelihood ratio positive (LR+) and negative (LR−) were calculated. Statistical analyses were performed with the Statistical Package for Social Sciences (SPSS Inc., Chicago, IL, USA), version 21.0 for Windows, Stata for Mac (Stata Corp LLC., Texas, TX, USA), version 13.0, and OpenEpi, version 3.01 [36]. A p value <0.05 was considered significant. Quintile's cut-off points to Apache II, MELD-Na, SOFA, CLIC-SOFA, CLIC-C OF (for all patients), CLIF-C ACLF (for patients with ACLF), and CLIF-C AD (for patients with cirrhosis with AD), in addition to sensitivity and specificity parameters, were obtained using TG-ROC curves with graphic analysis, using Prism for Mac, version 9.1.2 (GraphPad Software, San Diego, California USA).
Mortality rate was significantly associated with clinical features of bacterial infections, HE, and AKI (Table 2). Patients with ACLF assessed by either CLIF-C or NACSELD criteria had higher mortality when compared to their  Canadian Journal of Gastroenterology and Hepatology counterparts without ACLF ( Table 2). As expected, increased risk of death was associated with the number of organ failures, assessed by either CLIF-C or NACSELD criteria ( Figure 1). Other variables associated with in-hospital mortality were creatinine and bilirubin levels, INR, and leukocyte count, as well as the need for vasopressors and mechanical ventilation at admission. All general and liverspecific prognostic scores were significantly higher in nonsurvivors when compared to their counterparts who were discharged alive from the hospital, except for the CLIF-C AD score (Table 2). ROC curves were used to assess the ability of the scores calculated within 24 hours of admission to predict in-hospital mortality for all patients and those with either ACLF or AD ( Figure 2 and Table 3). SOFA (0.753; 95% CI: 0.708-0.796), CLIF-SOFA (0.776, 95% CI: 0.724-0.827), and CLIF-C OF scores (0.807; 95% CI: 0.758-0.855) had the highest AUC values in all critically ill cirrhotic patients, and these scores were not statistically different from each other (p � 0.083) (Figure 3). Since the CLIF-C OF score was considered the reference score (gold standard), the AUC values of the MELD (p � 0.013), MELD-Na (p � 0.037), and APACHE II (p � 0.042) scores were significantly lower than CLIF-C OF.
In patients with ACLF, higher AUC values were obtained with CLIF-C OF (0.749; 95% CI: 0.679-0.820), when compared to CLIF-C ACLF (0.665; 95% CI: 0.585-0.745; p � 0.029), SOFA (p � 0.037), and MELD-Na (p � 0.002), but no CLIF-SOFA score (p � 0.085). It is of note that CLIF-SOFA, SOFA, and CLIF-C AD had the highest AUC values in those patients with AD of cirrhosis, with no statistical difference (p � 0.971) (Figure 2 and Table 3).    e most discriminative cut-off point was determined using the highest Youden Index (for each prognostic score), and corresponding sensitivity, specificity, PPV, NPV, LR +, and LR− are shown in Table 4 and Figures 4(a)-4(c). Figure 4(a) shows that the most discriminative cut-off obtained to all patients was similar between patients with ACLF and AD to APACHE II score (cut-off � 17). But higher cutoff points considered optimal were observed for the other analyzed mortality prognostic scores in the group of patients with ACFL compared with patients with AD of cirrhosis (Figures 4(b) and 4(c)). Notably, the optimal cut-off points were associated with higher specificity values, reaching maximum values for MELD-Na (89.2% and 97.8%, respectively in patients with ACLF and AD of cirrhosis).
In Table 5, we observe that the quintiles of the cut-off points for the APACHE II, MELD-Na, SOFA, CLIF-SOFA, and CLIF-C OF scores produce a low prediction for death in patients with cirrhosis with AD, even considering the 80th percentile. More consistent values were observed in the group with ACLF.

Discussion
is 6-year retrospective single-center study evaluated 382 critically ill patients with cirrhosis with and without ACLF, which was assessed within 24 hours of ICU admission. Inhospital mortality rate was 31%, in accordance with previous reports, showing an increase in survival of cirrhotic patients admitted to the ICU in recent years [7,14]. In the present study, mortality was associated with clinical and laboratory parameters, previously associated with prognosis, in several other studies, including bacterial infections or sepsis [8,35,37], HE [38], AKI [32,39], and ACLF [26,31] at admission; baseline sodium [12,31,40], creatinine [12,31,40], bilirubin [12,31,40], INR [12,31,40], and leukocyte counts [12,31,40]; and need for mechanical ventilation [12,31] or vasopressors [12,31].  We included several prognostic scores to predict mortality in cirrhotic patients since liver and ICU scores have been proposed. However, the discriminatory power of these scores in critically ill cirrhotic patients is not well defined. Few studies have analyzed the predictive value of MELD score, including its modified versions to predict in-hospital mortality [12].
As previously described [31,33], occurrence and severity of ACLF, defined by either CLIF-C or NACSELD criteria, were robust predictors of hospital mortality. In the present cohort, 91% and 53% of the patients who had ACLF at admission, respectively, by NACSELD and CLIF-C definition, died at the hospital. Using AUC analysis, the authors have compared the performance of general and liver-specific scores in their ability to predict in-hospital mortality in all cirrhotic patients admitted to the ICU as well as in those patients with or without ACLF. We have found that SOFA, CLIF-SOFA, and CLIF-COF scores had a better performance when compared to other general ICU and liver-specific scores, such as APACHE II, CCI, MELD, and its variants and CTP scores in the entire group of patients. SOFA score was previously associated with better prognostication, particularly when compared to MELD and CTP scores, for every patient with cirrhosis in several [9,12,[41][42][43][44] but not all publications [45]. Due to these findings, CLIF-C investigators adapted SOFA score to incorporate INR, instead of platelet count (CLIF-SOFA), to better evaluate liver dysfunction and organ failure (CLIF-C OF) in critically ill cirrhotic patients [26,27,31]. It is worth mentioning that both scores outperformed SOFA in their ability to predict mortality in the present study. Other authors have found similar performance of SOFA and CLIF-SOFA [14,40,46,47] or better performance of CLIF-SOFA over SOFA [48] when calculated within 24 hours of ICU admission and even better prognostication when recalculated after 72 hours in some [14,46] but not all studies [47]. ose differences may be due to comparison of heterogeneous cohorts comprised of patients from different genetic backgrounds and more importantly with differing percentages of organ failures. Another key point to better understand these discrepancies is to recognize that most of those studies, evaluating the accuracy of the prognostic scores, used distinct time intervals to assess outcomes, including in-ICU [25], inhospital [40,42,45,46], 28-day [49,50], 6-week [9], and 90day mortality [50]. Few studies have assessed accuracy of general ICU and liver-specific prognostic scores in critically ill patients with cirrhosis according to the presence of ACLF [16,41,51,52], and none of them have compared the accuracy of those scores to predict in-hospital mortality. In the present study, CLIF-C OF outperformed other scores including CLIF-C ACLF and CLIF-SOFA in patients with ACLF and CLIF-C AD had a good accuracy to predict mortality in those patients with AD. Our findings were different from those reported by other North American and European authors [51,52], who have reported better prognostication in patients with ACLF using CLIF-C ACLF score. It is important to mention that this score was prospectively developed using the CANONIC cohort of hospitalized Caucasian patients with cirrhosis, not particularly in the ICU, with external validation using an independent French cohort [27]. e CLIF-C ACLF score is calculated combining the CLIF-C OF score with age and leukocyte count and outperformed, up to now, all other prognostic scores in the evaluation of cirrhotic patients with ACLF. One possible reason for the discrepancies observed in the present study, in face of all others, is not just the short time interval to assess mortality (in-hospital mortality), but also some clinical features presented by our patients, such as older age, with mean age at least 10 years higher when compared to other studies, and a high index of comorbidities. Different genetic backgrounds could also be responsible since CLIF-C ACLF was validated almost exclusively in European and North American Caucasian patients [27,51]. On the other hand, CLIF-C AD had better accuracy to predict mortality in comparison to CTP, MELD, and MELD-Na in our patients with AD of cirrhosis. CLIF-C AD was also developed by the CANONIC group of investigators [28], using the following parameters: age, INR, serum creatinine, and leukocyte count. Other [53][54][55], but not all [50], studies have disclosed similar findings. Our results report a good ability of CLIF-C AD to predict in-hospital mortality in cirrhotic patients admitted to the ICU with AD of cirrhosis, per definition without ACLF.
To increase the prognostic accuracy of the mentioned scores, several authors have suggested that incorporation of Table 4: Performance of different prognostic scores in predicting mortality using the optimal cut-off point in all patients, patients with ACLF (CLIF-C ACLF) and patients with AD of cirrhosis (CLIF-C AD).

Score
Cut-off point Youden         lactate in MELD [56] and CTP [25], as well as CLIF-C ACLF [52] scores, could increase their capacity of predicting mortality. One limitation of our study was the unavailability of data concerning lactate to confirm the findings.
It is noteworthy that in this study, we did not aim to assess the clinical course of patients with ACLF. erefore, the scores were not recalculated throughout the hospital stay. ese data can be evaluated in further studies. Additionally, the low rate of transplantation in this cohort helps to avoid a competing-risk analysis, since higher rates of transplantation during hospitalization could modify prognostic scores accuracy.
In conclusion, lower mortality rates are nowadays observed in cirrhotic patients admitted to the ICU, particularly in the absence of ACLF. In their ability to predict survival, for patients admitted in the ICU, the following scores outperformed other prognostic scores: CLIF-C OF and CLIF-SOFA, for all cirrhotic patients, CLIF-C OF, for patients with ACLF, and SOFA, CLIF-SOFA, and CLIF-C AD, for patients with AD. Stratification of patients with or without ACLF at admission, as well as during hospital stay, is important to improve prognostication. According to the clinical scenario, different scores should be used to provide prognosis to patients with cirrhosis in the ICU.

AD:
Acute decompensation ACLF: Acute-on-chronic liver failure CLIF-C: Chronic Liver Failure Consortium NACSELD: North American Consortium for the Study of End-Stage Liver Disease CCI: Charlson comorbidities index APACHE II: Acute Physiology and Chronic Health Evaluation II MELD: Model for End-Stage Liver Disease MELD-Na: Sodium MELD MESO index: MELD to serum sodium ratio index iMELD: Integrated-MELD CTP: Child-Turcotte-Pugh SOFA: Sequential Organ Failure Assessment CLIF-SOFA: CLIF Sequential Organ Failure Assessment CLIF-C, OFCLIF-C: Organ failure CLIF-C ACLF: CLIF acute-on-chronic liver failure CLIF-C AD, CLIF-C: Acute decompensation.

Conflicts of Interest
e authors declare no conflicts of interest. Table 5: Cut-off points, sensibility, specificity, area under curve, and positive predictive value (to predict mortality) of the quintiles of APACHE II, MELD-Na, SOFA, CLIF-SOFA, CLIF-C ACLF, and CLIF-C AD scores.