Construction and Validation of a Risk Prediction Model for Acute Kidney Injury in Patients Suffering from Septic Shock

Background Acute kidney injury (AKI) is an important complication in critically ill patients, especially in sepsis and septic shock patients. Early prediction of AKI in septic shock can provide clinicians with sufficient information for timely intervention so that improve the patients' survival rate and quality of life. The aim of this study was to establish a nomogram that predicts the risk of AKI in patients with septic shock in the intensive care unit (ICU). Methods The data were collected from the Medical Information Mart for Intensive Care III (MIMIC-III) database between 2001 and 2012. The primary outcome was AKI in the 48 h following ICU admission. Univariate and multivariate logistic regression analyses were used to screen the independent risk factors of AKI. The performance of the nomogram was evaluated according to the calibration curve, receiver operating characteristic (ROC) curve, decision curve analysis, and clinical impact curve. Results A total of 2415 patients with septic shock were included in this study. In the training and validation cohort, 1091 (64.48%) of 1690 patients and 475 (65.52%) of 725 patients developed AKI, respectively. The predictive factors for nomogram construction were gender, ethnicity, congestive heart failure, diabetes, obesity, Simplified Acute Physiology Score II (SAPS II), angiotensin-converting enzyme inhibitors (ACEI) or angiotensin receptor blockers (ARBs), bilirubin, creatinine, blood urea nitrogen (BUN), and mechanical ventilation. The model had a good discrimination with the area under the ROC curve of 0.756 and 0.760 in the training and validation cohorts, respectively. The calibration curve for probability of AKI in septic shock showed optimal agreement between prediction by nomogram and actual observation. Decision curve and clinical impact curve analysis indicated that the nomogram conferred high clinical net benefit. Conclusion The proposed nomogram can quickly and effectively predict the risk of AKI at an early stage in patients with septic shock in ICU, which can provide information for timely and efficient intervention in patients with septic shock in the ICU setting.


Introduction
Septic shock is a life-threatening severe disease caused by circulatory and cellular metabolic abnormalities; it affects 10%-30% of patients in the intensive care unit (ICU) [1,2]. The mortality rate of septic shock is estimated to be 45%-63% [3]. Septic shock is the most common cause of acute kidney injury (AKI) in critically ill patients. The decreased renal blood flow, secondary tubular epithelial cell death, or acute tubular necrosis is the core mechanisms underlying AKI in septic shock patients [4,5]. The prevalence of AKI in patients suffering from septic shock was up to 60.47%, and the mortality rate is as high as 62.1% [5,6]. In addition, the development of AKI during septic shock increases the patients' mortality and prolongs the hospital stay [7,8]. However, AKI can be prevented in the early stage of septic shock because of the compensatory and reverse recovery function of the kidneys [9]. Thus, a simple and convenient method can be used to quickly assess the risk of AKI in septic shock.
For the past decades, many researchers have focused on the risk factors of AKI in patients suffering from septic shock. Among them, blood or serum biomarkers, such as the delta neutrophil index (DNI), proenkephalin (PENK), urinary interleukin-18, urinary KIM-1, and neutrophil gelatinaseassociated lipocalin (NGAL), have attracted attention [10][11][12][13][14]. However, these indicators are difficult to apply to clinical practice due to the high cost and technical requirement. Moreover, several risk factors of AKI development in critically ill patients, including older age, obesity, mechanical ventilation, low white blood cell (WBC), and platelet counts, have been explored [15]. However, these indices are not stable when they solely acted as a single index due to the effects of confounding factors. Thus, it is more appropriated to incorporate them into a comprehensive model. A nomogram is a userfriendly tool with graphical representation that can be used to calculate the probability of a specific event for each individual. Compared with single indexes, nomograms can more accurately estimate the risk of AKI for individual patients by incorporating multiple risk factors. Recently, several nomograms have been developed for the prediction of AKI in many diseases. Deng et al. incorporated the information of 2917 patients with sepsis, including blood urea nitrogen (BUN), infusion volume, serum lactate, weight, serum chloride, body temperature, and age, to formulate a nomogram to predict AKI during the first 24 h of ICU stay, and the model showed excellent performance with a C index of 0.80 [16]. A nomogram using the routine information in ICU was wellcalibrated and clinically useful for the prediction of AKI in patients with diabetic ketoacidosis [17]. However, nomograms for predicting AKI in patients suffering from septic shock are yet to be reported. As the high mortality rate of AKI in septic shock is partly due to a delay in diagnosis, identifying the septic shock patients at high risk of AKI can help clinicians take timely and effective intervention measures, reduce the mortality, and improve the quality of life. Therefore, this study is aimed at developing and validating a nomogram for predicting AKI in patients suffering from septic shock in the ICU.

Methods
2.1. Source of Data. The data of this study was extracted from the Medical Information Mort for Intensive Care III (MIMIC-III) database [18]. MIMIC-III is a large, free accessible intensive care database that contains the detailed information of more than 40,000 patients admitted to the critical care units in the Beth Israel Deaconess Medical Center (BIDMC) from 2001 to 2012, including demographic characteristics, monitoring vital signs, laboratory and microbiological examination, imaging examination, observation and recording of intake and output, drug treatment, length of stay, survival data, and discharge or death records. To apply for access to the database, we passed the protection of human research participant examination and obtained the certificate (No. 9983480). Structured Query Language (SQL) was used to extract all patients' information from the MIMIC-III database.

2.2.
Participants. Inclusive criteria were as follows: (1) first admission to ICU and (2) diagnosis of septic shock upon admission. Exclusion criteria were as follows: (1)    or chronic kidney disease upon admission. For patients older than 89 years, the actual age could not be obtained due to the date of birth being 300 years before the first admission [19]. Thus, our study excluded age over 89 years old.

Diagnosis of Septic Shock and AKI.
According to the Third International Consensus Definitions (ICDs) for Sepsis and Septic Shock (Septic-3), septic shock was defined as an infection or a suspected infection with a vasopressor requirement to maintain a mean arterial pressure of 65 mmHg or greater and serum lactate level of >2 mmol/l (18 mg/dl) in the absence of hypovolemia [20]. In the present study, we identified patients suffering from septic shock based on the ICD-9 (78552). The primary outcome of this study was the development of AKI within 48 h after ICU admission. A diagnosis of AKI was made according to the acute kidney injury network (AKIN) criteria [21], including the increase of serum creatinine ≥ 26:5 μmol/l or 1.5 times higher than baseline within 48 h or urine out < 0:5 ml/(kg·h) for more than 6 h.
The reasons for the use of AKIN rather than KDIGO criteria for AKI diagnosis in this study are as follows: (1) the data recorded by the MIMIC-III database were earlier than the release time of KDIGO criteria and (2) KDIGO criteria were based on the changes of kidney function within 7 days, but many confounding factors, such as antibiotics and nosocomial infection, may affect renal function [22].

Research
Variables. Data of each patient, including demographic characteristics, comorbidity, vital signs, interventions, and laboratory examinations, were obtained from the MIMIC-III database [23,24]. Demographic characteristics, including age, gender, and race, were collected from the original database. Vital signs, including respiratory rate (beats/min), heart rate body (beats/min), and temperature (°C) upon ICU admission, were collected from charting in CHARTENS. Laboratory parameters, including WBC count, hemoglobin, platelet count, BUN, potassium, bicarbonate, chloride, neutrophils, lymphocytes, and lactic acid, were

Disease Markers
data of the first 48 h of the ICU stay. All comorbidities were identified according to the ICD-9 code records. Variables with >20% missing values were excluded from further analysis, and variables with ≤20% missing values were filled with the multiple imputation.
2.5. Statistical Analysis. All eligible patients were divided into the training and validation cohorts with the split ratio of 7 : 3. The data of the training cohort were used to perform the logistic regression analysis and construct the nomogram, whereas the data in the validation cohort were used to validate the model. Continuous variables were expressed as median with quartile. These data were compared by t-test or rank sum test, as appropriate. χ 2 test or Fisher's exact test was used to compare the categorical variables. Potential multicollinearity between variables was judged by the variance expansion factor (VIF). A VIF of ≥5 was considered evidence of multicollinearity [25]. Univariate and multivariate logistic regression models were used to select the independent risk factors of AKI in the training cohort. According to the results of multivariate logistic regression analysis, the nomogram was developed to predict the risk of AKI in patients suffering from septic shock in the ICU. The performance of the nomogram was first quantified in the training cohort and then in the validation cohort in terms of discrimination, calibration, and clinical utility. The discrimination of the nomogram was evaluated by the area under curve (AUC) of the receiver operating characteristics (ROC) curve. The calibration curve was drawn to examine the consistence of the predicted probabilities and the observed outcomes. The clinical applicability of the nomogram was estimated by the decision curve analysis (DCA) and clinical impact curve. The statistical significance of all analyses was set at a P level less than 0.05. All analyses were performed using R version 4.0.5 (http://www.r-project.org/).

Baseline Characteristics.
The procedure for subject selection is shown in Figure 1. A total of 2415 patients suffering from septic shock were included in the final analysis. Among these, 1566 patients (64.84%) developed AKI in 48 h after ICU admission. Thirty-eight variables, namely, age, gender, ethnicity, obesity, congestive heart failure, hypertension, diabetes, aminoglycoside, glycopeptide antibiotics, nonsteroidal antiinflammatory drugs (NSAIDs), stain, ACEI/ARBs, APSIII, SAPSII, heart rate, systolic pressure, diastolic pressure, respiratory rate, temperature, SpO2, anion gap, bicarbonate, bilirubin, creatinine, chloride, glucose, lactate, platelets, potassium, prothrombin time (PTT), activated partial thromboplastin time (APTT), BUN, white blood cell (WBC), neutrophils, lymphocytes, Gram-positive bacteria, Gram-negative bacteria, and mechanical ventilation, were collected from the MIMIC-III database. The clinicopathological characteristics of the eligible patients are shown in Table 1. The baseline clinicopathologic data were similar between the training and testing sets. AKI was detected in 64.48% (1091/1690) and 65.52% (475/725) of the patients in the training and validation sets, respectively.

Predictors of AKI and Nomogram Development.
After multicollinearity examination, no multicollinearity was found between variables, because all VIFs were less than 5 (Table 2). Thus, all 38 features were included in the univariate logistic regression analysis. Based on the univariate logistic regression analyses, 11 variables, namely, gender, race, congestive heart failure, diabetes, obesity, SAPS II, ACEI/ARBs, bilirubin, creatinine, BUN, and mechanical ventilation, were significantly associated with AKI development in patients suffering from septic shock (Table 3). In multivariate analysis, gender, race, congestive heart failure, diabetes, obesity, SAPS II, ACEI/ARBs, bilirubin, creatinine, BUN, and mechanical ventilation were identified as the independent risk factors for AKI in patients suffering from septic shock (P < 0:05, Table 4). Therefore, a nomogram for predicting AKI in patients suffering from septic shock was constructed based on these variables ( Figure 2).

Nomogram Validation.
First, the ROC curves are shown in Figure 3, and the discriminant results showed that the model has a good ability to distinguish AKI patients from non-AKI patients with AUCs of 0.756 and 0.760 in the training and validation cohorts, respectively. Second, the predictive probabilities of the model were in consistent agreement with the observation results in both the training and validation sets, thereby suggesting a good calibration (Figure 4). Third, the DCA ( Figure 5) and clinical impact curve analysis ( Figure 6) visually showed that the nomogram had superior overall net benefit within the wide and practical ranges of threshold probabilities and impacted patient outcomes, thereby indicating that the nomogram had significant predictive value. When the predicted probability thresholds were set as 30%-100% and 30%-93% in the training and validation cohorts, the net benefit ranges were 0%-50% and 0%-47%, respectively. The smaller the threshold was, the better the net benefit was.

Discussion
The incidence of AKI in septic shock patients reached as high as 64.5% in the ICU, similar to the findings of previously published studies. According to a report in Finland, a total of 488 (53.2%) AKI incidences were detected among 918 patients with severe sepsis during their stay in the ICU [26]. Another study found that AKI developed in 572 (57.7%) of the 992 patients with sepsis and septic shock patients [15]. However, the incidence rate of AKI was higher in patients suffering from septic shock than in patients suffering from other diseases. For example, the incidence of AKI was 40%-50% among patients with sepsis in the ICU [5]. Fan et al. reported that AKI was developed in 41.3% (314/760) of the patients with diabetic ketoacidosis after ICU admission [17]. Several mechanisms might contribute to the pathogenesis of AKI in sepsis, including inflamma-tion, microcirculatory dysfunction, and mitochondrial dysfunction [5]. The kidney is the most common organ involved in sepsis, and septic patients with kidney dysfunction have a higher risk of death than those without kidney dysfunction [27,28]. In sepsis, circulating toxins act on the vascular endothelium, reducing microcirculation blood flow and producing a large number of inflammatory mediators, such as tumor necrosis factor and transforming growth factor β. Inflammatory response waterfall can lead to "intrarenal shunting," in which the renal blood flow is shunted from the medulla to the cortex, resulting in medulla hypoperfusion even with increased renal blood flow [29]. Simultaneously, the tissue damage caused by sepsis increases the inducible nitric oxide (NO) synthase and decreases endothelial NO synthase activity. The dysfunction of NO synthase decreases nitricoxide-mediated endothelium-dependent vasodilation, resulting in the imbalance of local renal In DCA, the nomogram shows a more net benefit than full or no treatment across a threshold probability range. DCA: decision curve analysis; AKI: acute kidney injury.
8 Disease Markers microcirculation [30]. Moreover, in sepsis, the activation of bacterial phospholipids, related cytokines, and chemical factors in the circulatory system are involved in the occurrence of tubular epithelial cell damage, leading to renal tubular necrosis, which is an important cause of AKI [29]. In addition, under the stress state caused by sepsis, mitochondrial autophagy increases to remove damaged mitochondria, thus playing a role in protecting the kidney. However, with the progress of the disease, mitochondrial autophagy is inhibited, leading to the accumulation of damaged mitochondria, thereby mediating kidney damage [31].
Logistic regression analysis showed that gender, race, congestive heart failure, diabetes, obesity, SAPS II, ACEI/ ARBs, bilirubin, creatinine, BUN, and mechanical ventilation were significantly associated with AKI risk in septic shock. Compared with previous studies on sepsis-related AKI, this study had some new findings. Firstly, SAPS II score was the most sensitive indicator for predicting the risk of AKI in septic shock. SAPS II score is among the most commonly used methods to quantify the death risk of AKI patients in ICU. The higher the SAPS II score was, the greater the risk of AKI was. When patients develop AKI after ICU admission, SAPS II showed the best performance in the prediction of AKI among all scoring systems, including Acute Physiology and Chronic Health Evaluation II (APACHE II), Sequential Organ Failure Assessment (SOFA), Logistic Organ Dysfunction Score, and Organ System Failure [32]. SAPS II score was superior to APACHE II and SOFA scores, because it can predict the survival outcome in patients suffering from septic shock [33]. Secondly, hyperbilirubinemia is a common complication in septic patients; the bilirubin level in septic shock patients significantly increases in 72 h after ICU admission [34]. Hyperbilirubinemia can be found in 60% of patients with AKI [35]. Elevated serum bilirubin levels are an independent risk factor for the occurrence of AKI [36]. Elevated bilirubin may produce oxidative stress on renal tubular cells, trigger apoptosis, and aggravate renal ischemia-reperfusion injury that contributes to the development of AKI [37,38]. Thirdly, AKI is characterized by a sharp drop in the glomerular filtration rate and a rapid increase in serum creatinine, BUN, sodium, and water storage. Serum creatinine and BUN are common indicators of renal function dysfunction. Studies have shown that even slightly elevated levels of serum creatinine and BUN are significantly associated with an increasing risk of AKI after ICU admission [39]. Moreover, modest changes of serum creatinine were significantly associated with higher mortality, longer length of hospital stay, and heavier costs in patients with AKI [39].
Other risk factors were independently associated with AKI risk, including gender, complicating diseases (diabetes, congestive heart failure, and obesity), mechanical ventilation, and ACEI/ARBs therapy. Women were more prone to develop AKI due to the effect of estradiol level on patients suffering from septic shock. Ovarian levels of estradiol at >40 pg/ml were an independent risk factor of septic shockrelated AKI in women [40]. In a matched case-control study, Kim et al. found that the incidence rate of postoperative AKI was significantly higher in patients with diabetes than those without diabetes [41]. Diabetes can increase the susceptibility of renal ischemia/reperfusion injury [42]. Congestive heart failure causes the complex interaction between the heart and kidney and results in the damage of renal function. Decrease in renal perfusion is the main factor responsible for development of AKI in patients with congestive heart failure [43]. Gameiro et al. proposed that the risk of AKI development was 2.31 times higher in obese patients with sepsis than in nonobese patients [44]. A complex interplay of different mechanisms may increase the susceptibility to AKI in critically ill obese patients, such as glomerulopathy, low 9 Disease Markers inflammatory status, endothelial dysfunction, enhanced oxidative stress, activation of the renin-angiotensin-aldosterone system, and increased sympathetic nervous system activity [45]. Mechanical ventilation is connected with a threefold increase in the risk of developing AKI among critically ill patients [46]. Ventilator-induced lung injury might contribute to the development of AKI during the use of mechanical ventilation [47]. Moreover, the use of ACEI/ARBs can cause efferent arteriole dilatation, decrease renal blood flow, and reduce glomerular pressure, which may increase the risk of AKI in patients suffering from septic shock [48].
In this study, we developed a simple and rapid nomogram model for predicting AKI risk in patients suffering from septic shock. The variables identified by our nomogram can be easily obtained in clinic and can reflect the disease activity of patients, thereby providing clinically relevant information in the management of patients suffering from septic shock. The proposed nomogram showed a good performance in the discrimination, calibration, and clinical application and provided valuable information for the decision-making of the appropriated therapy options for individual patients. More importantly, it filled the gap between the high incidence of AKI in septic shock and the lack of reliable predictive model. We cited an example to show how to use the nomogram model. For example, we assumed that a Caucasian man with septic shock and diabetes had received mechanical ventilation with a SAPS II score of 45, a BUN level of 80 mg/dl, a bilirubin level of 15 mg/dl, and a creatinine level of 5 mg/dl. According to Figure 2, the score corresponding to each individual parameter was obtained from the first row (the "Point" axis). For this patient, the score of gender was 0 based on the male sex, and the score of diabetes was 6 based on the complication of diabetes. Finally, the overall score was calculated as the sum of points for all parameters (0 (sex) + 10 (Caucasian) + 0 (congestive heart failure) + 6 (diabetes) + 0 (obesity) + 42 (SAPS II) + 0 (ACEI/ARBs) + 21 (bilirubin) + 18 (creatinine) + 29 (BUN) + 11 (mechanical ventilation = 137). This score corresponded to a risk of developing AKI at an approximately 87% level.
This study had several limitations. First, selection bias might be inevitable, because this study was a retrospective analysis of secondary data. Second, missing data were handled with multiple imputations, which might decrease the accuracy of the model. Third, the model was constructed based on a US population. Thus, its generalizability to the global population was still unclear. Fourth, the nomogram was developed and validated by the same database. Thus, it was more reliable to validate it prospectively or at least in another database. Fifth, because the nomogram was built based on 11 indicators, the sensitivity of the model performance may decline if the data of one or two indicators for a patient were missing. Finally, we only considered traditional parameters and did not consider some valuable biomarkers that might contribute to AKI development.

Conclusion
By incorporating 11 independent risk factors of AKI in septic shock, a simplified score model was constructed for AKI risk estimation in patients suffering from septic shock in the ICU. The proposed nomogram showed good performance in terms of discrimination, calibration, and clinical application. By early assessment of the risk of AKI development, clinicians can implement more measures that are beneficial for patients suffering from septic shock. Further studies are needed to externally validate our model using a largesample prospective cohort study.

Data Availability
The data support the findings of this study and are available from the corresponding author upon reasonable request.

Ethical Approval
Ethical approval was waived.

Consent
Informed consent was unnecessary because the MIMIC-III research data are anonymous and publicly available.

Disclosure
The opinions expressed in this study are those of the authors and do not represent the opinions of the Beth Israel Deaconess Medical Center.