A cohort study for the derivation and validation of a clinical prediction scale for hospital-onset Clostridium difficile infection

1Department of Internal Medicine; 2Department of Medical Informatics; 3Department of Gastroenterology, Greater Baltimore Medical Center, Baltimore, Maryland, USA Correspondence: Dr Subhash Chandra, Department of Internal Medicine, Greater Baltimore Medical Center, 6565 North Charles Street, Baltimore, Maryland 21204, USA. Telephone 443-849-3760, e-mail subhash.budania@gmail.com Received for publication May 15, 2012. Accepted June 5, 2012 Clostridium difficile infection (CDI) is the leading infectious cause of hospital-acquired diarrhea and its prevalence is the on rise, doubling every five years between 1996 and 2003 (1,2). In addition to contributing to morbidity and mortality in hospitalized patients, it adds significant financial burden, with estimated attributable costs of $436 million to $580 million in 2003 (3). The presentation of CDI is variable, ranging from asymptomatic to diarrhea to fulminate colitis with septic shock. According to updates from the Society for Healthcare Epidemiology of America and the Infectious Diseases Society of America on clinical practice guidelines on CDI in 2010 (4), testing is recommended only for patients with diarrheal stool unless ileus is suspected and screening of asymptomatic patients is not considered to be clinically useful. In current practice, diagnosis is made by stool testing for C difficile toxin A and B via enzyme immunoassay and positive results are further confirmed by cell cytotoxicity assay. This approach lacks adequate sensitivity but the alternative culture approach is not clinically helpful in most cases due to prohibitive turnover time. originAl Article

In the past, multiple attempts were made to establish a clinical prediction scale, but they either lacked good discrimination power, or did not perform well on internal or external validation (5-7).Presently, multiple risk factors have been identified but no clinical prediction scale has been created that can be used widely.To fill this gap, our study aimed to develop and validate a clinical prediction scale for hospital-onset CDI.

Study design and settings
The present retrospective cohort study was conducted in a 360-bed community-based, not-for-profit hospital.The study protocol was approved by the institutional review board.

Subjects
All adult (>18 years of age) patients discharged from the medical service, including the medical intensive care unit, from October 2005 to September 2011 were included in the present study.Patients who were admitted <48 h or diagnosed to have CDI within the previous three months were also excluded to avoid possible bias from relapse or inadequate treatment of a recent infection.

Outcome measure
The primary outcome measure was hospital-onset CDI, which was defined as C difficile toxin-positive stool in cases of unformed stool after 48 h of hospital admission.Stool testing is performed on a routine basis using enzyme-immune assay at the Greater Baltimore Medical Center (Maryland, USA).Patients who tested positive within 48 h of hospital admission were assumed to have community-onset CDI.

data collection
Data collection was performed with the help of the medical information technology department.Data points collected include, but were not limited to, patient demographics (age, sex, living facility, etc), hospital admission, course and discharge details, laboratory values, medications used, discharge diagnosis based on International Classification of Diseases, Ninth Revision (ICD9-CM) code.Data were compiled using Access 2007 (Microsoft Corporation, USA).To test the accuracy of the data collection, the electronic medical records of 200 random patients were reviewed by two independent physician researchers.Data collection by the physician researchers was performed using Excel 2007 (Microsoft Corporation, USA).Physician researchers collected two random components of the developed risk score.Interobserver agreement was calculated between two physician researchers and data were populated with the assistance of the information technology department.

data analysis
The data collected using Access and Excel were imported into JMP (SAS Inc, USA) for analysis.The final data set was divided into two parts: the derivation cohort, which consisted of patients admitted during October 2005 through September 2009; and the validation cohort, which consisted of patients admitted during October 2009 through September 2011.Categorical variables are presented with median and interquartile range, and continuous variables as mean and SD.Univariate analysis for CDI was performed using the appropriate test based on distribution (parametric versus nonparametric) and type of variable (nominal, ordinal or continuous).The following variables were used to assess for possible association with CDI on univariate analysis: age, sex, living facility, high-risk antibiotics use, proton pump inhibitor use, new-onset unformed stool, hypoalbuminemia, mechanical ventilation, chemotherapy use and hospital length of stay.Individual antibiotic categories that were significantly associated with hospital-onset CDI on univariate analyisis were combined and defined as high-risk antibiotics.Variables with significant association on univariate analysis were entered into a stepwise logistic regression model.In the forward stepwise logistic regression model, the P value threshold to enter in the model was set at 0.25, and at 0.1 to exclude from the model.Variables entered into the model were given a score based on ORs to calculate a simple and easy clinical prediction scale.Discrimination performance of risk index was assessed using area under the receiver operating curve (AUC) and the optimum cut-off point was chosen for optimal sensitivity and specificity.The Hosmer-Lemeshow goodness-of-fit statistic was used to test the calibration.

ReSULtS
Over the six-year period, a total of 52,068 patients were admitted to the medical service.Of these, 2.0% expired in the hospital, 5.2% were transferred to other acute care hospitals, 0.65% left against medical advice, and 21.7% were discharged to an assisted-living facility, nursing home or skilled nursing care.Figure 1 illustrates the flow of the subject selection process.

derivation of clinical prediction scale
The derivation cohort included all patients admitted to the adult medical service over a period of four years, from October 2005 to September 2009.In the derivation phase, 35,588 patients were admitted to medical service and 21,541 stayed in hospital >48 h.A total of 266 cases of CDI were identified; of these, 121 were hospital onset and the rest were community onset.Demographic and clinical characteristics are summarized in Table 1.
On univariate analysis, penicillins, cephalosporins and quinolones were significantly associated with hospital-onset CDI.These antibiotics were considered high-risk antibiotics in the present study.Increasing age and hospital length of stay were associated with increased hospitalonset CDI rate.A receiver operating curve (ROC) was constructed to define optimal cut-off age and hospital length of stay with optimum combination of sensitivity and specificity.Age >65 years and hospital length of stay >7 days were defined as cut-off points.Table 2 presents other factors positively associated with hospital-onset CDI on univariate analysis.
In the forward stepwise logistic regression model, new-onset unformed stool (ie, diarrhea), high-risk antibiotic use, hypoalbuminemia, long-term care facility residence, age >65 years and hospital length of stay >7 days were included based on set P value for inclusion into the model.Each variable was allotted points based on ORs to develop the scale (Table 3).An increase in score on the scale ).The developed scale had excellent performance in predicting hospital-onset CDI: AUC 0.93 (95% CI 0.92 to 0.95) (Figure 2, panel A).At a cut-off score of 9, the scale had a specificity of 0.67 (0.66 to 0.67), with a sensitivity of 0.96 (0.90 to 0.98) and missed seven cases (Table 4).
Two physician researchers manually collected data regarding the use of high-risk antibiotics and hypoalbuminemia for 200 randomly selected patients from the derivation cohort.The two reviewers had excellent interobserver agreement, with kappas of 0.81 and 0.86.The final extraction after resolving disagreements had excellent agreement with data pooled by medical informatics (kappa of 0.96 for both highrisk antibiotics use and hypoalbuminemia), which attested to the validity of the pooled data according to the medical informatics department.

Summary of findings
In our study, we developed and validated a clinical prediction scale for hospital-onset CDI risk from a retrospective cohort of consecutive patients discharged from the medical service of a suburban community hospital.The risk index had an excellent predictive performance that was maintained in the validation cohort with good calibration.The risk index includes major risk factors identified previously and was developed using logistic regression and a stepwise forward selection model.
Once these finding are validated in a prospective manner, the scale could have major clinical implications.The patients scoring ≥9 on this scale were at high risk of acquiring CDI during their hospital stay and should be considered for contact isolation at least, if not treatment.Clinical practice guidelines for the management of CDI updated in 2010 contained level III recommendations to treat all patients with    clinical suspicion of CDI while awaiting stool testing results (4).Based on these recommendations, any patient in whom stool testing for C difficile is ordered should be treated.In our derivation cohort, stool testing for C difficile toxin was requested in slightly more than 4000 patients (Table 5).Of these, more than 1000 patients scored <9 on the scale and none tested positive.Therefore, use of the scale could have safely avoided unnecessary treatment of 25% of the patients.On the other hand, slightly more than 1000 patients were not tested for C difficile but scored ≥9 on this scale.Twenty per cent of these patients either died in hospital or were transferred to hospice care.Hence, an assumption that some of these patients were undiagnosed cases of CDI is reasonable.Empirical treatment based on this scale would lead to treating approximately 40 patients for one true hospital-onset CDI.This approach appears to be cost effective because oral metronidazole is fairly inexpensive, whereas attributable costs for CDI-related hospitalization is more than $7,000 per patient (3).Additionally, early institution of therapy can potentially decrease CDI-associated morbidity and mortality, and may decrease miss rates secondary to the suboptimal sensitivity of the current enzyme immunoassay for C difficile toxin.If a treatment decision based on this scale is made at a higher cut-off value, the number needed to treat falls accordingly, approximately 26 for 10, 19 for 12, etc.

Comparison with the existing literature
In efforts to develop a clinical prediction scale for hospital-onset CDI, in 1996, Katz et al (7) developed a simple index from a retrospective cohort of 480 consecutive patients that included the use of antibiotics and new-onset diarrhea.This scale did not have adequate discrimination (AUC 0.69), which fell further on prospective validations (AUC 0.64) (6,7).Simultaneously, Cooper et al ( 5) developed a predictive model based on 271 patients for whom stool testing for C difficile toxin was ordered for diarrhea and 81 tested positive (5).The model included readmission within two weeks of previous hospitalization, leukocytosis (white blood cell count ≥10×10 9 /L) and the presence of fecal leukocytes.This model had very few positive outcomes in the validation cohort and has never been prospectively validated.Hornbuckle et al (8) developed another model that included the use of low-intensity chemotherapy, lack of parenteral vancomycin and previous hospitalization from 29 CDI cases admitted to an oncology unit.Including lack of treatment in its predictive model makes it less impressive and it also had very few patients.In Israel, a significant effort by Peled et al (9) studied 217 hospitalized patients with diarrhea and a history of antibiotic use and had 52 positive outcomes.The predictive model included six variables and had excellent discrimination power (AUC 0.90).The model included only high-risk patients and had not been internally validated.Garey et al (10) developed another score from a cohort of more than 50,000 patients, which is simple to calculate with moderate discrimination power but failed on external validation (10,11).
When compared with earlier developed scales, our clinical prediction scale was developed from a fairly large cohort of an unselected patient population and has better discrimination, which was maintained in the validation cohort and calculation still appeared to be simple.

Strengths, limitations and future directions
Our clinical prediction scale was derived and validated from a fairly large cohort of patients from a community hospital, which is a closer representation of the majority of hospitals in the United States.This increases the external validity of our results.We followed rigorous methodological standards for developing clinical decision rules (12).All of the variables included in this clinical prediction scale are readily available in all hospitalized patients without need for additional testing.The ease in calculation of this clinical prediction scale should overcome one of the barriers to using a scoring system with complex computing such as the acute physiological and chronic health evaluation system (ie, APACHE) (13).
One of the major limitations of the present study was its retrospective design.Hence, bias from faulty documentation was unavoidable.This clinical prediction scale was developed from a cohort of adult patients discharged from the medical service and could not be used on the surgical service unless validated in surgical patients.Furthermore, all of the subjects in the present study were from a single centre.

CONCLUSiON
We reported a clinical prediction scale that had excellent discrimination in predicting hospital-onset CDI.It is an easy to calculate, six-variable scale, with these variables readily available in most hospitalized patients.Once the scale is prospectively validated in different settings, it could have significant implication in management of hospital-onset CDI in terms of early institution of contact precautions for infection control and empirical treatment, and avoidance of unnecessary treatment in a significant proportion of hospitalized patients.

Figure 1 )
Figure 1) Flow of subject selection for both the derivation and validation cohorts.CDI Clostridium difficile infection

Table 1 Clinical and demographic characteristics of the derivation and validation cohorts who stayed in medical service >48 h Variable Cohort Derivation (n=21,541) Validation (n=10,793)
Data presented as n (%) unless otherwise indicated.*High-risk antibiotics: cephalosporins, penicillins and quinolones.C difficile Clostridium difficile