Development and validation of an administrative case definition for inflammatory bowel diseases

System database. A novel scoring system was developed that detected Crohn disease and ulcerative colitis patients with a specificity of >99% and a sensitivity of 99.1% and 86.3%, respectively. CoNClusioN: Through a robust methodology, a reproducible set of criteria to capture IBD patients through administrative databases was developed. The methodology may be used to develop similar administrative definitions for chronic diseases.

The purpose of the present study was to develop and validate an administrative 'case definition' for IBD. We identified IBD and non-IBD individuals through an endoscopy database in a western Canadian health region (Calgary Heath Region [CHR], Calgary, Alberta). Subsequently, through a novel algorithm, we developed a series of case definitions to capture IBD cases in the administrative databases. In the second stage of the study, we validated our criteria in a separate Canadian health region (Capital Health Region [Edmonton, Alberta]). The study represents the first independently validated administrative case definition for IBD.

MetHoDs setting
Alberta is a western province of Canada with a population of approximately 3.7 million. To receive universal health insurance coverage, Alberta residents must register with the Alberta Health Care Insurance Plan. At the time of the study, administration of the health system in Alberta was provided by health regions, with the largest health regions being the CHR and the Capital Health Region (Edmonton) with populations of 1.2 million and 1.1 million, respectively. The initial phase of the study was conducted in the CHR while the validation phase was completed in the Capital Health Region.
Institutional ethics approvals were obtained from the Ethics Review Boards of the Universities of Alberta (Edmonton) and Calgary (Calgary).

endoscopy databases
Two separate endoscopy databases were used in the present study: • Physicians in the tertiary care centres of the CHR record all endoscopic procedures in an electronic endoscopy database (Endopro; Pentax Inc, USA). Each record contains patient demographics, a brief medical history and physical examination data. This report also includes the indication for the procedure, endoscopic findings, impressions and recommendations.
• In the Capital Health Region, all of the gastrointestinal endoscopic procedures performed in tertiary or secondary care centres are abstracted in an electronic database. Each record includes patient demographics, PHN, indication for the procedure and an ICD-coded diagnosis.

Development of the administrative case definition for iBD
From May 2000 to March 2004, endoscopic procedure reports in the CHR were digitally and manually searched for IBD-related findings, indication or diagnosis. In addition, structured reviews were conducted on selected outpatient and inpatient medical charts of patients who underwent endoscopy in this time period. Patients with an indication or diagnosis of IBD on their endoscopy report or patients who met Lennard-Jones criteria (11) on chart reviews were considered to be IBD. Patients were classified as possible IBD if the indication for endoscopy was suspicious for IBD (eg, rectal bleeding); endoscopic findings were compatible but not specific to IBD (eg, nonspecific inflammation); or IBD was considered as a possible diagnosis at the end of the procedure. The remaining subjects were classified as non-IBD.
Using patients' names and demographics, PHNs for each individual were captured in the HCIPR by CHR personnel. Subsequently, available PHNs were linked to the PC, DAD and ACCS databases.
iBD case ascertainment in administrative data A total of 100 case definitions were developed depending on the number of physician claims (two to six claims), ACCS encounters (two or three encounters) and hospitalizations (one or two) with a diagnosis of IBD (ie, ICD-9 codes of 555 or 556; and ICD-10 codes of K50 or K51) in various time frames (one to five years). Each time frame ended with an administrative contact and did not necessarily fall within calendar or fiscal years. For this purpose, a time-floating computerized algorithm was developed (11).
Fifty additional case definitions were developed without the incorporation of the ACCS database because the ACCS or similar databases of outpatient encounters are not currently available in several Canadian provinces.
Using medical chart reviews and the endoscopic database as the 'gold standard' diagnostic tool, diagnostic characteristics (sensitivity, specificity, positive predictive value and negative predictive value with exact 95% CIs) of each administrative case definition were determined using contingent 2×2 tables.

Validation of the administrative case definition for iBD
The Capital Health Region's endoscopy database was searched from FY 1997/1998 through FY 2006/2007. Patients with an indication and diagnosis of IBD for their endoscopic procedure(s) were classified as IBD. Because there was no ICD code for 'normal' or 'healthy', capturing non-IBD subjects was not possible.
PHNs were linked to the PC, DAD and ACCS databases (FY 1995/1996to FY 2006/2007) and IBD-related ICD codes were extracted. Each of the 'case definitions' created in the development phase were examined. Using the endoscopic database as the 'gold standard' test, the sensitivity of each case definition with exact 95% CIs were calculated. Sensitivity was calculated with 10 years of administrative data (FY 1995/1996to FY 2005) to match the development phase (FY 1994/1995to FY 2004/2005. Given the almost perfect specificity of preferable case definitions and lack of a population-based non-IBD cohort, validation of specificity was not feasible.
Additional administrative data from FY 2005/2006 and FY 2006/2007 were used to assess the stability of sensitivity of each case definition over 11 and 12 years.

statistical analysis
The exact two-sample binomial test was used to compare the sensitivity of each case definition in development versus the validation phase. Assessment of the temporal stability of each case definition was also performed using the same test.
An a level of 0.1, instead of 0.05, was chosen to further decrease the possibility of accepting the null hypothesis by chance. All statistical analyses were performed using STATA 10.0 (Stata Corporation, USA).

selection of the final case definition
In order of importance, selection of the final administrative case definition was founded on the following: • Statistically equal sensitivity in the development and validation phase (ie, reproducibility). • Stability of sensitivity over time. As administrative data accumulate over time, a proportion of non-IBD subjects who incorrectly received IBD diagnostic codes will be misclassified as IBD. This will lead to a false increment in sensitivity and, conversely, a decrease in specificity. A case definition with unstable characteristics is not suitable for surveillance of administrative data due to the lack of reliability. • A very high specificity. IBD is a rare disease; therefore, a minimal false-negative rate is essential for a population-based diagnostic tool. • A reasonably high sensitivity. While a very high sensitivity is ideal, it sacrifices specificity as the first priority for the selection process. A sensitivity >75% was sought.

Distinction between uC and CD patients
To identify definite CD and UC patients, the IBD patients who were captured with the final case definition were evaluated in the development phase. Corresponding endoscopy report(s) for each patient were reviewed for indication, findings, or impression of UC or CD. Patients were categorized as IBDU if the review of the endoscopy report(s) failed to clearly differentiate UC from CD. Within a 10-year time period (FY 1994/1995to FY 2004/2005), health care contacts of UC and CD patients in the ACCS, DAD and PC databsases were enumerated. Within each administrative database (ie, ACCS, DAD or PC), patients were given a +1 score for any ICD diagnostic code for UC or a score of -1 for each ICD code for CD. Subsequently, for every patient, a cumulative score was calculated in each of the three databases. Considering various diagnostic cut-off values and different weights for each database, multiple combinations of these three scores were evaluated. The final scoring system classifies the subjects into CD, UC or IBDU if the score does not reach the cut-off values for diagnosis of CD and UC. Selection of the final scoring system was founded on high specificity and sensitivity rates to detect CD and UC patients.
The coding system for indication of endoscopic procedures on IBD patients in the Capital Health Region does not differentiate UC from CD; hence, performance assessment of the scoring system was not feasible in the Capital Health Region. The various phases of the study are shown in Figure 1.

Development phase
A manual and computerized search was conducted on 23,527 endoscopy reports of 21,193 patients. Of 21,193 patients, 17,699 (83.5%) were assigned a valid PHN through the linkage to HCIPR. Inability to assign a PHN to 16.5% of the individuals was mainly due to the procedures performed on non-CHR residents and missing demographic data in the endoscopy database. Based on the endoscopic data and 186 selected chart reviews, 17,699 individuals were categorized as non-IBD (n=15,439), possible IBD (n=861) and IBD patients (n=1399). Because the charts were not randomly selected, performance characteristics of endoscopic data in the diagnosis of IBD could not be calculated.
The PHNs of IBD and non-IBD patients were linked to the DAD, PC and ACCS databases for FY 1994/1995 through FY 2004/2005.
In total, 28,785 hospitalization records were found for 8632 patients. Sixty-one per cent of IBD patients experienced at least one hospital admission (median number of hospitalizations = 2), of which 75.4% had an ICD code for IBD. Non-IBD patients had a hospitalization rate of 50.4%, of which 0.08% had an ICD code for IBD.
Overall, 2,386,552 physician claims were found for the cohort of IBD and non-IBD patients. A total of 92.2% of IBD patients had at least one physician claim (median number of claims = 13), of which 95.0% had an ICD code for IBD. Moreover, 97.3% of non-IBD patients had a claim in the PC database, of which 3.8% had an ICD code for IBD.
In total, 240,142 contacts were found in the ACCS database. A total of 97.5% of IBD patients had at least one contact (median number of contacts = 2), of which 85.4% had an ICD code for IBD; 95.6% of non-IBD patients had a contact in the ACCS database, of which 0.6% had an ICD code for IBD. The individual performance of each administrative database is summarized in Table 1. In total 21,760 individuals with 12,554, 54,963 and 14,162 health care contacts with a diagnosis of IBD were identified within the DAD, ACCS and PC databases, respectively. The median number of health care contacts could not be calculated because only IBD-related ICD codes were extracted from the administrative databases.
Each of the previously developed 150 case definitions was applied to the IBD cohort and statistical comparisons of case definition characteristics in the validation and development phases were made. Fifteen case definitions that included the ACCS database and 14 case definitions that did not incorporate the ACCS database had statistically similar sensitivity in the validation phase (Tables 2 and 3). Temporal stability of the sensitivity of each validated case definition was also measured.

selection of the final case definition
After consideration of sensitivity, specificity and temporal stability of each validated case definition, a diagnosis of IBD was assigned to individuals who experienced at least two hospitalizations or four physician claims, or two contacts in the ACCS database with an IBD diagnostic code within a two-year period (definition 2-4-2-2). A 2×2 contingency table and stability analysis for the final definition are shown in Table 4 and Figure 2.

selection of the final case definition without incorporation of the ACCs database
It was proposed that individuals with at least one hospitalization or four physician claims with an IBD diagnostic code within a two-year period should be considered as IBD (Figure 3) Table 5. Cut-off values of 'greater than +2' (for UC patients) and 'less than -2' (for CD patients) were found to elicit an optimal balance between specificity and sensitivity to detect UC and CD patients. Patients with a score equal to or between -2 and +2 were classified as IBDU.

DisCussioN
A large proportion of IBD patients are diagnosed and managed in outpatient clinics. Only 60% of IBD patients are managed by gastroenterologists (10); the present study showed that less than one-half of IBD patients are hospitalized within an 11-year period. Therefore, the epidemiology of IBD will be under-represented if patients are captured only through hospitals and outpatient gastroenterology clinics. Health care administrative databases are invaluable to investigate the population-based epidemiology of IBD and provide an effective tool for ongoing disease surveillance. Canada has a universal public health care insurance program that provides access to comprehensive coverage for inpatient and outpatient physician services. This is in contrast to the administrative databases in the United States, which only cover the populations they serve.
Identification of the initial cohort from an independent source is a critical factor to accurately estimate the sensitivity of an administrative case definition. For this reason, we identified IBD and non-IBD patients from an endoscopy database. In addition, we integrated medical chart reviews as another independent source to enhance the accuracy of disease status. Some of the signs and symptoms of IBD are shared by other diseases; therefore, rather than a binary classification of our patients into IBD and non-IBD, we implemented a third group as 'possible IBD'. Defining a 'possible IBD' category decreases the number of false positives and false negatives in IBD and non-IBD groups, respectively.
In practice, it is not uncommon for a patient's diagnosis of UC to change to CD or vice versa (12). Moreover, patients with IBDU, with no specific ICD code, may arbitrarily be assigned CD or UC codes. Separate definitions for CD or UC may fail to identify or may misclassify these two groups of patients; therefore, we initially developed    (Table 1). In addition, in the same time period, 54.0% of IBD patients did not experience a hospitalization. A study from Ontario suggested that 8.6% of the physicians who see IBD patients do not use specific ICD-9 codes (13). In this context, in addition to commonly used DAD and PC databases, we incorporated the ACCS database, which is of significant value for IBD patients who repeatedly undergo endoscopic procedures for diagnosis, evaluation of disease activity and cancer screening. The ACCS database enabled us to identify 28.1% of IBD patients who failed to be captured using the DAD and PC databases. The National Ambulatory Care Reporting System that is being implemented by the Canadian Institute for Health Information shares similar content and function with ACCS and may be used as an alternative.
The value of each administrative database to identify IBD and non-IBD cases was not homogenous. Whereas hospitalization with a diagnostic code of IBD correlated correctly with the gold standard 98.0% of the time, this number for one IBD claim in PC database was only 70.0%. Therefore, in the development of our IBD case definition and our scoring system to differentiate CD and UC, different weights were applied to the IBD health care contacts in the DAD, PC and ACCS databases.
To enhance the reproducibility of an administrative definition, the currently acceptable method is to fractionate the data into shorter fixed periods. For instance, Hux et al (14) suggested division of a 10-year period into two-year phases to develop an administrative definition for diabetes in Ontario. We used a novel algorithm to enumerate ICD codes in multiple time periods to define IBD cases with optimal generalizability. Each time period starts with a health care contact and does not necessarily fall within a fiscal or calendar year. This method may have the advantage of capturing IBD flares by identifying peaks of health care contacts. Moreover, it facilitates future updates of the IBD database with ongoing acquisition of administrative data.
The difference among the test characteristics of a large number of case definitions in the development phase was minimal, making it difficult to select the final case definition. This finding was also observed in a study involving pediatric IBD patients (15). Statistical methods, such as Youden's index (16), aim for a balance between sensitivity and specificity; however, in population-based case definitions for rare diseases, a higher weight should be given to the specificity rate to minimize the number of false positives (9). The ideal method to compare multiple case definitions is through external validation with an independent source of diseased and nondiseased individuals. In Alberta, we have a unique opportunity with two distinct health regions with similar populations (ie, Capital Health Region [Edmonton] and the CHR). In addition, both health regions benefit from endoscopy databases independently collected from the DAD, ACCS and PC databases that enabled us to successfully validate the final case definitions.
To our knowledge, no attention has been devoted to the importance of the temporal stability of test characteristics of administrative case definitions. Application of case definitions to longer than original time periods may disproportionally increase the number of false positives and decrease the number of false negatives. Consequently, sensitivity of the case definition will increase and specificity will decrease, leading to inaccurate interpretation of administrative data. To avoid such systematic error, through a novel approach, we examined the stability of case definitions based on 10, 11 and 12 years of data (Figures 2 and 3).
The sensitivity of the case definition is higher in younger age groups. A similar pattern has been observed in other administrative databases (15). This finding has been attributed to closer follow-up in pediatric patients and also to the presence of other comorbidities in older age groups. For example, an elderly patient with IBD who presents with cardiac chest pain might not receive an IBD code during his health care contact.
A potential limitation of the use of administrative databases is the accuracy of the collected data (17). However, coding accuracy of administrative databases in Alberta has been validated and data have been used extensively in epidemiological and outcome studies (18)(19)(20)(21). Inherently, surgical/procedural and administrative databases capture more severe cases of disease (22). Similarly, moderate and severe cases of IBD may be over-represented in our cohort. This may significantly affect the generalizability of the results.
The disease status of the majority of our patients was determined through the endoscopy reports. Diagnosis of IBD may be recorded prematurely before the pathological confirmation of the disease. In the development phase, 89.9% of the endoscopic procedures performed on 'definite IBD' patients had an indication of IBD. In the validation phase, diagnosis of IBD was only assigned to patients whose endoscopy reports had an indication of IBD. Therefore, given the diagnosis of IBD in the indication of the procedure, the majority of our patients were known to have IBD. However, misclassification of disease status remains a potential systematic error in our study.
In the development phase, we linked the HCIPR with an endoscopy database using a combination of personal identifiers such as date of birth, sex and name. The 'deterministic record linkage' method is inferior to the use of unique identifiers (ie, PHN). However, in a recent study conducted in Alberta, the correct linkage rate was estimated to be 96.9% (23). To improve the specificity of our case definition, we did not include the 'possible IBD' cohort in the development phase. However, a subset of 'possible IBD' patients who have IBD do not necessarily have the same clinical and coding characteristics as those in the 'definite IBD' cohort. This may have introduced a selection bias in the study.
In the validation phase, patients were considered to have IBD only if they underwent endoscopies with an indication and diagnosis of IBD. Given the fact they underwent an endoscopic procedure with an indication of IBD -in addition to a final diagnosis of IBD -we speculated that the accuracy of this strategy should be very high. Subjective assessment of this strategy was not performed due to feasibility of structured outpatient and inpatient chart reviews on a large random sample of patients who are scattered over a vast geographical region.
In the development phase, patients were identified through an endoscopy database; therefore, the prevalence of IBD in our cohort was higher than in the general population. Our reported positive predictive value, which is influenced by disease prevalence, is likely overestimated and, in contrast, negative predictive value is likely to be underestimated. This limitation does not apply to the sensitivity and specificity rates, which are independent of prevalence.

CoNClusioN
The present study demonstrated that IBD patients can be accurately identified in administrative databases. Through a novel methodology, we have developed a reproducible set of criteria to capture IBD patients through administrative databases and classify them as UC, CD or IBDU. Application of this case definition to the administrative databases in Alberta will result in the formation of the largest IBD database in North America. Our methodology might be used to develop similar administrative definitions for chronic diseases and also to establish population-based IBD databases.

DisClosuRes:
The authors have no financial disclosures or conflicts of interest to declare.