Childhood asthma surveillance using administrative data : Consistency between medical billing and hospital discharge diagnoses

Montreal Department of Public Health, Montreal, Quebec Correspondence: Dr France Labrèche, Institut national de santé publique, Occupational Health, 190 Crémazie Boulevard East, Montreal, Quebec H2P 1E2. E-mail france.labreche@inspq.qc.ca F Labrèche, T Kosatsky, R Przybysz. Childhood asthma surveillance using administrative data: Consistency between medical billing and hospital discharge diagnoses. Can Respir J 2008;15(4):188-192.

A sthma is a common cause of acute and chronic morbidity among children in Canada (1,2).Asthma prevalence increased dramatically between 1980 and 1995, and appears more recently to have levelled off (3,4).As the leading chronic disease in children, asthma has become a major public health challenge and its surveillance requires increased attention (3,4).
Administrative data have been used to study various aspects of asthma among children in Canada, from surveillance of trends in medical visits and hospitalizations (5), to the generation of hypotheses regarding the influence of prenatal and perinatal factors (6) or socioeconomic status (5-7) on the risk of developing asthma, to characterizing patterns of health service use (8)(9)(10).However, little information is available on the quality of the diagnosis of asthma in administrative data (11).
As a preliminary step to the development of ongoing surveillance of pediatric asthma in Montreal, Quebec, we explored the feasibility of linking administrative files derived from hospital discharge summaries with medical billing claims to characterize pediatric asthma occurrence and health service use, giving particular attention to the issue of completeness of the two files in identifying asthma hospitalizations.The present ©2008 Pulsus Group Inc.All rights reserved

ORIGINAL ARTICLE
paper reports on our assessment of the face validity of the two records of services received for asthma by examining the accuracy and completeness of each source against the other, with hospital discharge summaries considered here as being more accurate.

Starting databases
Two databases were used for the present study.The first one is an administrative hospitalization database, maintained by the Quebec Health and Social Services Ministry, that contains information from individual discharge summaries (age, sex, postal code of residence, dates of admission and of discharge, primary diagnosis and up to 15 secondary diagnoses).The second database collects billing claims submitted by physicians remunerated on a fee-for-service basis (ie, 88% of all physicians working in Quebec [12]) and contains information on presumed diagnosis and place of visit (office, specialized clinic, emergency room, in-hospital); these billing claims constitute what is referred to as the medical billing database.
Permission to access these databases was granted by the provincial agency administering the Act Respecting Access To Documents Held By Public Bodies.Data linkage was performed by the medical insurance administration which, before transferring the data, assigned an encrypted identification number to individual records, precluding nominal identification but permitting construction of longitudinal data records.

Original files and study population
The present feasibility study was conducted among three age groups (children who became one, four or eight years of age during 1998 while residing on the Island of Montreal [Montreal and suburbs, population 1.8 million]) to portray agerelated differences in asthma.The administrative data were provided in two files.
The hospitalization file contained information on all children hospitalized at least once in 1998 with a primary or secondary discharge diagnosis of asthma (International Classification of Diseases, Ninth Revision, code 493).For all such children, information on all respiratory medical billing claims for any respiratory condition (International Classification of Diseases, Ninth Revision, codes 460 to 519, Diseases of the respiratory system) during 1998 was then attached.
The medical billing file consisted of billing claims by physicians for all Montreal children seen for asthma or other respiratory conditions between January 1 and December 31, 1998.

Data handling and analysis
Estimates of the proportion of Montreal children visiting a physician or admitted to hospital were calculated using population estimates provided for each of the three age groups by the Quebec Institute of Statistics for the year 1998 (13).
Medical billing claims were enumerated for Montreal children who were hospitalized at least once with a primary diagnosis of asthma in 1998.The first hospitalization for asthma in 1998 was set as the 'index hospitalization'; subsequent hospitalizations were numbered consecutively.For physician billings for asthma in which the place of service was entered as in-hospital, the earliest data in 1998 were considered as an index against which the hospitalization file was assessed for correspondence.Correspondence between data sources was explored as described below.
First working file from the hospitalization file: For the first hospital admission in 1998, as well as for the two subsequent hospitalizations, a corresponding asthma or respiratory disease medical billing was sought for each hospitalized child.Three time periods were explored: the first day of hospitalization, the entire hospital stay, and the hospital stay plus a one-day margin before admission and after discharge (referred to hereafter as 'hospital stay ± 1 day').A two-day margin before and after the hospital stay was also explored, but there were very few additional billing claims identified on the second day (data not presented).Second working file from the medical billing file: For each child for whom there was a medical billing claim for an asthma diagnosis in-hospital, a corresponding hospitalization for asthma as primary or secondary diagnosis was looked for, again during three time periods: the day of hospital admission, the entire hospital stay and the hospital stay ± 1 day.
Concurrent validity, the extent to which the medical billing data correlate with the hospitalization file data, was estimated by Cramer's V and Goodman-Kruskal lambda ( 14).Cramer's V is a measure of the association between two variables (coefficient ranging between 0 and 1).A value of 0.45 or greater was interpreted as two variables measuring the same concept ('redundant' variables [15]).The Goodman-Kruskal lambda was interpreted as the percentage reduction in errors in predicting a variable (eg, number of hospitalizations for asthma in 1998), given knowledge of another variable (eg, medical billings for asthma in-hospital during the same year).

Population characteristics
In 1998, 364 Montreal children (0.6%) aged one, four and eight years were discharged with a primary diagnosis of asthma, accounting for a total of 449 hospitalizations (Table 1).During the same year, there were 32,759 children for whom a physician billed at least once for a respiratory diagnosis.Of these, 4702 children (14.4%) received an asthma 'diagnosis', according to the medical billing file.These numbers give an estimated annual prevalence rate of 7% to 9% of Montreal children aged one, four and eight years, with a medical billing claim for asthma in 1998.

Correspondence between diagnoses at hospital discharge and on medical billings
A medical billing for asthma in-hospital at any time during 1998 was found for 295 of 364 children (81%) hospitalized with a primary diagnosis of asthma (Table 2).The 'hospital stay ± 1 day' time period identified 78% of children who also had contemporaneous in-hospital medical billings for asthma and 84% with contemporaneous in-hospital medical billings for all respiratory diagnoses.The two other time periods captured a smaller proportion of medical billings, because there were no contemporaneous medical billings for asthma in-hospital during the first hospital day for 33% of the hospital records, and during the entire inpatient stay for 29% of hospitalizations, in which asthma was the primary diagnosis recorded at discharge.Even when medical billings for any respiratory diagnosis in-hospital were considered, there were still 16% of hospitalized children for whom one could not find any indication of medical billing in-hospital during the entire hospital stay ± 1 day.
When data for the two following hospitalizations were explored, the correspondence was somewhat better: an asthma billing claim was found during the hospital stay ± 1 day for 87% (second hospital stay) to 94% (third hospital stay) of hospitalized children (Table 2).
When looking at correspondence any time during 1998, a contingency table (Table 3) shows that 102 of 498 children (20.5%), reportedly hospitalized at least once in 1998, could not be found in the medical billing file.Nevertheless, the Cramer's V value of 0.495 depicts a fairly good association regarding hospitalization between the two data sources: children hospitalized with a primary diagnosis of asthma are more readily identified by a medical billing for asthma, and children hospitalized with a secondary diagnosis of asthma are more readily identified by a medical billing for other respiratory diseases.

Correspondence between medical billings in-hospital and hospital discharge diagnoses
Exploring correspondence the other way around, only 72% of the 409 children with a medical billing in-hospital were found in the hospitalization file with a primary diagnosis of asthma (Table 4), and 66% of them during dates corresponding to the entire hospital stay ± 1 day.For these children, most billing claims (169 of 269 claims, data not shown) had been submitted on the day before hospital admission, possibly because of the delay between a medical decision to admit a child from the emergency room and the actual date the child arrives on the ward, considered the date of admission in the administrative database.When hospitalizations for both primary and secondary diagnoses were considered, 82% of the children were found in the hospital discharge database, and 74% of these children were found during the entire hospital stay ± 1 day.For these children, most billings (178 of 304 claims, data not shown) had been submitted on the day before hospital admission.Inversely, no hospitalization data for a primary diagnosis of asthma could be found for 28% of children with a medical billing claim for asthma in-hospital; when hospitalizations with both primary and secondary diagnoses of asthma were considered, that proportion went down to 18% of children with a medical billing for asthma in-hospital.
Considering the 1055 medical billings for asthma or for other respiratory disorders in-hospital, one can estimate concordance anytime in 1998 with a contingency table (Table 5) that gives a Goodman-Kruskal lambda of 0.55 (95% CI 0.43 to 0.68).This means that if one takes into consideration the different types of medical billing claims submitted for asthma inhospital, one reduces, on average, estimation errors by 55% compared with only taking into consideration the existence of a billing claim in-hospital.

DISCUSSION
The universal aspect of Canada's Medicare system ensures virtually complete population coverage for most health services.Administrative data are available at relatively low cost (compared with dedicated surveys), and allow for follow-up of individuals over time, which makes these data suitable for use in surveillance systems.However, the quality of recorded information may vary with time and place, and among physicians and hospitals.Also, the clinical validity of data collected for billing or administrative purposes may be debatable, but their strengths and weaknesses are nonetheless worth exploring (16).

Methodological considerations
This was a feasibility study, using a one-year snapshot of data for children aged one, four or eight years in 1998.Any data linkage relies on the assumption that the identification number is correct in both files, and this could not be verified in the present study.As alluded to earlier, a certain proportion of physicians are on alternative or mixed fee payment plans (eg, university-appointed full-time staff) and are not required to submit billings for every patient seen.Although it is not possible to estimate precisely the importance of the resulting underestimation, it is unlikely to be large given that 88% of full-time equivalent physicians were remunerated on a fee-forservice basis in 1998 (12).This underestimation appears to be distributed without any particular pattern among the hospitals, because the discrepancy between number of billings and number of hospitalized children is proportional to the number of hospitalized children in each hospital (data not shown), even for hospitals whose medical staff is exclusively remunerated on a fee-for-service plan.
The importance of resource use to hospital management makes it less likely that a child would be hospitalized without any trace of the hospital stay in the hospitalization database as opposed to the medical billing database.This led us to consider hospital discharge data as a 'relative' gold standard in terms of completeness.The issue of diagnostic accuracy is more debatable: the child could have been admitted for asthma symptoms and later diagnosed with an upper respiratory illness.However, an asthma diagnosis would probably still appear as a secondary diagnosis.
As for the medical billing file, our study design did not allow for a check of completeness of capture of physician visits.However, a recent study estimated that more than 92% of medical billing claims from the Montreal area include a presumptive diagnosis (17), and thus this data source can be expected to be fairly complete in capturing most respiratory visits (18).Physicians have to submit a claim to get paid, so most of them probably do submit, unless they are on an alternative payment plan or have neglected to bill for an unknown reason.Diagnostic accuracy of labels attached to medical billing claims is more liable to interpretation: the 'diagnoses' are based on the impression of the treating physician at the time of visit.It is thus possible that some mislabelling takes place (eg, a 'known' asthmatic patient presenting with a persistent cough could be coded as consulting for asthma rather    than for another, and perhaps more appropriate, respiratory diagnosis).The physician could also use a more general diagnostic category, such as ill-defined respiratory symptoms, until the results of the investigation confirm a specific diagnosis.

Incidence and prevalence estimates of asthma
The effect estimates obtained in the present study may be considered a rough estimate of the incidence for one-year-old children and as period prevalence for four-and eight-year-old children.These estimates are derived from billing claims and hospital admissions during one single year, and thus differ from lifetime occurrence rates derived from questionnaires or medical record reviews.The annual effect estimates show that between 7% and 9% of Montreal children consulted a physician for asthma at least once during 1998.Of those, up to 10% were admitted to hospital.These numbers are comparable with those obtained in an independent study of asthma in Quebec using distinct hospitalization and medical visit files for 1992, 1994 and 1996 (17), as well as with numbers obtained from questionnaire surveys in Quebec (19,20).

Concurrent validity of administrative records of health services use
Our analyses focused on consistency of diagnoses attributed to each child from different data files rather than on their clinical validity.Our findings showed that approximately 22% of hospitalizations for asthma did not have a corresponding physician visit with an in-hospital billing claim for asthma, and even when in-hospital billing claims for any respiratory problem were examined, there still were 16% of hospitalizations that were not found in the billing file.Thus, not all hospitalized children were found in the billing file.Having said this, is the billing file useful?The discrepancy does not seem to be associated with any particular hospital (even though the proportion of physicians on a fee-for-service plan varies between hospitals) and is possibly attributable to patients' clinical evolution, diagnostic orientation of certain physicians, or diagnostic errors.Only a study including medical record review may properly validate the different sources of diagnoses.The main message is that there is appreciable overlap between the two administrative files and that both data sources can be helpful for surveillance purposes.Our study does suggest, however, that neither source captures all asthma hospitalizations.
Childhood asthma surveillance Can Respir J Vol 15 No 4 May/June 2008 191

TABLE 2
Correspondence between children listed in the hospitalization file and medical billings for the same children, by time of service: Children from the Island of Montreal, aged one, four and eight years, hospitalized for asthma as primary diagnosis in 1998

TABLE 1
Characteristics of hospital admissions and medical billings for respiratory conditions among children from the Island of Montreal hospitalized for asthma in 1998 (13)ed on estimates from the Quebec Institute of Statistics(13).Asthma diagnosis is according to International Classification of Diseases, Ninth Revision, code 493.Respiratory diagnoses are according to International Classification of Diseases, Ninth Revision, codes 460 to 519

TABLE 4
Correspondence between children listed in the medical billing file and those in the hospitalization file: Children from the Island of Montreal, aged one, four and eight years, with in-hospital medical billings for asthma in 1998

TABLE 3
Distribution of children from the Island of Montreal, aged one, four and eight years, hospitalized for asthma as primary or secondary diagnosis, according to labels attached to medical billings submitted for the same children, any time in 1998