Osteoarthritis (OA) is one of the most prevalent chronic health conditions that causes disability among the elderly [
The most common way to diagnose OA cases is the radiographic examination using Kellgren-Lawrence (K-L) grading system [
In this study we aimed to examine the validity of OA diagnoses recorded in the BC administrative database. Our primary objective was to determine the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and likelihood ratios of two administrative case definitions of OA. We examined the accuracy of these definitions using four reference standards that include X-rays, MRI, self-reports, and the ACR clinical criteria. Evaluating the validity of administrative OA diagnoses is an important step in conducting further research using these databases.
A cohort of 255 subjects with knee pain was recruited through population random sampling from Vancouver, BC, during the period August 2002 to February 2005. The subjects met inclusion criteria if they were between 40 and 79 years of age and had pain, aching, or discomfort in or around the knee at any time in the past 12 months. Subjects who had inflammatory arthritis, fibromyalgia, knee arthroplasty, a history of knee surgery/injury within the past 6 months, knee pain referred from the hip or back, and inability to undergo MRI were excluded. From the greater Vancouver telephone directory, 5,231 English-speaking persons were randomly contacted, of whom 3,269 (62.5%) agreed to participate in the survey. From the 3,269 subjects, 91.9% were ineligible due to age restriction and other exclusion criteria. Of the remaining 265 subjects, 10 were excluded due to missed appointments and for other reasons. The study sample recruitment procedure has been described elsewhere [
Administrative OA was defined in two ways based on ICD-9 and ICD-10 codes, referred to as AOA1 and AOA2. AOA1 required at least one visit to a health professional or one hospital admission with the ICD-9 code of 715 or the ICD-10 codes from M15 to M19, and AOA2 required at least two visits to health professionals in two years separated by at least one day or one hospital admission with these codes. For AOA2, the date of the second qualifying visit was used to assign the diagnosis date. These ICD codes include symptomatic and radiographic OA in any joint except the spine. The most commonly used pain medications for OA treatment are acetaminophen and nonsteroidal anti-inflammatory drugs [
Knee OA was assessed with a comprehensive questionnaire which included duration of knee pain, frequency of pain (number of days over the past month), and pain location using a knee diagram [
Knee radiography was completed within a month of the clinical assessment. Details on X-ray procedures have been described previously [
MRI for the most painful knee was performed within a month of clinical assessment. Detailed information regarding how MRI was performed has been described previously [
In the baseline questionnaire, knee OA was assessed by asking two questions. (1) “Has a doctor ever told you that you have osteoarthritis (also called degenerative or wear-and-tear arthritis) in your right knee?”, and (2) “has a doctor ever told you that you have osteoarthritis (also called degenerative or wear-and-tear arthritis) in your left knee?” Pain in the hip joints was assessed by the following instruction: “In the following homunculus diagram each circle represents a joint. Please mark each joint where you have experienced pain or discomfort over the past 12 months.” We counted subjects if they marked in the hip joints in the homunculus diagram.
For the selected subjects, knee OA was assessed based on the above four measurements. In addition, hand and hip OA were assessed using the ACR clinical criteria and the self-reported hip pain, respectively. Based on the knee, hand, and hip OA assessments, we defined four reference standards: RS1, RS2, RS3, and RS4. RS1 included assessments of knee and hand OA based on the ACR clinical criteria and hip OA based on self-reported hip pain. RS2 included assessments of knee, hand, and hip OA based on K-L grade, ACR clinical criteria, and self-reported hip pain, respectively. RS3 included assessments of knee, hand, and hip OA based on MRI cartilage score, ACR clinical criteria, and self-reported hip pain, respectively. RS4 included assessments of knee, hand, and hip OA based on self-reports, ACR clinical criteria, and self-reported hip pain, respectively. The same measurements for hand and hip OA were consistently included in the four reference standards.
Baseline characteristics of the cohort were age, body mass index (BMI) (kg/meter2), hip pain, symptomatic hand OA, and pain medication used. These characteristics were determined separately for men and women. We calculated the sensitivity, specificity, PPV, and NPV, for each case definition according to four reference standards. The 95% confidence intervals (CIs) were calculated for these statistics. For more detail about these measures, please refer to Rothman et al. [
Characteristics of 171 subjects by sex are presented in Table
Percentage of knee, hand, and hip osteoarthritis (OA) by each reference standard, knee OA assessment, and other baseline characteristics of 171 subjects who underwent comprehensive clinical assessment for knee OA by sex.
Characteristics | Women% | Men% | |
---|---|---|---|
Reference standards | |||
RS1 | 77.4 | 57.5 | <0.01 |
RS2 | 82.1 | 56.3 | <0.01 |
RS3 | 96.4 | 89.7 | 0.08 |
RS4 | 84.5 | 62.1 | <0.01 |
| |||
Knee OA assessment | |||
Clinical ACR criteria | 48.8 | 40.2 | 0.26 |
K-L grade ≥ 2 | 42.9 | 44.9 | 0.79 |
MRI cartilage score ≥ 2 | 91.7 | 88.5 | 0.48 |
Self-report | 50.0 | 48.3 | 0.82 |
| |||
Other characteristics | |||
Age in years | 0.44 | ||
40–49 | 14.3 | 21.8 | |
50–64 | 51.2 | 46.0 | |
65–79 | 34.5 | 32.2 | |
Body mass index (kg/m2) | 0.02 | ||
18.5–24.9 | 46.3 | 28.7 | |
25.0–29.9 | 27.4 | 46.0 | |
30+ | 26.2 | 25.3 | |
Hip pain | 42.9 | 18.4 | <0.01 |
Symptomatic hand OA | 43.4 | 18.4 | <0.01 |
RS1, RS2, RS3, and RS4 are four reference standards including knee, hand, and hip OA which are described in Methods.
K-L = Kellgren-Lawrence, self-report = self-reported physician diagnosed knee OA, MRI = magnetic resonance imaging, and ACR = American College of Rheumatology.
The validation results of two administrative OA definitions compared to the four reference standards are presented in Table
Validation results with the 95% confidence intervals of administrative definition of osteoarthritis using four reference standards that include knee, hand, and hip OA.
Reference standard | Administrative osteoarthritis | Sensitivity% (95% CI) | Specificity% (95% CI) | PPV% | NPV% | LR+ | LR− |
---|---|---|---|---|---|---|---|
RS1 | AOA1 | 55 (45–64) | 75 (61–85) | 82 (71–89) | 45 (34–55) | 2.2 (1.4–3.6) | 0.6 (0.5–0.8) |
AOA2 | 25 (17–34) | 91 (80–97) | 85 (68–94) | 37 (29–46) | 2.8 (1.2–6.9) | 0.8 (0.7–0.9) | |
| |||||||
RS2 | AOA1 | 55 (46–64) | 77 (63–87) | 84 (74–91) | 44 (34–54) | 2.4 (1.4–4.1) | 0.6 (0.5–0.7) |
AOA2 | 26 (18–35) | 94 (83–99) | 91 (75–98) | 36 (29–45) | 4.6 (1.5–14.5) | 0.8 (0.7–0.9) | |
| |||||||
RS3 | AOA1 | 47 (39–55) | 75 (43–93) | 96 (88–99) | 10 (5–18) | 1.9 (0.7–5.0) | 0.7 (0.5–1.0) |
AOA2 | 21 (15–29) | 100 (70–100) | 100 (87–100) | 9 (5–15) | >21 (21–∞) | 0.8 (0.7–0.9) | |
| |||||||
RS4 | AOA1 | 57 (48–66) | 87 (73–95) | 92 (83–97) | 42 (33–53) | 4.4 (2.0–9.3) | 0.5 (0.4–0.6) |
AOA2 | 26 (18–34) | 96 (84–99) | 94 (79–99) | 32 (24–41) | 6.5 (1.5–23.6) | 0.8 (0.7–0.9) |
AOA1 includes at least one visit to a health professional or one hospital admission for osteoarthritis and AOA2 includes at least two visits to health professionals in two years or one hospital admission for osteoarthritis. RS1, RS2, RS3, and RS4 are four reference standards that include knee, hand, and hip OA which are described in Methods.
PPV: positive predictive value; NPV: negative predictive value; LR+: positive likelihood ratio = sensitivity/(1 − specificity); LR−: negative likelihood ratio = (1 − sensitivity)/specificity.
Based on the BC administrative health records, we have assessed the validity of two case definitions of OA using four reference standards. The reference standards included radiographic K-L grade, MRI cartilage scores, self-reports, and the ACR clinical criteria for the knee OA assessments, the ACR clinical criteria for the hand OA assessments, and self-reported hip pain records for the hip OA assessments. Of the two administrative definitions, AOA1 had the higher sensitivity and NPV whereas AOA2 had the higher specificity and PPV. Validity measures were similar among the four reference standards in each case definition, while both case definitions of OA yielded a PPV of more than 82%.
Our validation results are comparable with those obtained in Lix et al.’s [
The limitations of the present study need to be acknowledged. First, we received written consent from 171 subjects to link their clinical data with the administrative records, which reduced the sample size. This reduction slightly changed the sample characteristics compared to those of the entire cohort [
The strengths of this study include the use of a representative clinical sample linked to administrative data. Our study featured a population-based cohort that included subjects with preradiographic as well as advanced radiographic knee OA. We compared two administrative OA definitions to the four reference standards. To our knowledge, this is the first study, to compare administrative case definitions and MRI-detected cartilage-based OA assessments. Administrative databases are frequently used in OA research. However, there are few validation studies of administrative OA diagnosis. The primary objective of selecting this study cohort was to assess MRI, X-rays, and symptomatic-based measures to detect early knee OA. In addition, symptomatic and self-reported data were collected for hand and hip OA, which enhances the present study. In a site-specific validation study focusing one joint at a time, the validation results may vary between sites. Since administrative diagnosis includes OA in any joint except the spine, our validation results are not affected by site-specific variations.
Population-based administrative data have great potential for facilitating investigations of OA occurrence as well as OA comorbidity and outcome research. However, the fundamental question to be addressed is whether the data are valid for such purposes. Our study addressed this question by comparing two case definitions with four reference standards. The next question to be addressed is which case definition should be applied for defining OA? It is noteworthy that the observed PPVs in both definitions were very high because the prevalence of OA was more than 70% based on the reference standards, whereas, in the general population, the prevalence of OA is 10–20%. The sensitivity of the definition that included one physician’s claim or hospital admission was 47–57%, and the specificity was 75–87%. This suggests that potential overreporting should be a concern in estimating the general population prevalence using this definition. On the other hand, the sensitivity of the definition that included at least two physician’s claims in two years or one hospital admission was 21–26%, and the specificity was 91–100%. This suggests that prevalence would likely be underreported using the latter definition. In addition, the observed specificity and the PPV in the latter case definition were higher than those in the former case definition, thus producing fewer false positives cases. The definition of at least two physician’s claims in two years or one hospital admission would, therefore, be more appropriate for studies in which avoiding false positives is critical, such as etiological research or studies assessing the effect of OA on other health conditions in the population.
In conclusion, the validity of OA diagnoses in administrative health records in British Columbia varied due to case definitions and reference standards. AOA2 is more suitable for identifying OA cases for research using this Canadian administrative database. Despite several limitations, we have validated two administrative case definitions wherein clinical and symptomatic diagnoses of knee, hand, and hip OA were included in the reference standards. Future validation studies, based on clinical diagnoses of all possible joints affected by OA, are needed. As the validation results may differ across administrative regions, further studies in different populations are needed to compare these results.
An earlier version of this work was presented as an abstract at OARSI (Osteoarthritis Research Society International) Annual Scientific Meeting in 2008.
The authors declare that they have no competing interests.
Dr. M. Mushfiqur Rahman acknowledged the Canadian Arthritis Network, The Arthritis Society, and the Canadian Institutes of Health Research for doctoral training awards. Dr. Jolanda Cibere acknowledged the Canadian Institutes of Health Research and The Arthritis Society for receiving grants for the population-based cohort study and for the Administrative Linkage Study.