Evaluation of an Automated Screening Assay, Compared to Indirect Immunofluorescence, an Extractable Nuclear Antigen Assay, and a Line Immunoassay in a Large Cohort of Asian Patients with Antinuclear Antibody-Associated Rheumatoid Diseases: A Multicenter Retrospective Study

We assessed the diagnostic utility of the connective tissue disease (CTD) screen as an automated screening test, in comparison with the indirect immunofluorescence (IIF), EliA extractable nuclear antigen (ENA), and line immunoassay (LIA) for patients with antinuclear antibody- (ANA-) associated rheumatoid disease (AARD). A total of 1115 serum samples from two university hospitals were assayed using these four autoantibody-based methods. The AARD group consisted of patients with systemic lupus erythematosus (SLE), systemic sclerosis (SSc), Sjögren's syndrome (SS), and mixed connective tissue disease (MCTD). The qualitative results of all four autoantibody assays showed a significant association with AARDs, compared to controls (P < 0.0001 for all). The areas under the receiver operating characteristic curves (ROC-AUCs) of the CTD screen for differentiating total AARDs, SLE, SSc, SS, and MCTD from controls were 0.89, 0.93, 0.73, 0.93, and 0.95, respectively. The ROC-AUCs of combination testing with LIA were slightly higher in patients with AARDs (0.92) than those of CTD screen alone. Multivariate analysis indicated that all four autoantibody assays could independently predict AARDs. CTD screening alone and in combination with IIF, EliA ENA, and LIA are potentially valuable diagnostic approaches for predicting AARDs. Combining CTD screen with LIA might be effective for AARD patients.


Introduction
Autoantibodies are closely related to clinical manifestations or the prognosis of patients with antinuclear antibody-(ANA-) associated rheumatoid diseases (AARDs), including systemic lupus erythematosus (SLE), systemic sclerosis (SSc), Sjögren's syndrome (SS), and mixed connective tissue disease (MCTD), who generally suffer from diffuse organ damage [1,2]. Antinuclear antibodies (ANAs), a kind of autoantibody, are directed against a variety of nuclear antigens. The detection of ANAs is useful for diagnosing patients with AARDs [3,4]. Indirect immunofluorescence (IIF) assays with cultured human epithelial carcinoma cells (HEp-2 cells) have been regarded as a gold standard method [5]. However, IIF is a labor-intensive and time-consuming procedure and exhibits poor reproducibility due to the subjective interpretation of results [4,6].
Enzyme immunoassays (EIAs) have been developed as alternatives to IIF for ANA screening and are widely used in clinical laboratories, enabling automation and quantitation of ANA screening [7]. The connective tissue disease (CTD) screen (Phadia AB, Uppsala, Sweden) used in this study is a recently introduced EIA-based assay employing 17 different human recombinant antigens. After the initial screen for ANAs, autoantibodies to extractable nuclear antigen (ENA) are frequently detected because of their diagnostic and prognostic significance. Identification of anti-ENA antibodies plays a critical role in the diagnosis and management of AARD [1,8]. EliA ENA assays (Phadia AB) for detecting autoantibodies to dsDNA, U1RNP, Sm, Ro/SSA, La/SSB, Scl-70, Pm-scl, Jo-1, and CENP have been introduced in the form of several different EIA kits, and line immunoassays (LIAs) have been widely applied for confirmatory testing [9]. Little information is available regarding evaluation of the performance of these autoantibody assays simultaneously for autoantibodies and consequent antibody-disease associations. Further, most previous population studies involved patients in Europe or the USA.
In this study, we evaluated the current diagnostic performance of an automated CTD screening assay in patients with AARDs. The diagnostic utility of the assay was compared with that of the HEp-2 cell-based IIF, EliA ENA, and LIA tests in a large Asian population. We also investigated the diagnostic performance of the CTD screen in combination with the other three autoantibody assays for each AARD.

Study Design.
A total of 1115 sera from patients who visited two university hospitals in Korea for AARD evaluation were collected to demonstrate the diagnostic performance of the CTD screen (Phadia AB, Uppsala, Sweden), as well as IIF (Fluoro HEPANA test, MBL Co., Nagoya, Japan), EliA ENA (Phadia AB), and LIA (Euroimmun AG, Lübeck, Germany) testing. The samples were collected randomly, and results from the same patients were not included repeatedly. This study was approved by the independent Institutional Review Board of Severance Hospital and Kosin University Gospel Hospital. Because residual serum samples were obtained from patients during routine screening for the detection of autoantibodies in our clinical laboratory, this study was exempted from the requirement for informed patient consent. The specimens were retrospectively classified according to predefined diagnoses as follows: total AARD (n = 112), SLE (n = 67), SSc (n = 21), SS (n = 19), MCTD (n = 5), and control (n = 1003). The total AARD value was derived from the number of patients with SLE, SSc, SS, or MCTD. The controls were consecutive patients who consulted the rheumatology clinics and for whom the rheumatologists considered it necessary to request ANA testing. After work-ups, these patients were diagnosed not to have AARDs.
All patients were diagnosed by specialized rheumatologists in the clinics of Severance Hospital and Kosin University Gospel Hospital, based on the criteria of the American College of Rheumatology and Systemic Lupus International Collaborating Clinics [10,11] for SLE, the American College of Rheumatology/European League [12] for SSc, the American-European Consensus Classification [13] for SS, and Alarcon-Segovia and Cardiel [14] for MCTD. The control group consisted of patients with several nonsystemic rheumatic diseases, considering actual clinical laboratory status.
2.2. IIF for ANAs. IIF microscopy on HEp-2 cells for ANAs was performed using a commercially available Fluoro HEPANA Kit (MBL Co.), according to the manufacturer's instructions. The serum samples were diluted with phosphate-buffered saline to a 1 : 40 ratio. The procedures for IIF were conducted as described previously [15].

CTD Screen.
Testing with the CTD screen (Phadia AB) was conducted on a Phadia 250 instrument (Phadia AB). The assays were conducted according to the manufacturer's instructions, as described before [15,16]. The previously applied cutoffs were also used in this study; cutoffs greater than 1.0 were considered positive, ratios ranging from 0.7 to 1.0 were equivocal, and values less than 0.7 were negative.
2.6. Statistical Analysis. Statistical analyses were performed using PASW software, version 24.0 (formerly, SPSS Statistics) (SPSS Inc., Chicago, IL, USA) and Analyse-it Method Evaluation Edition software, version 2.26 (Analyse-it Software Ltd., Leeds, UK). Comparisons using chi-squared test and the Mann-Whitney U test were performed as described before [15]. Cohen's kappa coefficients were calculated to estimate the agreement among the results of the CTD screen and the other three autoantibody assays, and McNemar's test was also conducted. Receiver operating characteristic (ROC) curves were plotted for data from the CTD screen alone or in combination with the other assays, in order to assess their diagnostic abilities to distinguish between the AARD groups and the control group. The areas under the ROC curves (ROC-AUCs) for the CTD screen and its combination with the other three assays were compared. Binary logistic regression analysis was performed to estimate the ROC-AUCs for IIF and CTD screen combinations, similar to the previously described method [15,18]. Multivariate logistic regression analysis was performed by setting the presence of an AARD as the dependent variable and age, sex, and the results of IIF, CTD screen, EliA ENA, and LIA testing as covariables. P values less than 0.05 were considered statistically significant.

Study Population Characteristics.
The basic characteristics of the study population are presented in Table 1. The median ages of patients in the total AARD and control groups were 39.0 and 50.0 years, respectively (P < 0 0001). The proportion of female patients between the total AARD and control group was not significant (97.3% versus 99.2%, P = 0 0561).

3.2.
Comparison of the CTD Screen, IIF, EliA ENA, and LIA Results according to Predefined AARD Criteria. The qualitative results of the CTD screen, IIF, EliA ENA, and LIA tests in patients with predefined AARDs and control subjects are summarized in Table 1. The results from all four assays were significantly associated with the total AARD group, compared to the control group (P < 0 0001 for all) ( Table 1).

3.3.
Overall Agreement among the CTD Screen, IIF, EliA ENA, and LIA Test Results according to the AARD Groups. An overall comparison of the results, including the concordance rates, kappa coefficient, and P values of McNemar testing, among the four autoantibody assays is presented in Table 2. 3.3.1. Concordance Rates. When the CTD screen ratios were considered as qualitative results with a cutoff of 1.0 (equivocal ratios scored as negative results), as suggested by the manufacturer, the overall concordance rates were 80.4% for the CTD screen versus IIF, 89.3% versus EliA ENA, and 79.5% versus LIA in the total AARD group. In addition, the overall concordance rates were 78.6% for the CTD screen with a cutoff of 0.7 (equivocal ratios scored as positive results) versus IIF, 89.3% versus EliA ENA, and 81.3% versus LIA.

Kappa
Coefficients. The kappa coefficients between the CTD screen and EliA ENA were good (0.70), while those between the CTD screen and IIF were moderate (0.47 for cutoff 1.0 and 0.41 for cutoff 0.7), and those between the CTD screen and LIA were fair (0.29 for cutoff 1.0 and 0.32 for cutoff 0.7).

McNemar Test.
The P values of McNemar tests showed no statistically significant differences between the CTD screen and LIA (0.6776 for cutoff 1.0 and 1.0000 for cutoff 0.7), whereas the differences between the CTD screen versus IIF (0.0169 for cutoff 1.0 and 0.0066 for cut-off 0.7) and CTD screen versus EliA ENA (0.0063 for cutoff 1.0 and 0.0005 for cutoff 0.7) were significant in the total AARD group.
3.4. Diagnostic Performances of the CTD Screen, IIF, EliA ENA, and LIA Tests. The calculated ROC-AUCs, sensitivities, specificities at the best cutoffs, likelihood ratios, and odds ratios are shown in Table 3.
3.4.1. ROC-AUCs. The best cutoff was determined when the sum of sensitivity and specificity was maximized. When these cut-off values were applied, the sensitivities of the CTD screen and its combination with IIF, EliA ENA, and LIA were 81.3%, 85.7%, 83.0%, and 92.9%, respectively, and the specificities were 92.1%, 85.2%, 89.6%, and 83.7%, respectively, in the total AARD group. The ROC-AUCs of the CTD screen and its combination with IIF, EliA ENA, and LIA were over 0.90 for all subgroups, indicating excellent diagnostic performances, except for the ROC-AUCs in the SSc group, which showed fair performance ( Figure 1). The differences between the ROC curves of the CTD screen and its combinations when differentiating AARDs from the control group were also analyzed. The ROC-AUCs for CTD screen and its combination with LIA were 0.89 and 0.92, respectively, demonstrating a significant difference (P = 0 0267) in the total AARD group, while the other combinations in the total AARD group, as well as in the SLE, SSc, SS, and MCTD groups, did not show significant differences.

Likelihood Ratio and Odds
Ratio. The positive and negative likelihood ratios of the CTD screen and its combination with IIF, EliA ENA, and LIA in the total AARD, SLE, SS, and MCTD groups were considered very useful (positive likelihood ratio > 5.0 or ≤0.2 for all groups). The positive likelihood ratios for the CTD screen and its combination with IIF, EliA ENA, and LIA in the SSc group were also very useful, while the negative likelihood ratios among them ranged from 0.3 to 0.4, indicating that they were just useful, based on the criteria of Solomon et al. [4]. Additionally, the odds ratios of CTD screen with LIA (67.0) were marginally higher than those of CTD screen alone (50.7) in the total AARD group.
3.5. Multivariate Analysis of CTD Screen, IIF, EliA ENA, and LIA Test Results. Multivariate analysis was performed using the presence of an AARD as the binary dependent variable and age and sex of the patient and the CTD screen, IIF, EliA ENA, and LIA results as predictors. We found that age and the results from CTD screen, IIF, EliA ENA, and LIA testing were independently associated with total AARD (Table 4). Our results showed that age, CTD screen, IIF, and LIA results for SLE; age, sex, IIF, and LIA results for SSc; and EliA ENA and LIA results for SS independently correlated with the respective rheumatic diseases. Sex was not a predictor in the SS and MCTD groups in the multivariate analysis because all subjects were female.

Discussion
Diagnostic applications of the CTD screen was assessed in Korean patients with AARDs, retrospectively. The performance of an automated screening assay was compared to HEp-2 cell-based IIF, EliA ENA, and LIA methods. In addition, diagnostic values of combining the CTD screen and these 3 autoantibody assays were investigated for each AARD.
The proportions of CTD screen, IIF, EliA ENA, and LIA positivities were higher in patients diagnosed with AARDs than in the control group, indicating that these four assays would be useful for detecting the patients with AARDs. Previous reports have shown similar results with these assays [19][20][21][22]. The sensitivities and specificities of these four assays would be affected because the classification criteria of each AARD were applied to our study as a standard. Regarding the sensitivity of the HEp-2 IIF, the values in this study were considerably low when compared to those in the previous reports [23,24]. Meanwhile, there are several studies with decreased sensitivities of HEp-2 IIF such as 69.0%, 36.4%, and 53.7% [16,22,25], even lower than those of our study. Although our laboratory obtained accreditation, tested kits, condition of microscopes, interobserver variations, and low number of the patient population may have an influence on the values, similar to these studies with lowered sensitivities.
The negative results in the AARD groups were caused by earlier clinical observations and subsequent development of positivity in follow-up or a negative conversion after initial positivity at presentation because of immunosuppressive treatment, based on a review of each medical chart. Clinical and serological presentations can fluctuate over time in AARD patients, according to the results of several studies [19,26].
Regarding the overall concordance rates between the CTD screen and IIF test, the agreements ranged from 87% to 95% when comparing the EIA assays and IIF in a previous study [27]. Previously, Cohen's kappa value observed when combining EIA results of ENA and IIF tests was reported to be 0.30 [19], which was the same as that found with combined EliA ENA and IIF testing (0.30) and lower than that of the CTD screen and the IIF test (0.70), in this study. The previous report showed that the EIA for ENA and IIF results exhibited fair agreement for patients with SLE and that the kappa agreement for patients with scleroderma was good. Our results also demonstrated that the kappa value between EliA ENA and IIF of the SSc group (0.45) was higher than that of the SLE group (0.34). The kappa values between the CTD screen and IIF test were higher than those of EliA ENA and IIF in all subgroups, indicating that the CTD screen was more appropriate for screening with more diverse antigens than the EliA ENA test.
With respect to the LIA results, the kappa coefficients compared to IIF were similar to the previous results in the total AARD group (0.45 versus 0.404), as well as the SLE (0.28 versus 0.298) and SSc (0.48 versus 0.573) groups [28]. These agreements between the LIA and IIF methods were also similar to those between the LIA test and CTD screen, reflecting the characteristics of CTD screen and IIF as screening assays. There were slight differences between the different study populations and the cutoff used for each method. A detailed description of the agreements found between these tests (including concordance rates, Cohen's kappa values, and P values for McNemar tests) among the widely used autoantibody assays has been scarcely reported in the literature with large groups.
The diagnostic abilities of the CTD screen to discriminate each AARD subgroup from the control group were analyzed because the clinical utility of the CTD screen     can vary with the characteristics of AARD patients and nonrheumatic conditions. The portion of SSc patients in the total AARD group may explain the lower CTD screen performance in the total AARD group, based on the ROC-AUCs. A previous study was conducted to evaluate automated ENA screening assay with 9 antigens and showed an ROC-AUC of 0.823, sensitivity of 63%, and specificity of 91% for AARD patients [22], which were slightly lower than the values obtained in our study. Another previous report showed that the sensitivities of the CTD screen for SLE, SSc, SS, and MCTD groups were 74%, 72%, 89%, and 100%, respectively [20]. Although the sensitivities reported for previous studies and those of our study were slightly different, the highest sensitivity for the MCTD group and lowest sensitivity for the SSc group [20] were concordant to our study. With respect to the cutoffs, we described the CTD screen results at 0.7 and 1.0 because these values are suggested by the manufacturer's instructions. Based on the best cutoff of ROC-AUCs in our study, 1.0 of CTD screen is closer to it than 0.7 for total AARD, SLE, SSc, and MCTD and vice versa for SS. Therefore, 1.0 would be the best cutoff for patients with AARD, including SLE, SSc, and MCTD while 0.7 would be more appropriate for patients with SS. After verification, applying the optimal cutoff for each clinical laboratory is recommended.
Regarding clinical implications based on our data, CTD screen was comparable to IIF or the other assays because the 95% confidence intervals (CIs) of sensitivities and ROC-AUCs were overlapped with those in the others, indicating no statistical significance. Further, CTD screen exhibited slightly better specificity than the other assays according to the results of Tables 1 and 3, raising the possibility of an alternative screening assay. However, sufficient review of previous literatures for ANA testing [5,29] is necessary when these assays are adopted to individual clinical laboratories. A variety of confounding factors such as patient populations, controls, applied methods, and cutoffs might affect the analyzed results.
The positive likelihood ratios of the CTD screen were very useful and conclusive, with respect to pretest and posttest probabilities [4,30]. Previously published data also demonstrated positive results for the CTD screen at a cutoff of 1.0, with positive likelihood ratios greater than 5.0 in the SLE, SSc, SS, and MCTD groups [20]. The negative likelihood ratios for the CTD screen were less than 0.2 with the exception of SSc patients. Op De Beeck et al. [20] showed a similar pattern for the CTD screen in patients with SLE (0.3), SSc (0.3), and SS (0.1), which was considered to be very useful or useful tests, based on the negative likelihood ratios [4].
Based on the values of ROC-AUCs and odds ratios, concurrent determination with the CTD screen and LIA test demonstrated slightly improved diagnostic utilities in patients in the total AARD group, compared to the CTD screen alone. According to a previous report, the diagnostic performance of the CTD screen was significantly improved when used in combination with IIF, based on the likelihood ratios and ROC-AUCs [31].
The study population was patients of the university hospital in Belgium, and the results were slightly varied according to the diagnosis, suggesting different strategies for each AARDs. Therefore, uniformed and simultaneous screening by the CTD screen and IIF [29] would not be effective in cases with strong clinical manifestations associated with each AARD. Further, investigating laboratory findings with respect to other immunologic disorders might be essential for diagnosing each AARD when the screening assay shows positive results.
Multivariate analysis was also conducted for age and sex to assess CTD screen, IIF, EliA ENA, and LIA results as predictor variables for AARD groups and to control for potential confounding factors. ANAs were previously found in healthy individuals, particularly women older than 40 years [32]. Therefore, the factors of age and sex were incorporated into our multivariate model. The multivariate analysis results revealed that CTD screen, IIF, EliA ENA, and LIA results were independently related to the total AARD group, indicating that these four autoantibody assays could be applied for diagnosing AARDs. However, not all assays showed significant P values in the subgroups. Based on our multivariate analysis, a combination with LIA for diagnostic work-ups of patients with SSc or SS would be necessary, when the CTD screen was applied to the clinical laboratories.
The profiles for the individual autoantibodies in the EliA ENA and LIA tests were generally concordant with previous reports [17,28]. The high specificities of EliA ENA and LIA for individual autoantibodies (over 95.0%) were similar to those of a previous study, which showed at least 97% specificities [17]. These assays, especially the LIA, have been used as confirmatory tests, based on their high specificities [9]. The relatively low specificities of anti-Ro/SSA and Scl-70, which are related to SS and SSc, respectively [17,24], might contribute to the lowered specificities of LIA.
Our study design has some limitations in that most of our study population was composed of females, especially the SS and MCTD groups. Among the hospitalized SLE patients, 165 out of 180 (91.8%) were female, based on the information in the Korean lupus network registry [33]. Although our data reflected a predominant portion of female patients, further studies involving male patients should be followed. Moreover, the number of patient population was small, especially MCTD, because of low prevalence. Further studies involving large study populations are necessary, considering the statistical power, interpretation, and application of the results to AARD patients.

Conclusions
In conclusion, we evaluated the diagnostic utilities of CTD screen, IIF, EliA ENA, and LIA testing to predict AARDs. Although a few published reports have discussed the usefulness of the CTD screen alone, no report has assessed a large population of Korean patients in Asia or the diagnostic performance of these frequently used autoantibody assays for AARDs in current use.
CTD screening and the other three assays are potentially valuable diagnostic tools for detecting AARDs. In particular, the CTD screen showed considerable diagnostic performance for patients with SLE rather than SSc, and its combination with the LIA test might be more effective for diagnosing total AARDs than the CTD screen alone. Our results provided recent information regarding the CTD screen, IIF, EliA ENA, and LIA methods for testing patients with AARDs to facilitate appropriate patient management, with cautious clinical impressions of patients' diagnosis.

Conflicts of Interest
The authors declare that there is no conflict of interest regarding the publication of this article.   Figure 1: Diagnostic performance of the CTD screen, tested independently and in combination with autoantibody assays. (a) Receiver operating characteristic (ROC) curves of the CTD screen (0.89) and its combination with autoantibody assays for discriminating total antinuclear antibody-associated rheumatoid disease patients (n = 112) from control subjects (n = 1003); (b) ROC curves for the CTD screen (0.93) and its combination with autoantibody assays, differentiating patients with systemic lupus erythematosus (SLE) (n = 67) from control subjects; (c) ROC curves for the CTD screen (0.73) combined with autoantibody assays, differentiating patients with systemic sclerosis (SSc) (n = 21) from control subjects; (d) ROC curves for the CTD screen (0.93) and its combination with autoantibody assays, discriminating patients with Sjögren's syndrome (SS) (n = 19) from control subjects; (e) ROC curves for the CTD screen (0.95) and its combination with autoantibody assays, differentiating patients with mixed connective tissue disease (MCTD) (n = 5) from control subjects.