Journal of Immunology Research
Volume 2018 (2018), Article ID 9094217, 12 pages
https://doi.org/10.1155/2018/9094217

Research Article

Evaluation of an Automated Screening Assay, Compared to Indirect Immunofluorescence, an Extractable Nuclear Antigen Assay, and a Line Immunoassay in a Large Cohort of Asian Patients with Antinuclear Antibody-Associated Rheumatoid Diseases: A Multicenter Retrospective Study

Seri Jeong,¹ Hyunyong Hwang,¹ Juhye Roh,² Jung Eun Shim,² Jinmi Kim,³ Geun-Tae Kim,⁴ Hee-Sang Tag,⁴ and Hyon-Suk Kim²

¹Department of Laboratory Medicine, Kosin University College of Medicine, 262 Gamcheon-ro, Seo-gu, Busan 49267, Republic of Korea
²Department of Laboratory Medicine, Severance Hospital, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea
³Department of Statistics, Pusan National University Hospital, 179 Gudeok-ro, Seo-gu, Busan 49241, Republic of Korea
⁴Department of Rheumatology, Kosin University College of Medicine, 262 Gamcheon-ro, Seo-gu, Busan 49267, Republic of Korea

Correspondence should be addressed to Hyon-Suk Kim; kimhs54@yuhs.ac

Received 7 September 2017; Revised 15 January 2018; Accepted 12 February 2018; Published 2 May 2018

Academic Editor: Jacek Tabarkiewicz

Copyright © 2018 Seri Jeong et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

We assessed the diagnostic utility of the connective tissue disease (CTD) screen as an automated screening test, in comparison with the indirect immunofluorescence (IIF), EliA extractable nuclear antigen (ENA), and line immunoassay (LIA) for patients with antinuclear antibody- (ANA-) associated rheumatoid disease (AARD). A total of 1115 serum samples from two university hospitals were assayed using these four autoantibody-based methods. The AARD group consisted of patients with systemic lupus erythematosus (SLE), systemic sclerosis (SSc), Sjögren’s syndrome (SS), and mixed connective tissue disease (MCTD). The qualitative results of all four autoantibody assays showed a significant association with AARDs, compared to controls ( for all). The areas under the receiver operating characteristic curves (ROC-AUCs) of the CTD screen for differentiating total AARDs, SLE, SSc, SS, and MCTD from controls were 0.89, 0.93, 0.73, 0.93, and 0.95, respectively. The ROC-AUCs of combination testing with LIA were slightly higher in patients with AARDs (0.92) than those of CTD screen alone. Multivariate analysis indicated that all four autoantibody assays could independently predict AARDs. CTD screening alone and in combination with IIF, EliA ENA, and LIA are potentially valuable diagnostic approaches for predicting AARDs. Combining CTD screen with LIA might be effective for AARD patients.

1. Introduction

Autoantibodies are closely related to clinical manifestations or the prognosis of patients with antinuclear antibody- (ANA-) associated rheumatoid diseases (AARDs), including systemic lupus erythematosus (SLE), systemic sclerosis (SSc), Sjögren’s syndrome (SS), and mixed connective tissue disease (MCTD), who generally suffer from diffuse organ damage [1, 2]. Antinuclear antibodies (ANAs), a kind of autoantibody, are directed against a variety of nuclear antigens. The detection of ANAs is useful for diagnosing patients with AARDs [3, 4]. Indirect immunofluorescence (IIF) assays with cultured human epithelial carcinoma cells (HEp-2 cells) have been regarded as a gold standard method [5]. However, IIF is a labor-intensive and time-consuming procedure and exhibits poor reproducibility due to the subjective interpretation of results [4, 6].

Enzyme immunoassays (EIAs) have been developed as alternatives to IIF for ANA screening and are widely used in clinical laboratories, enabling automation and quantitation of ANA screening [7]. The connective tissue disease (CTD) screen (Phadia AB, Uppsala, Sweden) used in this study is a recently introduced EIA-based assay employing 17 different human recombinant antigens. After the initial screen for ANAs, autoantibodies to extractable nuclear antigen (ENA) are frequently detected because of their diagnostic and prognostic significance. Identification of anti-ENA antibodies plays a critical role in the diagnosis and management of AARD [1, 8]. EliA ENA assays (Phadia AB) for detecting autoantibodies to dsDNA, U1RNP, Sm, Ro/SSA, La/SSB, Scl-70, Pm-scl, Jo-1, and CENP have been introduced in the form of several different EIA kits, and line immunoassays (LIAs) have been widely applied for confirmatory testing [9]. Little information is available regarding evaluation of the performance of these autoantibody assays simultaneously for autoantibodies and consequent antibody-disease associations. Further, most previous population studies involved patients in Europe or the USA.

In this study, we evaluated the current diagnostic performance of an automated CTD screening assay in patients with AARDs. The diagnostic utility of the assay was compared with that of the HEp-2 cell-based IIF, EliA ENA, and LIA tests in a large Asian population. We also investigated the diagnostic performance of the CTD screen in combination with the other three autoantibody assays for each AARD.

2. Materials and Methods

2.1. Study Design

A total of 1115 sera from patients who visited two university hospitals in Korea for AARD evaluation were collected to demonstrate the diagnostic performance of the CTD screen (Phadia AB, Uppsala, Sweden), as well as IIF (Fluoro HEPANA test, MBL Co., Nagoya, Japan), EliA ENA (Phadia AB), and LIA (Euroimmun AG, Lübeck, Germany) testing. The samples were collected randomly, and results from the same patients were not included repeatedly. This study was approved by the independent Institutional Review Board of Severance Hospital and Kosin University Gospel Hospital. Because residual serum samples were obtained from patients during routine screening for the detection of autoantibodies in our clinical laboratory, this study was exempted from the requirement for informed patient consent. The specimens were retrospectively classified according to predefined diagnoses as follows: total AARD (), SLE (), SSc (), SS (), MCTD (), and control (). The total AARD value was derived from the number of patients with SLE, SSc, SS, or MCTD. The controls were consecutive patients who consulted the rheumatology clinics and for whom the rheumatologists considered it necessary to request ANA testing. After work-ups, these patients were diagnosed not to have AARDs.

All patients were diagnosed by specialized rheumatologists in the clinics of Severance Hospital and Kosin University Gospel Hospital, based on the criteria of the American College of Rheumatology and Systemic Lupus International Collaborating Clinics [10, 11] for SLE, the American College of Rheumatology/European League [12] for SSc, the American-European Consensus Classification [13] for SS, and Alarcon-Segovia and Cardiel [14] for MCTD. The control group consisted of patients with several nonsystemic rheumatic diseases, considering actual clinical laboratory status.

2.2. IIF for ANAs

IIF microscopy on HEp-2 cells for ANAs was performed using a commercially available Fluoro HEPANA Kit (MBL Co.), according to the manufacturer’s instructions. The serum samples were diluted with phosphate-buffered saline to a 1 : 40 ratio. The procedures for IIF were conducted as described previously [15].

2.3. CTD Screen

Testing with the CTD screen (Phadia AB) was conducted on a Phadia 250 instrument (Phadia AB). The assays were conducted according to the manufacturer’s instructions, as described before [15, 16]. The previously applied cutoffs were also used in this study; cutoffs greater than 1.0 were considered positive, ratios ranging from 0.7 to 1.0 were equivocal, and values less than 0.7 were negative.

2.4. EliA ENA

Nine kinds of EliA ENA assays for detecting autoantibodies against dsDNA, U1RNP, Sm, Ro/SSA, La/SSB, Scl-70, Pm-scl, Jo-1, and CENP were performed using the respective EliA ENA kits (Phadia AB) on a Phadia 250 instrument (Phadia AB). All procedures were conducted as indicated in the manufacturer’s instructions. The procedures followed were similar to those described above for the CTD screen. The cutoff value for dsDNA was 15 IU/ml, and that for U1RNP, Sm, Ro/SSA, La/SSB, Scl-70, Pm-scl, Jo-1, and CENP was 10 U/ml.

2.5. LIA

The EUROLINE test (Euroimmun AG) enabled detection of 14 autoantibodies against RNP, Sm, Ro/SSA60, Ro/SSA52, La/SSB, Scl-70, Pm-Scl, Jo-1, CENP, PCNA, nucleosomes, histones, ribosomal-P, and AMA-M2. These antigens were detected as discrete lines on a nylon membrane with a plastic backing. These strips were incubated sequentially with sera, alkaline phosphatase-labeled anti-human IgG antibodies, and substrate solution (containing nitroblue tetrazolium chloride and 5-bromo-4-chloro-3-indolyl phosphate), according to the manufacturer’s instructions. Reaction intensities were automatically interpreted using Eurolinescan software (Euroimmun AG), similar to the procedures described in a previous report [17].

2.6. Statistical Analysis

Statistical analyses were performed using PASW software, version 24.0 (formerly, SPSS Statistics) (SPSS Inc., Chicago, IL, USA) and Analyse-it Method Evaluation Edition software, version 2.26 (Analyse-it Software Ltd., Leeds, UK). Comparisons using chi-squared test and the Mann–Whitney test were performed as described before [15]. Cohen’s kappa coefficients were calculated to estimate the agreement among the results of the CTD screen and the other three autoantibody assays, and McNemar’s test was also conducted. Receiver operating characteristic (ROC) curves were plotted for data from the CTD screen alone or in combination with the other assays, in order to assess their diagnostic abilities to distinguish between the AARD groups and the control group. The areas under the ROC curves (ROC-AUCs) for the CTD screen and its combination with the other three assays were compared. Binary logistic regression analysis was performed to estimate the ROC-AUCs for IIF and CTD screen combinations, similar to the previously described method [15, 18]. Multivariate logistic regression analysis was performed by setting the presence of an AARD as the dependent variable and age, sex, and the results of IIF, CTD screen, EliA ENA, and LIA testing as covariables. values less than 0.05 were considered statistically significant.

3. Results

3.1. Study Population Characteristics

The basic characteristics of the study population are presented in Table 1. The median ages of patients in the total AARD and control groups were 39.0 and 50.0 years, respectively (). The proportion of female patients between the total AARD and control group was not significant (97.3% versus 99.2%, ).

Table 1: Demographic characteristics of the study population and the qualitative values for four autoantibody assays.

Variables
Total AARD ()
SLE ()
SSc ()
SS ()
MCTD ()
Control ()
value

Age (years)
39.0 (23.0–51.0)
37.0 (17.2–44.8)
39.0 (26.7–49.7)
54.0 (42.0–62.0)
43.0 (35.0–47.3)
50.0 (37.0–61.0)
<0.0001

Number of females, %
109, 97.3
66, 98.5
19, 90.5
19, 100.0
5, 100.0
995, 99.2
0.0561

CTD screen, cutoff ratio 0.7 (number), %
Positive
93, 83.0
58, 86.6
13, 61.9
17, 89.5
5, 100.0
100, 10.0
<0.0001

Negative
19, 17.0
9, 13.4
8, 38.1
2, 10.5
0, 0.0
903, 90.0

CTD screen, cutoff ratio 1.0 (number), %
Positive
91, 81.3
57, 85.1
13, 61.9
16, 84.2
5, 100.0
79, 7.9
<0.0001

Negative
21, 18.7
10, 14.9
8, 38.1
3, 15.8
0, 0.0.
924, 92.1

HEp-2 IIF (number), %
Positive
79, 70.5
51, 76.1
13, 61.9
11, 57.9
4, 80.0
99, 9.9
<0.0001

Negative
33, 29.5
16, 23.9
8, 38.1
8, 42.1
1, 20.0
904, 90.1

EliA ENA (number), %
Positive
81, 72.3
51, 76.1
9, 42.9
17, 89.5
4, 80.0
93, 9.3
<0.0001

Negative
31, 27.7
16, 23.9
12, 57.1
2, 10.5
1, 20.0
910, 90.7

LIA (number), %
Positive
94, 83.9
58, 86.6
14, 66.7
18, 94.7
4, 80.0
141, 14.1
<0.0001

Negative
18, 16.1
9, 13.4
7, 33.3
1, 5.3
1, 20.0
862, 85.9

AARD: antinuclear antibody-associated rheumatoid disease; SLE: systemic lupus erythematosus; SSc: systemic sclerosis; SS: Sjögren’s syndrome; MCTD: mixed connective tissue disease; IIF: indirect immunofluorescence; ENA: extractable nuclear antigen; LIA: line immunoassay. Chi-squared test for nominal variables and Mann–Whitney test for continuous variables in the total AARD group versus control. Data are expressed as the median (1st to 3rd quartiles).

3.2. Comparison of the CTD Screen, IIF, EliA ENA, and LIA Results according to Predefined AARD Criteria

The qualitative results of the CTD screen, IIF, EliA ENA, and LIA tests in patients with predefined AARDs and control subjects are summarized in Table 1. The results from all four assays were significantly associated with the total AARD group, compared to the control group ( for all) (Table 1).

3.3. Overall Agreement among the CTD Screen, IIF, EliA ENA, and LIA Test Results according to the AARD Groups

An overall comparison of the results, including the concordance rates, kappa coefficient, and values of McNemar testing, among the four autoantibody assays is presented in Table 2.

Table 2: Overall agreements among the four autoantibody assays, according to the AARD groups.

Group ()
Assays
CTD screen, cutoff ratio 0.7
CTD screen, cutoff ratio 1.0
HEp-2 IIF
EliA ENA
LIA

Concordance (%)
Kappa
McNemar
Concordance (%)
Kappa
McNemar
Concordance (%)
Kappa
McNemar
Concordance (%)
Kappa
McNemar
Concordance (%)
Kappa
McNemar

Total AARD ()
CTD screen, cutoff ratio 0.7
—
—
—
98.2
0.94 (0.86–1.02)
0.5000
78.6
0.41 (0.22–0.60)
0.0066
89.3
0.70 (0.54–0.85)
0.0005
81.3
0.32 (0.09–0.55)
1.0000

CTD screen, cutoff ratio 1.0
98.2
0.94 (0.86–1.02)
0.5000
—
—
—
80.4
0.47 (0.29–0.65)
0.0169
89.3
0.70 (0.55–0.86)
0.0063
79.5
0.29 (0.07–0.51)
0.6776

HEp-2 IIF
78.6
0.41 (0.22–0.60)
0.0066
80.4
0.47 (0.29–0.65)
0.0169
—
—
—
71.4
0.30 (0.11–0.49)
0.8601
74.1
0.28 (0.09–0.47)
0.0081

EliA ENA
89.3
0.70 (0.54–0.85)
0.0005
89.3
0.70 (0.55–0.86)
0.0063
71.4
0.30 (0.11–0.49)
0.8601
—
—
—
75.9
0.31 (0.11–0.50)
0.0192

LIA
81.3
0.32 (0.09–0.55)
1.0000
79.5
0.29 (0.07–0.51)
0.6776
74.1
0.28 (0.09–0.47)
0.0081
75.9
0.31 (0.11–0.50)
0.0192
—
—
—

SLE ()
CTD screen, cutoff ratio 0.7
—
—
—
98.5
0.94 (0.82–1.06)
1.0000
83.6
0.47 (0.21–0.73)
0.0654
89.6
0.66 (0.44–0.88)
0.0156
82.1
0.23 (−0.08 to 0.54)
1.0000

CTD screen, cutoff ratio 1.0
98.5
0.94 (0.82–1.06)
1.0000
—
—
—
85.1
0.53 (0.28–0.78)
0.1094
91.0
0.72 (0.51–0.92)
0.0313
80.6
0.20 (−0.10 to 0.50)
1.0000

HEp-2 IIF
83.6
0.47 (0.21–0.73)
0.0654
85.1
0.53 (0.28–0.78)
0.1094
—
—
—
76.1
0.34 (0.08–0.60)
1.0000
77.6
0.28 (0.01–0.54)
0.1185

EliA ENA
89.6
0.66 (0.44–0.88)
0.0156
91.0
0.72 (0.51–0.92)
0.0313
76.1
0.34 (0.08–0.60)
1.0000
—
—
—
77.6
0.28 (0.01–0.54)
0.1185

LIA
82.1
0.23 (−0.08 to 0.54)
1.0000
80.6
0.20 (−0.10 to 0.50)
1.0000
77.6
0.28 (0.01–0.54)
0.1185
77.6
0.28 (0.01–0.54)
0.1185
—
—
—

SSc ()
CTD screen, cutoff ratio 0.7
—
—
—
100.0
1.00 (1.00-1.00)
1.0000
81.0
0.60 (0.24–0.95)
1.0000
81.0
0.63 (0.33–0.93)
0.1250
76.2
0.48 (0.09–0.87)
1.0000

CTD screen, cutoff ratio 1.0
100.0
1.00 (1.00-1.00)
1.0000
—
—
—
81.0
0.60 (0.24–0.95)
1.0000
81.0
0.63 (0.33–0.93)
0.1250
76.2
0.48 (0.09–0.87)
1.0000

HEp-2 IIF
81.0
0.60 (0.24–0.95)
1.0000
81.0
0.60 (0.24–0.95)
1.0000
—
—
—
71.4
0.45 (0.10–0.80)
0.2188
76.2
0.48 (0.09–0.87)
1.0000

EliA ENA
81.0
0.63 (0.33–0.93)
0.1250
81.0
0.63 (0.33–0.93)
0.1250
71.4
0.45 (0.10–0.80)
0.2188
—
—
—
66.7
0.36 (0.02–0.71)
0.1250

LIA
76.2
0.48 (0.09–0.87)
1.0000
76.2
0.48 (0.09–0.87)
1.0000
76.2
0.48 (0.09–0.87)
1.0000
66.7
0.36 (0.02–0.71)
0.1250
—
—
—

SS ()
CTD screen, cutoff ratio 0.7
—
—
—
94.7
0.77 (0.35–1.20)
1.0000
57.9
0.04 (−0.28 to 0.36)
0.0703
100.0
1.00 (1.00-1.00)
1.0000
84.2
−0.08 (−0.18 to 0.03)
1.0000

CTD screen, cutoff ratio 1.0
94.7
0.77 (0.35–1.20)
1.0000
—
—
—
63.2
0.17 (−0.20 to 0.55)
0.1250
94.7
0.77 (0.35–1.20)
1.0000
78.9
−0.09 (−0.22 to 0.05)
0.6250

HEp-2 IIF
57.9
0.04 (−0.28 to 0.36)
0.0703
63.2
0.17 (−0.20 to 0.55)
0.1250
—
—
—
57.9
0.04 (−0.28 to 0.36)
0.0703
63.2
0.14 (−0.12 to 0.40)
0.0156

EliA ENA
100.0
1.00 (1.00-1.00)
1.0000
94.7
0.77 (0.35–1.20)
1.0000
57.9
0.04 (−0.28 to 0.36)
0.0703
—
—
—
84.2
−0.08 (−0.18 to 0.03)
1.0000

LIA
84.2
−0.08 (−0.18 to 0.03)
1.0000
78.9
−0.09 (−0.22 to 0.05)
0.6250
63.2
0.14 (−0.12 to 0.40)
0.0156
84.2
−0.08 (−0.18 to 0.03)
1.0000
—
—
—

MCT ()
CTD screen, cutoff ratio 0.7
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—

CTD screen, cutoff ratio 1.0
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—

HEp-2 IIF
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—

EliA ENA
—
—
—
—
—
—
60
−0.25 (−0.59 to 0.09)
1.0000
—
—
—
—
—
—

LIA
—
—
—
—
—
—
60
−0.25 (−0.59 to 0.09)
1.0000
60
−0.25 (−0.59 to 0.09)
1.0000
—
—
—

Total ()
CTD screen, cutoff ratio 0.7
—
—
—
97.9
0.92 (0.89–0.95)
<0.0001
85.6
0.48 (0.41–0.55)
0.2698
97.6
0.91 (0.88–0.94)
0.0003
85.7
0.54 (0.48–0.60)
0.0011

CTD screen, cutoff ratio 1.0
97.9
0.92 (0.89–0.95)
<0.0001
—
—
—
87.4
0.52 (0.45–0.59)
0.5543
96.4
0.86 (0.82–0.90)
—
86.5
0.55 (0.48–0.61)
<0.0001

HEp-2 IIF
85.6
0.48 (0.41–0.55)
0.2698
87.4
0.52 (0.45–0.59)
0.5543
—
—
—
85.3
0.45 (0.38–0.52)
0.8149
83.4
0.45 (0.39–0.52)
<0.0001

EliA ENA
97.6
0.91 (0.88–0.94)
0.0003
96.4
0.86 (0.82–0.90)
0.6358
85.3
0.45 (0.38–0.52)
0.8149
—
—
—
85.7
0.53 (0.46–0.59)
<0.0001

LIA
85.7
0.54 (0.48–0.60)
0.0011
86.5
0.55 (0.48–0.61)
<0.0001
83.4
0.45 (0.39–0.52)
<0.0001
85.7
0.53 (0.46–0.59)
<0.0001
—
—
—

IIF: indirect immunofluorescence; ENA: extractable nuclear antigen; LIA: line immunoassay; AARD: antinuclear antibody-associated rheumatoid disease; SLE: systemic lupus erythematosus; SSc: systemic sclerosis; SS: Sjögren’s syndrome; MCTD: mixed connective tissue disease; NS: not stated because of impractical calculation. Kappa coefficient between the assays are shown (95% confidence interval). values of the McNemar tests.

3.3.1. Concordance Rates

When the CTD screen ratios were considered as qualitative results with a cutoff of 1.0 (equivocal ratios scored as negative results), as suggested by the manufacturer, the overall concordance rates were 80.4% for the CTD screen versus IIF, 89.3% versus EliA ENA, and 79.5% versus LIA in the total AARD group. In addition, the overall concordance rates were 78.6% for the CTD screen with a cutoff of 0.7 (equivocal ratios scored as positive results) versus IIF, 89.3% versus EliA ENA, and 81.3% versus LIA.

3.3.2. Kappa Coefficients

The kappa coefficients between the CTD screen and EliA ENA were good (0.70), while those between the CTD screen and IIF were moderate (0.47 for cutoff 1.0 and 0.41 for cutoff 0.7), and those between the CTD screen and LIA were fair (0.29 for cutoff 1.0 and 0.32 for cutoff 0.7).

3.3.3. McNemar Test

The values of McNemar tests showed no statistically significant differences between the CTD screen and LIA (0.6776 for cutoff 1.0 and 1.0000 for cutoff 0.7), whereas the differences between the CTD screen versus IIF (0.0169 for cutoff 1.0 and 0.0066 for cut-off 0.7) and CTD screen versus EliA ENA (0.0063 for cutoff 1.0 and 0.0005 for cutoff 0.7) were significant in the total AARD group.

3.4. Diagnostic Performances of the CTD Screen, IIF, EliA ENA, and LIA Tests

The calculated ROC-AUCs, sensitivities, specificities at the best cutoffs, likelihood ratios, and odds ratios are shown in Table 3.

Table 3: ROC-AUC, sensitivity, and specificity of the CTD screen, tested independently and in combination.

Predefined diseases
Parameter
CTD screen
CTD screen + IIF
CTD screen + EliA ENA
CTD screen + LIA

Total AARD ()
ROC-AUC
0.89 (0.85–0.93)
0.89 (0.85–0.93)
0.89 (0.85–0.93)
0.92 (0.88–0.95)

value
<0.0001
<0.0001
<0.0001
<0.0001

Sensitivity (%)
81.3 (72.8–88.0)
85.7 (77.8–91.6)
83.0 (74.8–89.5)
92.9 (86.4–96.9)

Specificity (%)
92.1 (90.3–93.7)
85.2 (82.9–87.4)
89.6 (87.6–91.4)
83.7 (81.3–86.0)

+LR
10.3
5.8
8.0
5.7

−LR
0.2
0.2
0.2
0.1

Odds ratio
50.7 (29.9–85.9)
34.7 (19.9–60.5)
42.3 (24.8–72.1)
67.0 (32.0–140.2)

SLE ()
ROC-AUC
0.93 (0.88–0.97)
0.92 (0.88–0.97)
0.92 (0.88–0.96)
0.95 (0.91–0.98)

value
<0.0001
<0.0001
<0.0001
<0.0001

Sensitivity (%)
85.1 (74.3–92.6)
80.6 (69.1–89.2)
88.1 (77.8–94.7)
95.5 (87.5–99.1)

Specificity (%)
93.7 (92.0–95.1)
95.7 (94.3–96.9)
88.3 (86.2–90.3)
83.8 (81.4–86.1)

+LR
13.5
18.8
7.6
5.9

−LR
0.2
0.2
0.1
0.1

Odds ratio
85.0 (41.4–174.5)
92.7 (47.1–182.7)
55.8 (26.0–119.8)
110.7 (34.4–356.8)

SSc ()
ROC-AUC
0.73 (0.58–0.88)
0.75 (0.59–0.90)
0.72 (0.57–0.87)
0.76 (0.61–0.91)

value
0.0017
0.0011
0.0018
0.0003

Sensitivity (%)
61.9 (38.4–81.9)
71.4 (47.8–88.7)
61.9 (38.4–81.9)
76.2 (52.8–91.8)

Specificity (%)
92.5 (90.7–94.1)
87.2 (85.0–89.2)
90.6 (88.7–92.4)
85.2 (82.9–87.4)

+LR
8.3
5.6
6.6
5.2

−LR
0.4
0.3
0.4
0.3

Odds ratio
20.1 (8.1–50.0)
17.1 (6.5–44.8)
15.7 (6.4–38.9)
18.5 (6.7–51.2)

SS ()
ROC-AUC
0.93 (0.87–0.99)
0.93 (0.88–0.98)
0.94 (0.88–0.99)
0.97 (0.95–0.99)

value
<0.0001
<0.0001
<0.0001
<0.0001

Sensitivity (%)
89.5 (66.9–98.7)
94.7 (74.0–99.9)
89.5 (66.9–98.7)
100.0 (82.4–100.0)

Specificity (%)
90.7 (88.8–92.5)
83.7 (81.3–86.0)
91.7 (89.8–93.4)
85.1 (82.8–87.3)

+LR
9.7
5.8
10.8
6.7

−LR
0.1
0.1
0.1
0.0

Odds ratio
83.2 (18.9–365.6)
92.8 (12.3–699.7)
94.2 (21.4–414.8)
NC

MCTD ()
ROC-AUC
0.95 (0.92–0.98)
0.95 (0.90–100.0)
0.94 (0.91–0.98)
0.94 (0.89–100.0)

value
<0.0001
<0.0001
<0.0001
<0.0001

Sensitivity (%)
100.0 (47.8–100.0)
100.0 (47.8–100.0)
100.0 (47.8–100.0)
100.0 (47.8–100.0)

Specificity (%)
92.1 (90.3–93.7)
85.2 (82.9–87.4)
90.5 (88.5–92.3)
83.5 (81.1–85.8)

+LR
12.7
6.8
10.6
6.1

−LR
0.0
0.0
0.0
0.0

Odds ratio
NC
NC
NC
NC

ROC-AUC: areas under the receiver operating characteristic curve; IIF: indirect immunofluorescence; ENA: extractable nuclear antigen; LIA: line immunoassay; LR: likelihood ratio; AARD: antinuclear antibody-associated rheumatoid disease; SLE: systemic lupus erythematosus; SSc: systemic sclerosis; SS: Sjögren’s syndrome; MCTD: mixed connective tissue disease; NC: not calculated. Data are shown with the 95% confidence interval in parentheses, and the values presented for the ROC-AUCs were greater than 0.5.

3.4.1. ROC-AUCs

The best cutoff was determined when the sum of sensitivity and specificity was maximized. When these cut-off values were applied, the sensitivities of the CTD screen and its combination with IIF, EliA ENA, and LIA were 81.3%, 85.7%, 83.0%, and 92.9%, respectively, and the specificities were 92.1%, 85.2%, 89.6%, and 83.7%, respectively, in the total AARD group. The ROC-AUCs of the CTD screen and its combination with IIF, EliA ENA, and LIA were over 0.90 for all subgroups, indicating excellent diagnostic performances, except for the ROC-AUCs in the SSc group, which showed fair performance (Figure 1). The differences between the ROC curves of the CTD screen and its combinations when differentiating AARDs from the control group were also analyzed. The ROC-AUCs for CTD screen and its combination with LIA were 0.89 and 0.92, respectively, demonstrating a significant difference () in the total AARD group, while the other combinations in the total AARD group, as well as in the SLE, SSc, SS, and MCTD groups, did not show significant differences.

(a)

(b)

(c)

(d)

(e)

Figure 1: Diagnostic performance of the CTD screen, tested independently and in combination with autoantibody assays. (a) Receiver operating characteristic (ROC) curves of the CTD screen (0.89) and its combination with autoantibody assays for discriminating total antinuclear antibody-associated rheumatoid disease patients () from control subjects (); (b) ROC curves for the CTD screen (0.93) and its combination with autoantibody assays, differentiating patients with systemic lupus erythematosus (SLE) () from control subjects; (c) ROC curves for the CTD screen (0.73) combined with autoantibody assays, differentiating patients with systemic sclerosis (SSc) () from control subjects; (d) ROC curves for the CTD screen (0.93) and its combination with autoantibody assays, discriminating patients with Sjögren’s syndrome (SS) () from control subjects; (e) ROC curves for the CTD screen (0.95) and its combination with autoantibody assays, differentiating patients with mixed connective tissue disease (MCTD) () from control subjects.

3.4.2. Likelihood Ratio and Odds Ratio

The positive and negative likelihood ratios of the CTD screen and its combination with IIF, EliA ENA, and LIA in the total AARD, SLE, SS, and MCTD groups were considered very useful (positive likelihood ratio > 5.0 or ≤0.2 for all groups). The positive likelihood ratios for the CTD screen and its combination with IIF, EliA ENA, and LIA in the SSc group were also very useful, while the negative likelihood ratios among them ranged from 0.3 to 0.4, indicating that they were just useful, based on the criteria of Solomon et al. [4]. Additionally, the odds ratios of CTD screen with LIA (67.0) were marginally higher than those of CTD screen alone (50.7) in the total AARD group.

3.5. Multivariate Analysis of CTD Screen, IIF, EliA ENA, and LIA Test Results

Multivariate analysis was performed using the presence of an AARD as the binary dependent variable and age and sex of the patient and the CTD screen, IIF, EliA ENA, and LIA results as predictors. We found that age and the results from CTD screen, IIF, EliA ENA, and LIA testing were independently associated with total AARD (Table 4). Our results showed that age, CTD screen, IIF, and LIA results for SLE; age, sex, IIF, and LIA results for SSc; and EliA ENA and LIA results for SS independently correlated with the respective rheumatic diseases. Sex was not a predictor in the SS and MCTD groups in the multivariate analysis because all subjects were female.

Table 4: Multivariate analysis of the outcomes of antinuclear antibody-associated rheumatoid disease.

Dependent variables
Covariate
OR
SE
95% CI
value

Total AARD ()
Age (years)
0.9784
0.0081
0.9631 to 0.9940
0.0069

Female sex
0.3754
0.8086
0.0769 to 1.8311
0.2256

CTD screen
1.1017
0.0306
1.0374 to 1.1699
0.0016

IIF
4.2961
0.2977
2.3972 to 7.6993
<0.0001

EliA ENA
2.8036
0.3589
1.3875 to 5.6653
0.0041

LIA
7.6024
0.3342
3.9493 to 14.6345
<0.0001

SLE ()
Age (years)
0.9680
0.0103
0.9485 to 0.9878
0.0017

Female sex
0.7809
1.1675
0.0792 to 7.6986
0.8323

CTD screen
1.1309
0.0336
1.0588 to 1.2080
0.0003

IIF
4.5804
0.3904
2.1309 to 9.8456
0.0001

EliA ENA
2.0365
0.4773
0.7991 to 5.1899
0.1362

LIA
8.7782
0.4639
3.5360 to 21.7918
<0.0001

SSc ()
Age (years)
0.9688
0.0143
0.9420 to 0.9963
0.0265

Female sex
0.0707
0.9631
0.0107 to 0.4669
0.0265

CTD screen
0.9501
0.0668
0.8336 to 1.0830
0.4435

IIF
7.6505
0.5496
2.6051 to 22.4672
0.0002

EliA ENA
1.8686
0.6595
0.5130 to 6.8058
0.3431

LIA
4.7949
0.5912
1.5049 to 15.2774
0.0080

SS ()
Age (years)
1.0273
0.0163
0.9950 to 1.0606
0.0985

CTD screen
1.0595
0.0489
0.9626 to 1.1661
0.2374

IIF
1.8878
0.6031
0.5789 to 6.1559
0.2920

EliA ENA
12.7179
0.8660
2.3295 to 69.4327
0.0033

LIA
22.8184
1.1071
2.6055 to 199.8360
0.0047

MCTD ()
Age (years)
0.9947
0.0277
0.9421 to 1.0502
0.8466

CTD screen
1.0226
0.0629
0.9039 to 1.1569
0.7221

IIF
8.4246
1.2892
0.6733 to 105.4167
0.0983

EliA ENA
7.6465
1.3879
0.5036 to 116.1116
0.1427

LIA
3.5924
1.3567
0.2515 to 51.3162
0.3459

OR: odds ratio; SE: standard error; CI: confidence interval; IIF: indirect immunofluorescence; ENA: extractable nuclear antigen; LIA: line immunoassay; AARD: antinuclear antibody-associated rheumatoid disease; SLE: systemic lupus erythematosus; SSc: systemic sclerosis; SS: Sjögren’s syndrome; MCTD: mixed connective tissue disease. Multivariate analysis was performed using the presence of each AARD as a binary dependent variable and with the age, patient sex, and the results of CTD screen, IIF, EliA ENA, and LIA tests as covariates. The SS group and MCTD group were composed of females only.

4. Discussion

Diagnostic applications of the CTD screen was assessed in Korean patients with AARDs, retrospectively. The performance of an automated screening assay was compared to HEp-2 cell-based IIF, EliA ENA, and LIA methods. In addition, diagnostic values of combining the CTD screen and these 3 autoantibody assays were investigated for each AARD.

The proportions of CTD screen, IIF, EliA ENA, and LIA positivities were higher in patients diagnosed with AARDs than in the control group, indicating that these four assays would be useful for detecting the patients with AARDs. Previous reports have shown similar results with these assays [19–22]. The sensitivities and specificities of these four assays would be affected because the classification criteria of each AARD were applied to our study as a standard. Regarding the sensitivity of the HEp-2 IIF, the values in this study were considerably low when compared to those in the previous reports [23, 24]. Meanwhile, there are several studies with decreased sensitivities of HEp-2 IIF such as 69.0%, 36.4%, and 53.7% [16, 22, 25], even lower than those of our study. Although our laboratory obtained accreditation, tested kits, condition of microscopes, interobserver variations, and low number of the patient population may have an influence on the values, similar to these studies with lowered sensitivities.

The negative results in the AARD groups were caused by earlier clinical observations and subsequent development of positivity in follow-up or a negative conversion after initial positivity at presentation because of immunosuppressive treatment, based on a review of each medical chart. Clinical and serological presentations can fluctuate over time in AARD patients, according to the results of several studies [19, 26].

Regarding the overall concordance rates between the CTD screen and IIF test, the agreements ranged from 87% to 95% when comparing the EIA assays and IIF in a previous study [27]. Previously, Cohen’s kappa value observed when combining EIA results of ENA and IIF tests was reported to be 0.30 [19], which was the same as that found with combined EliA ENA and IIF testing (0.30) and lower than that of the CTD screen and the IIF test (0.70), in this study. The previous report showed that the EIA for ENA and IIF results exhibited fair agreement for patients with SLE and that the kappa agreement for patients with scleroderma was good. Our results also demonstrated that the kappa value between EliA ENA and IIF of the SSc group (0.45) was higher than that of the SLE group (0.34). The kappa values between the CTD screen and IIF test were higher than those of EliA ENA and IIF in all subgroups, indicating that the CTD screen was more appropriate for screening with more diverse antigens than the EliA ENA test.

With respect to the LIA results, the kappa coefficients compared to IIF were similar to the previous results in the total AARD group (0.45 versus 0.404), as well as the SLE (0.28 versus 0.298) and SSc (0.48 versus 0.573) groups [28]. These agreements between the LIA and IIF methods were also similar to those between the LIA test and CTD screen, reflecting the characteristics of CTD screen and IIF as screening assays. There were slight differences between the different study populations and the cutoff used for each method. A detailed description of the agreements found between these tests (including concordance rates, Cohen’s kappa values, and values for McNemar tests) among the widely used autoantibody assays has been scarcely reported in the literature with large groups.

The diagnostic abilities of the CTD screen to discriminate each AARD subgroup from the control group were analyzed because the clinical utility of the CTD screen can vary with the characteristics of AARD patients and nonrheumatic conditions. The portion of SSc patients in the total AARD group may explain the lower CTD screen performance in the total AARD group, based on the ROC-AUCs. A previous study was conducted to evaluate automated ENA screening assay with 9 antigens and showed an ROC-AUC of 0.823, sensitivity of 63%, and specificity of 91% for AARD patients [22], which were slightly lower than the values obtained in our study. Another previous report showed that the sensitivities of the CTD screen for SLE, SSc, SS, and MCTD groups were 74%, 72%, 89%, and 100%, respectively [20]. Although the sensitivities reported for previous studies and those of our study were slightly different, the highest sensitivity for the MCTD group and lowest sensitivity for the SSc group [20] were concordant to our study. With respect to the cutoffs, we described the CTD screen results at 0.7 and 1.0 because these values are suggested by the manufacturer’s instructions. Based on the best cutoff of ROC-AUCs in our study, 1.0 of CTD screen is closer to it than 0.7 for total AARD, SLE, SSc, and MCTD and vice versa for SS. Therefore, 1.0 would be the best cutoff for patients with AARD, including SLE, SSc, and MCTD while 0.7 would be more appropriate for patients with SS. After verification, applying the optimal cutoff for each clinical laboratory is recommended.

Regarding clinical implications based on our data, CTD screen was comparable to IIF or the other assays because the 95% confidence intervals (CIs) of sensitivities and ROC-AUCs were overlapped with those in the others, indicating no statistical significance. Further, CTD screen exhibited slightly better specificity than the other assays according to the results of Tables 1 and 3, raising the possibility of an alternative screening assay. However, sufficient review of previous literatures for ANA testing [5, 29] is necessary when these assays are adopted to individual clinical laboratories. A variety of confounding factors such as patient populations, controls, applied methods, and cutoffs might affect the analyzed results.

The positive likelihood ratios of the CTD screen were very useful and conclusive, with respect to pretest and posttest probabilities [4, 30]. Previously published data also demonstrated positive results for the CTD screen at a cutoff of 1.0, with positive likelihood ratios greater than 5.0 in the SLE, SSc, SS, and MCTD groups [20]. The negative likelihood ratios for the CTD screen were less than 0.2 with the exception of SSc patients. Op De Beeck et al. [20] showed a similar pattern for the CTD screen in patients with SLE (0.3), SSc (0.3), and SS (0.1), which was considered to be very useful or useful tests, based on the negative likelihood ratios [4].

Based on the values of ROC-AUCs and odds ratios, concurrent determination with the CTD screen and LIA test demonstrated slightly improved diagnostic utilities in patients in the total AARD group, compared to the CTD screen alone. According to a previous report, the diagnostic performance of the CTD screen was significantly improved when used in combination with IIF, based on the likelihood ratios and ROC-AUCs [31]. The study population was patients of the university hospital in Belgium, and the results were slightly varied according to the diagnosis, suggesting different strategies for each AARDs. Therefore, uniformed and simultaneous screening by the CTD screen and IIF [29] would not be effective in cases with strong clinical manifestations associated with each AARD. Further, investigating laboratory findings with respect to other immunologic disorders might be essential for diagnosing each AARD when the screening assay shows positive results.

Multivariate analysis was also conducted for age and sex to assess CTD screen, IIF, EliA ENA, and LIA results as predictor variables for AARD groups and to control for potential confounding factors. ANAs were previously found in healthy individuals, particularly women older than 40 years [32]. Therefore, the factors of age and sex were incorporated into our multivariate model. The multivariate analysis results revealed that CTD screen, IIF, EliA ENA, and LIA results were independently related to the total AARD group, indicating that these four autoantibody assays could be applied for diagnosing AARDs. However, not all assays showed significant values in the subgroups. Based on our multivariate analysis, a combination with LIA for diagnostic work-ups of patients with SSc or SS would be necessary, when the CTD screen was applied to the clinical laboratories.

The profiles for the individual autoantibodies in the EliA ENA and LIA tests were generally concordant with previous reports [17, 28]. The high specificities of EliA ENA and LIA for individual autoantibodies (over 95.0%) were similar to those of a previous study, which showed at least 97% specificities [17]. These assays, especially the LIA, have been used as confirmatory tests, based on their high specificities [9]. The relatively low specificities of anti-Ro/SSA and Scl-70, which are related to SS and SSc, respectively [17, 24], might contribute to the lowered specificities of LIA.

Our study design has some limitations in that most of our study population was composed of females, especially the SS and MCTD groups. Among the hospitalized SLE patients, 165 out of 180 (91.8%) were female, based on the information in the Korean lupus network registry [33]. Although our data reflected a predominant portion of female patients, further studies involving male patients should be followed. Moreover, the number of patient population was small, especially MCTD, because of low prevalence. Further studies involving large study populations are necessary, considering the statistical power, interpretation, and application of the results to AARD patients.

5. Conclusions

In conclusion, we evaluated the diagnostic utilities of CTD screen, IIF, EliA ENA, and LIA testing to predict AARDs. Although a few published reports have discussed the usefulness of the CTD screen alone, no report has assessed a large population of Korean patients in Asia or the diagnostic performance of these frequently used autoantibody assays for AARDs in current use. CTD screening and the other three assays are potentially valuable diagnostic tools for detecting AARDs. In particular, the CTD screen showed considerable diagnostic performance for patients with SLE rather than SSc, and its combination with the LIA test might be more effective for diagnosing total AARDs than the CTD screen alone. Our results provided recent information regarding the CTD screen, IIF, EliA ENA, and LIA methods for testing patients with AARDs to facilitate appropriate patient management, with cautious clinical impressions of patients’ diagnosis.

Conflicts of Interest

The authors declare that there is no conflict of interest regarding the publication of this article.

Acknowledgments

The authors would like to thank Phadia AB, Euroimmun AG, and the Cancer Research Institute (Kosin University College of Medicine) for providing technical support and the Biomedical Research Institute (Pusan National University Hospital) for statistical consultation. This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (Ministry of Science, ICT & Future Planning) (NRF-2017R1C1B2004597).

References

A. Kavanaugh, R. Tomar, J. Reveille, D. H. Solomon, and H. A. Homburger, “Guidelines for clinical use of the antinuclear antibody test and tests for specific autoantibodies to nuclear antigens. American College of Pathologists,” Archives of Pathology & Laboratory Medicine, vol. 124, no. 1, pp. 71–81, 2000.

Y. Liu, L. Dai, W. Liu, G. Shi, and J. Zhang, “Autoantibody to MDM2: a potential serological marker of systemic lupus erythematosus,” Journal of Immunology Research, vol. 2015, Article ID 963568, 6 pages, 2015.

A. Doria, M. Zen, M. Canova et al., “SLE diagnosis and treatment: when early is early,” Autoimmunity Reviews, vol. 10, no. 1, pp. 55–60, 2010.

D. H. Solomon, A. J. Kavanaugh, P. H. Schur, and American College of Rheumatology Ad Hoc Committee on Immunologic Testing Guidelines, “Evidence-based guidelines for the use of immunologic tests: antinuclear antibody testing,” Arthritis & Rheumatism, vol. 47, no. 4, pp. 434–444, 2002.

N. Agmon-Levin, J. Damoiseaux, C. Kallenberg et al., “International recommendations for the assessment of autoantibodies to cellular antigens referred to as anti-nuclear antibodies,” Annals of the Rheumatic Diseases, vol. 73, no. 1, pp. 17–23, 2014.

M. Fenger, A. Wiik, M. Høier-Madsen et al., “Detection of antinuclear antibodies by solid-phase immunoassays and immunofluorescence analysis,” Clinical Chemistry, vol. 50, no. 11, pp. 2141–2147, 2004.

P. L. Meroni and P. H. Schur, “ANA screening: an old test with new recommendations,” Annals of the Rheumatic Diseases, vol. 69, no. 8, pp. 1420–1422, 2010.

S. M. Orton, A. Peace-Brewer, J. L. Schmitz, K. Freeman, W. C. Miller, and J. D. Folds, “Practical evaluation of methods for detection and specificity of autoantibodies to extractable nuclear antigens,” Clinical and Vaccine Immunology, vol. 11, no. 2, pp. 297–301, 2004.

D. Chandratilleke, R. Silvestrini, S. Culican et al., “Comparison of two extractable nuclear antigen testing algorithms: ALBIA versus ELISA/line immunoassay,” Pathology, vol. 48, no. 5, pp. 491–497, 2016.

M. Petri, A. M. Orbai, G. S. Alarcón et al., “Derivation and validation of the Systemic Lupus International Collaborating Clinics classification criteria for systemic lupus erythematosus,” Arthritis & Rheumatism, vol. 64, no. 8, pp. 2677–2686, 2012.

M. C. Hochberg, “Updating the American College of Rheumatology revised criteria for the classification of systemic lupus erythematosus,” Arthritis & Rheumatism, vol. 40, no. 9, p. 1725, 1997.

F. van den Hoogen, D. Khanna, J. Fransen et al., “2013 classification criteria for systemic sclerosis: an American College of Rheumatology/European League Against Rheumatism collaborative initiative,” Arthritis & Rheumatism, vol. 65, no. 11, pp. 2737–2747, 2013.

C. Vitali, S. Bombardieri, R. Jonsson et al., “Classification criteria for Sjӧgren’s syndrome: a revised version of the European criteria proposed by the American-European Consensus Group,” Annals of the Rheumatic Diseases, vol. 61, no. 6, pp. 554–558, 2002.

D. Alarcon-Segovia and M. H. Cardiel, “Comparison between 3 diagnostic criteria for mixed connective tissue disease. Study of 593 patients,” The Journal of Rheumatology, vol. 16, no. 3, pp. 328–334, 1989.

S. Jeong, H. Yang, and H. Hwang, “Evaluation of an automated connective tissue disease screening assay in Korean patients with systemic rheumatic diseases,” PLoS One, vol. 12, no. 3, article e0173597, 2017.

J. C. Parker and C. C. Bunn, “Sensitivity of the Phadia EliA connective tissue disease screen for less common disease-specific autoantibodies,” Journal of Clinical Pathology, vol. 64, no. 7, pp. 631–633, 2011.

A. H. L. Low, S. Wong, J. Thumboo et al., “Evaluation of a new multi-parallel line immunoassay for systemic sclerosis-associated antibodies in an Asian population,” Rheumatology, vol. 51, no. 8, pp. 1465–1470, 2012.

S. Jeong, Y. Park, Y. Cho, Y. R. Kim, and H. S. Kim, “Diagnostic values of urine CYFRA21-1, NMP22, UBC, and FDP for the detection of bladder cancer,” Clinica Chimica Acta, vol. 414, pp. 93–100, 2012.

R. Baronaite, M. Engelhart, T. Mørk Hansen et al., “A comparison of anti-nuclear antibody quantification using automated enzyme immunoassays and immunofluorescence assays,” Autoimmune Diseases, vol. 2014, Article ID 534759, 8 pages, 2014.

K. Op De Beeck, P. Vermeersch, P. Verschueren et al., “Detection of antinuclear antibodies by indirect immunofluorescence and by solid phase assay,” Autoimmunity Reviews, vol. 10, no. 12, pp. 801–808, 2011.

A. Ghirardello, R. Bendo, M. E. Rampudda, N. Bassi, S. Zampieri, and A. Doria, “Commercial blot assays in the diagnosis of systemic rheumatic diseases,” Autoimmunity Reviews, vol. 8, no. 8, pp. 645–649, 2009.

C. González, B. García-Berrocal, M. Pérez, J. A. Navajo, O. Herraez, and J. M. González-Buitrago, “Laboratory screening of connective tissue diseases by a new automated ENA screening assay (EliA Symphony) in clinically defined patients,” Clinica Chimica Acta, vol. 359, no. 1-2, pp. 109–114, 2005.

P. van der Pol, L. E. Bakker-Jonges, J. H. S. A. M. Kuijpers, and M. W. J. Schreurs, “Analytical and clinical comparison of two fully automated immunoassay systems for the detection of autoantibodies to extractable nuclear antigens,” Clinica Chimica Acta, vol. 476, pp. 154–159, 2018.

M. Mahler, P. L. Meroni, X. Bossuyt, and M. J. Fritzler, “Current concepts and future directions for the assessment of autoantibodies to cellular antigens referred to as anti-nuclear antibodies,” Journal of Immunology Research, vol. 2014, Article ID 315179, 18 pages, 2014.

H. G. Otten, W. J. Brummelhuis, R. Fritsch-Stork et al., “Measurement of antinuclear antibodies and their fine specificities: time for a change in strategy?” Clinical and Experimental Rheumatology, vol. 35, no. 3, pp. 462–470, 2017.

S. C. Simmons, M. L. Smith, A. Chang-Miller, and M. T. Keddis, “Antinuclear antibody-negative lupus nephritis with full house nephropathy: a case report and review of the literature,” American Journal of Nephrology, vol. 42, no. 6, pp. 451–459, 2015.

Y. Kumar, A. Bhatia, and R. Minz, “Antinuclear antibodies and their detection methods in diagnosis of connective tissue diseases: a journey revisited,” Diagnostic Pathology, vol. 4, no. 1, p. 1, 2009.

S. A. Lee, J. Kahng, Y. Kim et al., “Comparative study of immunofluorescent antinuclear antibody test and line immunoassay detecting 15 specific autoantibodies in patients with systemic rheumatic disease,” Journal of Clinical Laboratory Analysis, vol. 26, no. 4, pp. 307–314, 2012.

N. Agmon-Levin, J. Damoiseaux, and Y. Shoenfeld, “Response to: “Detection of antinuclear antibodies: added-value of solid phase assay?” by Bossuyt and Fieuws,” Annals of the Rheumatic Diseases, vol. 73, no. 3, article e11, 2014.

X. Bossuyt, “Clinical performance characteristics of a laboratory test. A practical approach in the autoimmune laboratory,” Autoimmunity Reviews, vol. 8, no. 7, pp. 543–548, 2009.

X. Bossuyt and S. Fieuws, “Detection of antinuclear antibodies: added value of solid phase assay?” Annals of the Rheumatic Diseases, vol. 73, no. 3, article e10, 2014.

N. Bizzaro and A. Wiik, “Appropriateness in anti-nuclear antibody testing: from clinical request to strategic laboratory practice,” Clinical and Experimental Rheumatology, vol. 22, no. 3, pp. 349–355, 2004.

J. W. Lee, D. J. Park, J. H. Kang et al., “The rate of and risk factors for frequent hospitalization in systemic lupus erythematosus: results from the Korean lupus network registry,” Lupus, vol. 25, no. 13, pp. 1412–1419, 2016.