The type and anatomical site of specimen collection for laboratory diagnosis of
In 2006, a new variant of
In Belgium, the reimbursement by the social security insurance of the detection of microbes using molecular techniques by medical laboratories was specifically introduced into the legislation in 2008 [
From 2008 to 2012, 9 EQA sample panels were provided to the participants. It means two panels per year (CTA and CTB) except for 2012 where only one panel was provided. These panels consisted of urine and simulated swabs samples. In this paper only urine samples were considered. The EQA samples (Table
Name | Year | Content (copies/vial) | Status | Penalty | Name | Year | Content (copies/vial) | Status | Penalty |
---|---|---|---|---|---|---|---|---|---|
CTA08-01 |
|
570 | Detected | 2 | CTA10-01 |
|
280 | Detected | 2 |
CTA08-02 |
|
6 | Infrequently detected | 1 | CTA10-02 |
|
5700 | Frequently detected | 3 |
CTA08-03 |
|
5700 | Frequently detected | 3 | CTA10-03b |
|
ND | Frequently detected | 3 |
CTA08-04a |
|
57 | Infrequently detected | 1 | CTA10-04 |
|
57 | Infrequently detected | 1 |
CTA08-05 |
|
0 | Negative | 3 | CTA10-05 |
|
280 | Detected | 2 |
CTA08-06 |
|
570 | Detected | 2 | CTA10-06 |
|
0 | Negative | 3 |
CTA08-07 |
|
57 | Infrequently detected | 1 | CTB10-01 |
|
280 | Detected | 2 |
CTB08-01 |
|
570 | Detected | 2 | CTB10-02 |
|
57 | Detected | 2 |
CTB08-02 |
|
5700 | Frequently detected | 3 | CTB10-03 |
|
5700 | Frequently detected | 3 |
CTB08-03 |
|
57 | Infrequently detected | 1 | CTB10-04 |
|
0 | Negative | 3 |
CTB08-04 |
|
0 | Negative | 3 | CTB10-05b |
|
ND | Frequently detected | 3 |
CTB08-05 |
|
57 | Infrequently detected | 1 | CTB10-06 |
|
280 | Detected | 2 |
CTB08-06 |
|
570 | Detected | 2 | CTA11-01 |
|
50 | Infrequently detected | 1 |
CTB08-07b |
|
ND | Infrequently detected | 1 | CTA11-02 |
|
0 | Negative | 3 |
CTA09-01 |
|
57 | Infrequently detected | 1 | CTA11-03 |
|
400 | Detected | 2 |
CTA09-02 |
|
570 | Detected | 2 | CTA11-04b |
|
106 | Frequently detected | 3 |
CTA09-03 |
|
0 | Negative | 3 | CTA11-05 |
|
5000 | Frequently detected | 3 |
CTA09-04 |
|
570 | Detected | 2 | CTB11-01 | 2011 | 50 | Infrequently detected | 1 |
CTA09-05 |
|
5700 | Frequently detected | 3 | CTB11-02 | 2011 | 5000 | Frequently detected | 3 |
CTA09-06 |
|
57 | Infrequently detected | 1 | CTB11-03b | 2011 | 106 | Frequently detected | 3 |
CTA09-07b |
|
ND | Frequently detected | 3 | CTB11-04 | 2011 | 0 | Negative | 3 |
CTB09-01 |
|
570 | Detected | 2 | CTB11-05 | 2011 | 400 | Detected | 2 |
CTB09-02b |
|
ND | Frequently detected | 3 | CTA12-01b | 2012 | 2000 | Detected | 2 |
CTB09-03 |
|
0 | Negative | 3 | CTA12-02 | 2012 | 0 | Negative | 3 |
CTB09-04 |
|
570 | Detected | 2 | CTA12-03 | 2012 | 1000 | Detected | 2 |
CTB09-05 |
|
5700 | Frequently detected | 3 | CTA12-04 | 2012 | 250 | Detected | 2 |
CTB09-06a |
|
57 | Infrequently detected | 1 | CTA12-05 | 2012 | 4000 | Frequently detected | 3 |
CTB09-07 |
|
57 | Infrequently detected | 1 |
bSwedish variant missing 377 bp of the cryptic plasmid [
The samples varied in their amount of target
In Belgium, EQA is mandatory for the clinical biology laboratories [
The registered laboratories received the EQA samples (Table
After the closure of the results return period, participants received an individual and an overall final report. At the end of a cycle, the Belgian participants received an annual report including their results for all the panels of the previous year.
Sample status is assigned by peer-group consensus based on the qualitative results returned by all participants in the full EQA program. It was not a measure of the “strength” of a positive sample nor was it technology-dependent and was used solely for the scoring of the EQA data. The rationale for the sample status was as follows. Frequently detected: more than 95% of datasets recorded the correct positive result. Detected: between 65 and 95% of the datasets recorded the correct positive result. Infrequently detected: less than 65% of the datasets recorded the correct positive result. Negative: a sample that does not contain the target produced an unequivocal negative result.
A scoring system was established by QCMD for individual performance assessment (Table
Scoring system.
Sample status | Negative | Positive | Not determined |
---|---|---|---|
Frequently detected | +3 | 0 | Not scored |
Detected | +2 | 0 | Not scored |
Infrequently detected | +1 | 0 | Not scored |
Negative | 0 | +3 | Not scored |
The different methods used by the Belgian laboratories during the surveys were listed in Table
Method used per year and per participant.
Methods | 2008a | 2008b | 2009 | 2010 | 2011a | 2011b | 2012 |
---|---|---|---|---|---|---|---|
Real time PCR | |||||||
Abbott real time CT1 | 1 | 2 | 2 | 2 | 2 | 2 | 2 |
Abbott real time CT/NG1 | 1 | 2 | 8 | 10 | 13 | 15 | |
Nanogen Chlamydia tr. Q-PCR Alert kit2 | 1 | 1 | 1 | 1 | 1 | 1 | |
Shangai bio-tech IMtec CT real time PCR kit3 | 1 | ||||||
Roche Cobas TaqMan CT4 | 4 | 4 | 9 | ||||
Roche Cobas TaqMan CT v2.0 Roche4 | 14 | 14 | 12 | 12 | |||
Roche Cobas TaqMan CT/NG4 | 1 | 3 | 7 | ||||
Qiagen artus |
5 | 5 | 7 | 5 | 5 | 5 | 4 |
Diagenode DIA-CT-0506 | 1 | 1 | 2 | 1 | |||
Real time in house | 1 | 3 | 5 | 3 | 4 | 4 | |
Gen-Probe Aptima combo 2 Assay7 | 1 | ||||||
Gen-Probe PACE CT7 | 1 | ||||||
Gen-Probe PACE 2 CT7 | |||||||
SDA | |||||||
Becton Dickinson ProbeTec ET8 | 11 | 10 | 9 | 7 | 6 | 6 | 5 |
PCR-ELISA | |||||||
Roche Amplicor CT4 | 1 | ||||||
Roche Amplicor CT/NG4 | 12 | 11 | 8 | 4 | 2 | 3 | |
Roche Cobas Amplicor CT/NG4 | 10 | 10 | 17 | 3 | 4 | 1 | 1 |
Other | |||||||
Nasba in house | 2 | ||||||
Hain Lifescience Genoquick CT9 | 1 | ||||||
Total of the participants | 47 | 50 | 48 | 54 | 50 | 51 | 55 |
2Nanogen Advanced Diagnostics, Trezzano sul Naviglio, Italy.
3IMTEC, Berlin, Germany.
4Roche Molecular diagnostics, Pleasanton, USA.
5Qiagen, Venlo, Netherlands.
6Diagenode, Liège, Belgium.
7Gen-Probe Incorporated, San Diego, USA.
8Becton Dickinson and Co, Sparks, USA.
9Hain Lifesceince GmBH, Nehren, Germany.
Between January 2008 and June 2012, nine surveys were organized.
The number of participating laboratories ranged from 51 to 54 per survey. The percentage of responding laboratories ranged from 85 to 96%. The number of datasets returned to QCMD was always higher than the number of responding laboratories indicating that some laboratories introduced more than one dataset. Indeed, EQA is very often an occasion for the laboratories to validate new methods.
From 2008 to 2012, 58 different laboratories responded; among them 33 (56.9%) participated in 9 surveys, 5 (8.6%) in 8 surveys, 6 (10.3%) in 7 surveys, 1 (1.7%) in 6 surveys, 4 (6.9%) in 5 surveys, 2 (3.4%) in 4 surveys, 1 (1.7%) in 3 surveys, 3 (5.2%) in 2 surveys, and 3 (5.2%) in 1 survey.
The overall number of Belgian results was 2917. The returned results were always qualitative (presence or absence) results. The number of correct results (accuracy) was 2198/2917 (75.4%). The number of incorrect results was 719/2917 (24.6%). Of the incorrect results, the number of false positive results was 6/719 (0.8%) and the number of false negative results was 693/719 (96.4%). The number of inhibition results was 20/719 (2.8%). The number of negative and frequently detected samples was 445 and 747, respectively. The number of clinically relevant faults was 61/1192 (5.1%) including 6/445 (1.3%) false positive and 55/747 (7.4%) false negative. The overall sensitivity and specificity were 71.4% (1764/2472) and 97.6% (434/445), respectively. When the yearly evolution was considered (Figure
Parameters per survey. Ac: accuracy; Se: sensitivity; Sp: specificity.
The proficiency per sample is shown in Figure
Score of the laboratories per sample. The number (
The samples with regard to the amount of target genome present were divided into frequently detected, detected, infrequently detected, and negative sample status. The percentage of correct answers was 92.4% for frequently detected, 86.4% for detected, 45.3% for infrequently detected, and 97.5% for negative samples, respectively.
When the detection methods used were considered (Table
The proficiency per detection methods.
# | Method |
|
Spls2 | +3 | −4 | Ac5 (%) | Se6 (%) | Sp7 (%) | Swedish variant | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
B08-07 | A09-07 | B09-02 | A10-03 | B10-05 | A11-04 | B11-03 | A12-01 | Total | % | |||||||||
1 | Nanogen Chlamydia tr. Q-PCR Alert | 8 | 48 | 40 | 8 | 93.70 | 90.00 | 100.00 | 1/1 | 1/1 | 1/1 | 1/1 | 1/1 | 1/1 | 1/1 | 1/1 | 8/8 | 100.0 |
2 | Roche Cobas TaqMan CT8 | 93 | 547 | 454 | 93 | 90.10 | 88.10 | 100.00 | 0/4 | 0/9 | 10/10 | 14/14 | 14/14 | 14/14 | 12/12 | 6/12 | 76/89 | 85.4 |
3 | Abbott real time CT/NG | 56 | 303 | 247 | 56 | 87.40 | 92.30 | 98.20 | 1/1 | 2/2 | 2/2 | 5/5 | 8/8 | 10/10 | 13/13 | 11/15 | 52/56 | 92.9 |
4 | Abbott real time CT | 17 | 103 | 86 | 17 | 83.50 | 80.20 | 100.00 | 2/2 | 1/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 1/2 | 14/16 | 87.5 |
5 | Qiagen artus |
48 | 297 | 249 | 48 | 81.80 | 78.30 | 100.00 | 5/5 | 7/7 | 6/6 | 6/6 | 5/5 | 5/5 | 5/5 | 3/4 | 42/43 | 97.7 |
6 | BD Diagnostics BD ProbeTec ET | 70 | 441 | 371 | 70 | 75.90 | 78.20 | 95.70 | 10/10 | 9/9 | 8/8 | 8/8 | 7/7 | 6/6 | 6/6 | 2/5 | 56/59 | 94.9 |
7 | Roche Cobas Amplicor CT/NG | 53 | 359 | 306 | 53 | 75.20 | 71.20 | 98.10 | 0/10 | NA | 4/8 | 0/4 | 0/2 | 0/2 | 0/1 | NA | 4/27 | 14.8 |
8 | Roche Amplicor CT/NG | 51 | 329 | 278 | 51 | 64.10 | 59.40 | 94.10 | 0/11 | 4/17 | 1/8 | 0/7 | 0/5 | 0/3 | 0/2 | 0/3 | 5/56 | 8.9 |
Total | 396 | 2427 | 2031 | 396 | 80.2 | 78.8 | 98.0 | 22/47 | 24/47 | 36/48 | 39/50 | 42/49 | 41/46 | 39/41 | 26/42 | 257/354 | 72.6 | |
46.8% | 51.1% | 75.0% | 78.0% | 85.7% | 89.1% | 95.1% | 61.9% | 72.6% |
2The number of analyzed samples.
3+: the number of positive samples analyzed.
4−: the number of negative samples analyzed.
5Ac: accuracy = percentage of correct results.
6Se: sensitivity percentage of correct results for positive samples.
7Sp: specificity = percentage of correct results for negative samples.
8Roche Cobas TaqMan CT (for 2008 and first survey of 2009) and Roche Cobas TaqMan CT v2.0 (since second survey of 2009).
NA: not applicable.
During the nine surveys from 2008 to 2012, 8 samples (1 infrequently detected, 1 detected, and six frequently detected) consisted of the Swedish variant of
The laboratories were evaluated by the attribution of a score (Table
Means and standard deviation of the scores of the laboratories per survey for the detection of
In Belgium, mandatory EQA for the detection of
Between January 2008 and December 2012, nine surveys were organized. Only urine results are discussed in this paper since swabs results gave no additional information. For Belgium, 58 laboratories participated in at least one survey. These laboratories are clinical biology laboratories according to the Belgian legislation [
The Belgian participants represented around 28% of the total number of the participants in QCMD survey.
For
When the results were considered in relation to the type of samples, two major issues were observed. First, the sensitivity was low (below 50%) for the detection of infrequently detected samples. The results indicated that when the number of copies per vial was under 60, the detection power was low. It raises the question of the clinical cut-off. It seems clear that in case of a true infection the level of
Second, the sensitivity for the detection of the Swedish variant strain of
The new Swedish variant (nvCT) represents 20 to 64% of the detected Chlamydia cases in Sweden [
Among the most used methods, the Abbott real time PCR CT/NG and the Roche Cobas TaqMan CT/NG v2.0 gave the best results with an accuracy of 87.4 and 90.1%, respectively. It is not clear from the results that the homemade methods were less proficient than the commercial kits.
When the technologies used were compared, real time PCR methods gave better results than SDA methods or than conventional PCR methods. The TMA (Transcription Mediated Amplification) methodology was only used by one participant in two surveys and was not considered here.
Finally, when the proficiency over time was considered, a decrease in the penalty points scored was observed indicating an increase in proficiency. This is encouraging for the future and that fully justifies the need of EQA.
EQA allows comparing the proficiency of the laboratories but also of the methodologies. For a small country such as Belgium, it is also useful to participate in international surveys including other countries in order to be able to compare the result of a lab with other laboratories using the same methodology. It is particularly true for the methods that are not frequently used.
The major issue for the EQA is to have samples as close as possible to clinical samples. But these samples must also be homogeneous and stable. Moreover, the pre- and postanalytical process should also be assessed. For the preanalytical phase, the sampling and the transport conditions are important. The postanalytical evaluation is particularly important in case of low contaminated samples. It is not evident that all the laboratories will give the same answer to the clinician. The determination of an analytical and a clinical cut-off is sometimes required. Nevertheless, the improvement of the quality of the diagnosis is our priority and a powerful EQA is a major tool in this goal.
A correct diagnosis is closely related to powerful diagnostic tools. The proficiency of these methods can be evaluated using External Quality Assessment. In Belgium, for many years, the participation in EQA is mandatory for the medical laboratories. Moreover, for molecular testing, the ISO15189 accreditation is also required. The results of the EQA for the detection of
The authors declare that there is no conflict of interests regarding the publication of this paper.
The authors thank all the participating laboratories and the members of the experts committee for their input.