AIDS Research and Treatment

Volume 2014 (2014), Article ID 465932, 14 pages

http://dx.doi.org/10.1155/2014/465932

Review Article

Poor Reporting of Outcomes Beyond Accuracy in Point-of-Care Tests for Syphilis: A Call for a Framework

Yalda Jafari,¹ Mira Johri,2,3 Lawrence Joseph,¹ Caroline Vadnais,⁴ and Nitika Pant Pai4,5

¹Department of Epidemiology, Biostatistics & Occupational Health, McGill University, Montreal, QC, Canada, H3A 1A2

²Department of Health Administration, Université de Montréal, Montreal, QC, Canada, H3C 3J7

³Unité de Santé Internationale (USI), Centre de Recherche Hospitalièr de l’Université de Montréal (CRCHUM), Montreal, QC, Canada, H2X 0A9

⁴Division of Clinical Epidemiology, Department of Medicine, Royal Victoria Hospital, McGill University Health Centre, V Building (V2.19), 687 Pine Avenue West, Montreal, QC, Canada, H3A 1A1

⁵Department of Medicine, McGill University, Montreal, QC, Canada, H3A 1A1

Received 16 July 2013; Revised 23 February 2014; Accepted 27 February 2014; Published 27 March 2014

Academic Editor: David Dowdy

Copyright © 2014 Yalda Jafari et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Background. Point-of-care (POC) diagnostics for syphilis can contribute to epidemic control by offering a timely knowledge of serostatus. Although accuracy data on POC syphilis tests have been widely published, few studies have evaluated broader outcomes beyond accuracy that impact patients and health systems. We comprehensively reviewed evidence and reporting of these implementation research outcomes (IROs), and proposed a framework to improve their quality. Methods. Three reviewers systematically searched 6 electronic databases from 1980 to 2014 for syphilis POC studies reporting IROs. Data were abstracted and findings synthesised narratively. Results. Of 71 studies identified, 38 documented IROs. IROs were subclassified into preference (7), acceptability (15), feasibility (15), barriers and challenges (15), impact (13), and prevalence (23). Using our framework and definitions, a pattern of incomplete documentation, inconsistent definitions, and lack of clarity was identified across all IROs. Conclusion. Although POC screening tests for syphilis were generally favourably evaluated across a range of outcomes, the quality of evidence was compromised by inconsistent definitions, poor methodology, and documentation of outcomes. A framework for standardized reporting of outcomes beyond accuracy was proposed and considered a necessary first step towards an effective implementation of these metrics in POC diagnostics research.

1. Introduction

Syphilis is an important public health issue; the latest available worldwide estimates conducted by the World Health Organization in 1999 suggested approximately 12 million cases of syphilis [1]. With 90% of infected individuals unaware of their serostatus, a lack of timely diagnosis is a driving force of the syphilis epidemic, particularly in resource limited settings where use of standard diagnostics presents important challenges [2]. Standard syphilis screening involves initial testing with a nontreponemic specific assay and confirmation with a treponemal specific assay [3–6]. These tests are usually expensive, must be conducted by skilled, laboratory-based personnel, and have a long turnaround time resulting in loss of patients to followup [7, 8]. Point-of-care (POC) tests therefore offer a novel, low cost, easy to use solution that provides results in a single visit, expediting linkages to confirmatory testing, treatment, and referral. Designed for settings with limited infrastructure, POC technologies are potentially transformative tools for global syphilis control [3].

Systematic reviews demonstrated the accuracy of syphilis POC tests [9, 10]. However, as advisory bodies on diagnostics have emphasized, accuracy is a necessary but not a sufficient condition for widespread uptake of syphilis POC tests [11]. Our objective is to facilitate decision-making related to introduction of syphilis POC tests for health planners and policymakers in diverse geographical settings, by synthesising evidence on outcomes beyond accuracy, referred to as implementation research outcomes (IRO). These IROs are relevant to patients, caregivers, and health systems. This systematic review presents the first comprehensive portrait of available evidence on IROs for syphilis POC tests.

2. Materials and Methods

2.1. Information Sources

We searched the MEDLINE, EMBASE, GLOBAL HEALTH, CINAHL, Web of Science, and SCOPUS electronic databases over the period from January 1, 1980, to January 31, 2014. All languages were considered and non-English articles were translated.

2.1.1. Search String

We used the following terms: (syphilis OR Treponema pallidum) AND (point-of-care OR rapid test OR rapid assays). Keywords rather than MeSH terms were used to capture recent papers not yet indexed and to be able to search multiple databases at once. No filters for diagnostic studies were applied as these have been shown not to capture all relevant studies [12].

2.2. Study Selection

Three reviewers (Yalda Jafari [YJ], Sushmita Shivkumar [SS], and Rohit Vijh [RV]), independently conducted database searches and reviewed the titles and abstracts of articles retrieved. Articles satisfying the eligibility criteria were retained for full-text screening. Study inclusion was determined by discussion between the initial reviewers who, in cases of discordance, appealed to a fourth reviewer (Nitika Pant Pai [NPP]).

2.3. Eligibility Criteria

We considered studies satisfying the following criteria.

Study Design. All studies conducted on humans or human samples reporting IROs.

Participants. Live participants of any age and risk group using whole blood or serum specimens, either from participants or serum panels.

Interventions. Studies documenting the use of syphilis POC tests.

Outcome Measures. To systematize presentation, documented IROs were tabulated according to the working definitions in a framework proposed below.

(A)Preference: documented as a proportion (numerator/denominator) with 95% confidence intervals (CI). Numerator was defined as the number of study participants (i.e., patients, doctors, nurses, and lab technicians) in the study interviewed that preferred the POC test or testing strategy to the conventional reference testing strategy. Denominator was defined as the total number of participants (i.e., patients, doctors, nurses, and lab technicians) in whom the new POC test or strategy was evaluated.(B)Acceptability: documented as a proportion (numerator/denominator) with 95% CI. Numerator was defined as the number of study participants (i.e., patients, providers, etc.) who accepted the new POC test/strategy over the conventional reference test/strategy. Denominator was defined as the total number of participants that were offered the strategy. Participants who consented to study participation were offered the new test/strategy and acceptability was thereby documented.(C)Feasibility: feasibility of test/strategy evaluated in the study was documented. Feasibility of a test/strategy was defined with completion rate of implementation of POC strategy [13]. The completion rate (a misnomer) was documented as a proportion with 95% CI. Completion typically referred to completion of the test or screening procedure. This included screening with a POC test, typically followed by initiation of linkages to improve clinical management of patients. These linkages varied from confirmatory testing of POC test results to receipt of treatment based on POC results or to downstream treatment in referral clinics. Other associated metrics/factors relevant to feasibility of a strategy were patient experience related such as convenience, comfort, noninvasiveness, or ease of use of POC tests (reported as yes/no) and/or time metric related, that is, turnaround time to test results (TAT) with POC versus standard laboratory tests also included. Turnaround time to test results with POC was documented in minutes/hours. This TAT included waiting time to test result. Turnaround time to treatment initiation or time to receipt of confirmatory test results, if reported, was also documented. All time was reported in minutes/hours. Mean time with interquartile ranges or average (median) if available was also documented. Reported barriers and challenges to implementation of POC tests were abstracted and reported sometimes under feasibility.(D)Barriers and challenges: reported barriers and challenges to implementation of POCs are recorded.(E)Impact: impact was described by its different facets including (1) proportional increase in new cases picked up with the new POC strategy compared to the reference conventional strategy, (2) proportional increase in numbers that received treatment with the new POC strategy over reference conventional strategy, (3) a reduction in time to confirmatory testing with novel POC versus conventional strategy, (4) an increase in the number of sexual partners notified with novel POC test versus reference conventional, (5) documentation of provision of interventions to the mother and infant (as in numbers/proportions) with same day testing and treatment linked with new POC strategy over a conventional strategy, and (6) lastly, a reduction in time to referral linkages with novel POC versus conventional strategy. Data permitting, proportions were expressed with 95% CI.(F)Prevalence: study prevalence (typically, period prevalence and, sometimes, point prevalence) was defined as a proportion, where numerator consisted of individuals who tested seropositive during the study period, and denominator consisted of the total study sample [14]. If available, 95% CI was expressed. For the purposes of the review, studies that used POC tests solely to determine prevalence of syphilis in the study sample were included in this category.

2.4. Data Extraction

Reviewers extracted relevant information independently using a standardized data abstraction form pretested on a subset of the sample. One reviewer (YJ) extracted data on 100% of papers, and second reviewers (SS & RJ) abstracted data on 50% of papers.

2.5. Summary Measures

A narrative approach was taken to synthesize outcome data. This is standard practice for systematic reviews with considerable methodological heterogeneity such as IROs.

3. Results

Of 71 full-text articles, 38 (53.5%) documented IROs. Following PRISMA guidelines [15], our study selection process has been presented in Figure 1. Of 38 articles, three studies (7.9%) were in languages other than English (2 in Spanish, 1 in Portuguese) and were translated. Thirty-four (89.5%) articles referred to studies conducted in low- and middle-income country settings. Thirty-four studies employed a cross-sectional design (89.5%), one was a case-control (2.6%), and one was a clustered randomized trial (2.6%) design. Study descriptions for IROs are available in Table 1.

(A)Preference: seven articles discussed preference for POC strategies as compared to conventional strategies [16–22]. In a Brazilian study [16], 60% of clinicians and 52% of patients preferred conventional testing over the POC strategy. Other studies documented preference for the POC strategy amongst nurses (68% [17]) and patients (62% [19], 66.1% [23]) over the conventional approach. In a study conducted in India, 99.3% (95% CI: 98.8, 99.8) of patients preferred testing with 3 POC tests simultaneously to detect syphilis, HIV, and hepatitis B [20]. In an Australian study [18], 79% preferred self-testing with a POC test. In a multicountry study [22] most clients expressed preference for finger prick rather than venipuncture.(B)Acceptability: fifteen studies reported on patient and provider acceptability of POC tests [16, 17, 20, 22–33]. The POC testing strategy was found to be highly acceptable to patients [16, 20, 23, 24, 26–29, 31–34], clinicians [17, 22, 26], and lab technicians [16, 26].(C)Feasibility: fifteen articles reported on the feasibility of using POC tests and test performance characteristics [4, 16–20, 22, 23, 25, 32, 34–38]. Completion rate of POC testing was high in two studies [20, 23], as patient and provider satisfaction were reported with POC testing in one study [38]. Clinicians and laboratory staff found use and interpretation of results easy [4, 17, 19, 22, 35, 37]. In six studies, a reduction in turnaround time to testing and obtaining results was well received [16, 18, 19, 23, 35, 38]. In Brazil, 90% of patients reported that they would wait up to 30 minutes for results [35]. Simultaneous testing with 3 singleton POC tests (i.e., syphilis, HIV, and hepatitis B) was conducted in India and a TAT of 25 minutes (range 21–27 minutes) was reported [20]. Similarly, in a study conducted in South Africa, a TAT of 15–20 minutes was reported [36]. In three other studies, patients experienced low pain and discomfort with a finger prick, in comparison to tests requiring venipuncture: 77% [38], 91.1% [18], and 68% [16]. In a multicountry study, almost all clients who received a positive POC result received treatment, ranging from 93.6% in Brazil to 103.6% in Tanzania [22]. In an article describing studies in Uganda and Zambia with a POC strategy, almost all participants (99.0% in Uganda and 95.8% in Zambia) received same day testing and treatment with the novel POC strategy [32]. Additionally, partner testing for syphilis was also performed on-site, thereby proving feasibility of the strategy [32].(D)Barriers and challenges: barriers and challenges hindered an ideal implementation of POC testing strategies and impeded feasibility. Fifteen studies provided qualitative discussions of these challenges [16–20, 23, 26, 27, 29, 34, 35, 37–40]. From the patient perspective, assumed pain from finger prick was a hindrance to testing [19, 23, 29], as were reports of pain or discomfort [16, 20, 40]. Patients also expressed a distrust of POC test results [19]. From the providers’ perspective, challenges included a lack of confidence in POC test results due to the inability to differentiate between past and present infections leading to overtreatment [16, 17, 19, 35]. In addition, disagreements between test readers [37], suboptimal field conditions [34], unreadable [36] or difficulty reading results [39], concerns over procurement of POC tests [38], and the time-consuming nature of offering one-stop services with POC [19, 38] posed challenges in delivery of these services. Furthermore, not all patients agreed to undergo confirmation testing [27], hampering ideal execution of testing process. In two articles evaluating POC tests in Mongolia, inadequacies of partner notification strategies for women were brought to light, raising issues such as risk of reinfection of women if partners are not treated [40] and risk of partner violence [40]. A lack of political leadership, required to ensure the success of antenatal care (ANC) programs to prevent congenital syphilis, was also highlighted as a major barrier to uptake of POC testing [26].(E)Impact: fourteen studies reported on the impact highlighting increased testing and treatment of study participants with the implementation of a novel POC screening strategy in comparison to the reference conventional screening strategy [17, 20, 22, 27, 29, 30, 32, 36, 39–44]. A study in Mongolia measured impact of POC tests in a cluster randomized trial design [40]. In comparison to clinics that used conventional testing, clinics with POC testing showed a significantly increased number of women who were tested and a significantly increased number of women and their partners who were treated [40]. In a multicountry study, the increase in numbers screened compared to preimplementation of POC testing ranged from 1.0% in China to 9.2% in Zambia [22].(F)Prevalence: twenty-three articles [18, 20, 21, 23–28, 30–32, 39, 41, 44–52] documented prevalence in various populations and estimated the seropositivity in the study sample. Prevalence estimates varied in populations and regions and varied with samples and risk profile of participants. In ANC attendants, prevalence ranged from 0% in Afghanistan [51] to 9.2% in Zambia [32]. In high-risk populations, the prevalence ranged from 0% in Guatemala [48] to 8.9% in eastern Sudan [46].

Table 1: Description of studies.

Study
Setting
Population and sample size
Index test
Study design
IRO preference
IRO acceptability
IRO feasibility
IRO barriers and challenges
IRO impact
IRO prevalence

Amadi et al., 2010 [45]
Nigeria, urban (LM)
100 dental clinic patients
Syphilis Ultra Rapid Test Strip
CS

1%

Benzaken et al., 2007 [35]
Brazil, urban (UM)
Study 1: 541 STD clinic attendees.

Study 2: 248 STD clinic attendees
Study 1: SD Bioline Syphilis and Syphicheck-WB

Study 2: VisiTect Syphilis and Determine.
CS

100% of professionals rated instructions and results interpretation as easy.

Study 1: reproducibility

between clinic and lab professionals had agreement of 99% and kappa index >0.95% for both tests.

Study 2: agreement between clinic and lab professionals was 100%.

In both studies, 100% of participants were willing to wait up to 30 minutes for their results.
Unnecessary treatment was an issue.

Benzaken et al., 2008 [19]
Brazil, urban (UM)
510 STI clinic attendees including MSW and FSW; operational characteristics were collected from 12 clinic staff and 60 clients.
Visitect
CS
Clients: 62% preferred POC strategy.

Clinic staff: 75% found test easy to use and 67% found the test easy to interpret.

Clients: 93% were not deterred by the waiting time.
Clients:

waiting time (7%), cost of transport (10%), opening hours (25%) or lack of trust in POC test results (3%), pain caused by finger prick (57%), and preference for venous blood (38%) collection were some barriers.

Clinic staff: only half trusted its results, mostly cited reason being due to inability of the test to differentiate between old and recent syphilis leading to overtreatment and also experienced frequent discordant results between POC and confirmatory tests.

Bronzan et al., 2007 [17]
South Africa, rural (UM)
1285 ANC attendees.
Determine
CS
Nurses: 68.2% preferred on-site ICS over on-site RPR and off-site RPP/TPHA.
Nurses: on-site testing was acceptable and favoured over off-site testing as it allowed for prompt diagnosis, patient education, and immediate treatment.
100% found on-site ICS test easy to perform, fast, and reliable, compared to 95% who found on-site RPR time consuming, unreliable, and difficult to perform and read.
Unnecessary treatment.
89.4% of women with high-titer syphilis received treatment with ICS versus 63.9% in women tested with on-site RPR and 60.8% tested with off-site approach.

Callegari et al., 2014 [24]
Brazil, urban (UM)
438 adult patents (18 years plus) of outpatient clinic.
Rapid check syphilis immunochromatographic treponemal test.
CS

5.3% (95% CI: 3.3–7.3)

Campos et al., 2006 [34]
Peru, urban (UM)
3586 FSW in commercial sex venues.
Determine
CS

97.4% agreed to POC test.
Easy to adapt, implement, and integrate with existing into existing work-based STI prevention services.
Inadequate lighting in the field was a problem.

Chen et al., 2013 [25]
China, urban (LM)
1808 FSW
Wantai Anti-TP Antibody Rapid Test (RST).
CS

94.2% (95% CI 93.1–95.2%) got the testing.
95.2% (95% CI 94.1–96.1%) were willing to get RST on-site.

8.5%

Chen et al., 2012 [23]
China, urban (LM)
2812 FSW
Wantai Anti-TP Antibody Rapid Test.
CS
66.1% preferred finger prick over blood draw.
95.0% (95% CI: 94.1–95.7%) accepted rapid test.
99.3% (95% CI: 98.9–99.5%) of those accepted testing got the test.

52.5% preferred verbal on-site method of result notification.
57.7% listed pain as reason for not wanting to be finger pricked.

6.8%

Dayan et al., 2013 [21]
Turkey, urban (UM)
266,035 healthy blood donor samples.
Architect (Abbott).
Retrospective cohort.

0.07

Dlamini et al., 2014 [36]
South Africa, urban and rural (UM).
297 samples from different health facilities.
SD Bioline, Hexagon
CS

Testing with both tests was completed within the recommended 15–20 min.

3.4% of SD Bioline results were unreadable. Visibility was improved on repeat testing. Hexagon results were more clear and quick in comparison to SD Bioline.

Elhadi et al, 2013 [46]
Sudan, urban (LM)
4220 FSW
SD Bioline
CS

Ranged from 1.5% in the northern zone to 8.9% in the eastern zone (of their division of Sudan).  

Garcia et al., 2013 [39]
Peru, urban (UM)
17,155 ANC attendees.
SD Bioline
CS

9 of 604 (44.5%) health providers participating in the training had difficulties with near vision which did not allow them to recognize the line in the POCT" and 0.3% were colour blind and could not see the red line.
Improved treatment coverage with 91.6% receiving at least one dose of penicillin and 80% with two doses reduced a process that consisted of 27 days in 6 visits to 1 visit in one day; total screening coverage was 94.8%.
0.90%

García et al., 2007 [26]
Bolivia, urban and rural (LM)
11,618 ANC attendants.
Determine
CS

POC testing highly acceptable to participants, clinicians, and laboratory technicians.

Political instability; difficult to ensure continuity.

5%

Gupte et al., 2011 [27]
India, urban (LM)
19,809 female, male, and transgender sex workers.
Syphicheck-WB
CS

Acceptance was high, ranging from 76.0% in MSW to 57.1% bar-based FSW.

Only 2/3 of those positive with POC agreed to RPR confirmation.
Using POC, during 3-month intervention period, syphilis test uptake was 63.1%, more than 4-fold higher than the monthly average uptake of 14.3% at clinic sites during the preceding 9-month period.
Life time infection 3%

active syphilis confirmed with RPR-1.2%.

Herring et al., 2006 [4]
South Africa (UM), Gambia (L), Tanzania (LI), China (LM), Sri Lanka (LM), Haiti (L), USA (H), Russian Federation (UM).
Evaluation panel from archived specimens.
Determine

Syphilis Fast

Espline TP

Syphicheck-WB

SD Bioline

Visitect Syphilis.
CC

Multisite evaluation six kits scored on clarity of instructions, technical complexity, ease of interpretation, and equipment required but not provided. Determine scored the highest and Syphilis Fast scored the lowest.

Test reproducibility variability was low.

Hurtado et al., 2010 [47]
Spain, urban (H)
500 MSM in saunas and flats.
Determine
CS

5% in saunas.

2.3% in apartments.   

Juárez-Figueroa et al., 2007 [37]
Mexico, urban (UM)
548 FSW and women within 24 h postpartum.
Determine
CS

All 3 readers found POC test highly user-friendly, and there was agreement in 544 of 548 results (99.3%).
Disagreements were due to appearance of faint lines.

Lahuerta et al., 2011 [48]
Guatemala, urban (LM)
2874 FSW, MSM, TG MSM, NR (MV: 1336, STI clinic: 1538).
Determine
CS

MV:

NR: 0.8%, MSM/TG: 0%, FSW: 4.6%.

STI clinic:

NR: 0.6%, MSM/TG: 1.4%, FSW: 1.3% .

Lee et al., 2010 [18]
Australia, urban (H)
183 MSM
Determine
CS
79% preferred rapid testing at clinic to venipuncture and serology.

54% preferred self-test if it was available.

Of those who preferred POC at clinic over venipuncture and serology, reasons were immediacy of the result (32.6%), reduced pain or invasiveness (8.9%), and the convenience of not requiring a second clinic visit for test results (4.4%).

3.8%

Mabey et al., 2012 [22]
Brazil, rural (UM) China, rural Peru, rural and urban (UM) Tanzania, rural (L) Uganda, rural (L) Zambia, rural and urban (LM).
Over 100,000 ANC attendees.
China: Rapid Syphilis test

Rest: SD Bioline Rapid Syphilis test.
CS
Most clients preferred a finger prick over venipuncture

because of the smaller volume of blood required.
POC tests were well accepted by health care workers.
All health care workers in all sites thought POC tests were easy to perform. For instance, 82% of Ugandan health care workers reported POC tests as “very easy to perform.”

Clients liked receiving results and treatment on the same day as testing rather than having to return.

Almost all who tested positive received treatment: 100% in Brazil, 93.6% in China, 97% in Peru, 90.1% in Tanzania, 103.6% in Uganda, and 95.2% in Zambia.

Change in percent of population that were screened following a POCT introduction was 1.6% in Brazil’s sexually active population and 1.4% in their ANC population. Increase in screening was seen in other sites: 1.9% in China, 1.0% in Peru, 10.9% in Tanzania, 5.3% in Uganda, and 9.2% in Zambia.

Manavi et al., 2012 [28]
UK, urban (H)
405 men attending gay pride event.
OraSure assay
CS

96% accepted testing.

0.5%

Miranda et al., 2009 [41]
Brazil, urban (UM)
1380 ANC attendees.
Determine
CS

5.1% with no previous prenatal care learned serostatus among which 1 was positive detected.
0.4% 

Mishra et al., 2010 [29]
India, urban (LM)
4871 FSW attending STI clinic.
Qualpro Syphicheck.
CS

Of the 4157 first-time attendees offered testing,

1117 (26.9%) accepted.

3% of refusals were due to unwillingness to undergo a finger prick due to perceived pain, discomfort, or other.
Acceptance of syphilis increase significantly by 8.0% during the POC protocol versus the standard.

Compared to a historical comparison, POC protocol allowed for significantly increased treatment coverage at 16.4%.

Munkhuu et al., 2009 [40]
Mongolia, urban (LM)
3850 ANC attendees in intervention and 3850 in control group.
SD Bioline Syphilis.
CRT

Some women received unnecessary treatment.

Some partners did not receive treatment; thus women were left at risk of reinfection.
Significantly higher number of women at intervention clinics than at control clinics were tested for syphilis at 1st visit (99% versus 79.6%). Similar result was found at 3rd trimester visit. As well, significantly higher number of infected women and their partners were treated at intervention clinics while lower number of congenital syphilis cases occurred.

Munkhuu et al., 2009 [38]
Mongolia, urban (LM)
246 ANC attendants
SD Bioline Syphilis
CS

All clients preferred receiving results the same day. Women were well satisfied with POC testing. Some cited reasons were time savings (88%), rapid results and no pain (77%), and counselling (42%).
Risk of intimate partner violence in case of discordant results between patient and husband.

Time consuming as provider is still expected to complete their “regular” tasks.

Concerns over procurement of POC tests.

Onwuezobe et al., 2011 [49]
Nigeria, urban (LM)
415 ANC attendees.
ACON Ultra Rapid Syphilis Test Strip.
CS

2.2%

Pai et al., 2012 [20]
India, rural (LM)
1066 ANC attendees
Determine
CS
99.3% (95% CI: 98.8, 99.8) preferred 3-in-1 testing to conventional strategies.
98% consented to testing.
96% completed study procedure.

Time it took for STPOC was 25 min (range 21–27) versus 45 min (range 40–47) for strategy.
32.1% (95% CI: 29.2–35.0%) reported prick pain.
In the same visit, women were tested and all positives were treated, unlike the traditional method.
0.2% (95% CI: 0–0.48)

Parthasarathy et al., 2013 [42]
India, urban and rural (LM)
330,000 FSW, 82,000 MSM, 10,000 IDU.
Immunochromatographic strip test (ICST) used from 2007 (study conducted from 2004 to 2007).
Retrospective analysis

In comparison to RPR testing, ICST used for screening increased from 7.4% in 2007 to 77.0% () in 2009. During the same period, the syphilis screening rates among clinic attendees increased from 9.0% to 21.6% ().

Read et al., 2013 [30]
Australia, urban (H)
98 MSM attending testing tent at a fair
Determine
CS

Participating rate of 0.23% (2010) and 0.26% (2011).

All testing tent clients were successfully contacted and treated and undertook partner notification.
1.02% (95% CI: 0.03–5.55)  

Revollo et al., 2007 [50]
Bolivia, urban (LM)
1594 postpartum women in hospital.
Determine
CS

7.2% (95% CI: 6.5–7.9)

Sabidó et al., 2009 [16]
Brazil, urban (UM)
60 high risk populations, 10 clinical and 2 lab staff.
Visitect Syphilis test.
CS
60% of clinical staff and 52% of clients preferred conventional over POC testing.
Acceptable to patient and laboratory technicians.
Staff: 9/12 found test instructions easy/very easy.

Clients: 69% of clients found time waiting for testing as short. 68% found that POC did not cause any discomfort.

95% of patients would recommend this test to others.
6/10 clinical staff lacked confidence in POC results.

Seguy et al., 2008 [31]
Guyana, rural (LM)
5618 miners
Determine
CS

80% of miners approached accepted testing.

6.4% (95% CI: 4.5, 9.1)

Smit et al., 2013 [43]
Tanzania, urban and rural (L).
2099 ANC attendees.
SD Bioline
CS

If POC testing is in ANC clinics, 82% of pregnant women would receive testing and treatment, compared to 16% if no POC is used.

Strasser et al., 2012 [32]
Uganda, rural and urban (L)

Zambia, rural and urban (LM).
Uganda: 14,540 ANC attendees; Zambia: 11,985 ANC attendees.
SD Bioline
CS study, with pre-post intervention design and retrospective review.

Uganda: 90.3% tested.

Zambia: 95.6% tested.
Uganda:

99.0% of those treated received STAT

9.9% of partners tested for syphilis.

Zambia: 95.8% of those treated received STAT

3.0% of partners tested for syphilis.

Significantly higher number of women tested and treated and their partners tested using POC intervention versus the traditional method.
Uganda: 5.3%.

Zambia: 9.2%.

Todd et al., 2011 [52]
Afghanistan, urban (L).
483 IDU.
SD Bioline
CS

2.1% (95% CI: 1.0–3.8)

Todd et al., 2008 [51]
Afghanistan, urban (L).
4452 ANC attendees.
Determine
CS

0%

Tucker et al., 2011 [33]
China, urban (LM).
2061 STI clinic attendees.
An immunochromatographic rapid treponemal test (Wantai, Beijing, China).
CS

Among those eligible, 81.6% agreed to syphilis rapid test.

Yang et al., 2013 [44]
China, rural (LM).
27,150 ANC attendees.
Acon Biotech.
CS

73.6% of those diagnosed received treatment.
0.39%

CS: cross-sectional; CC: case control; CRT: clustered randomized trial; STPOC: simultaneous triple point-of-care testing; STAT: same day testing and treatment; ICS: immunochromatographic strip; RPR: rapid plasma reagin; TPHA: Treponema  pallidum particle agglutination assay;L: low-income economies; LM: low-middle income economies; UM: upper-middle income economies; H: high-income economies [56]; STD: sexually transmitted diseases; FSW: female sex workers; ANC: antenatal clinic; MSM: men who have sex with men; TG: transgender; NR: not reported being member of a risk group; MV: mobile van; STI: sexually transmitted infections; MSW: male sex workers; IDU: injecting drug users.

Figure 1: Study selection following PRISMA guidelines.

4. Discussion

Using our framework, all the studies could be evaluated for quality of their reporting. Although evidence was primarily obtained from cross-sectional studies, outcomes were incompletely defined and reported in many studies. IROs such as preference, acceptability, feasibility, barriers and challenges, and impact outcomes were not well defined. Furthermore, the quality of evidence was impaired by unclear definitions/framework for IROs for diagnostic studies. Documentation was unclear and IROs were often expressed as percentages often without confidence intervals or without a denominator. Heterogeneity of definitions, lack of reporting of study methodology, and poor quality of reporting hampered drawing of meaningful results, although the bulk of evidence was reported in favour of POC strategies.

A determined effort needs to be made to improve the quality of assessment and reporting of IROs beyond accuracy. Impact outcomes have been clearly defined by the International Initiative for Impact Evaluation [53]. Often a vast majority of studies are cross-sectional and conducted to evaluate the feasibility of a POC strategy. Patient centered outcomes of acceptability, preference, and patient experience become equally important to justify the benefits of an alternative POC strategy to stakeholders or to scale up a project once feasibility is proven. A framework for reporting outcomes that has been proposed and utilized in this review could then be used. Clear definitions and transparency of IROs will not only help improve reporting of studies and quality of research with standardized reporting, but also facilitate comparability of findings for evaluation, across all settings. Standardized reporting will allow policymakers and providers to draw meaningful conclusions for their practice and policy. Reporting results with 95% CIs should also be emphasized. These help understand and appreciate the range of variability possible for an outcome. With these improvements, research on POC diagnostics beyond accuracy will not only be more meaningful but will also positively impact public health and clinical practice [54, 55].

5. Conclusion

In view of the growing interest in the elimination of mother to child transmission of HIV and syphilis promoted by WHO and UNICEF and an ever growing need for simultaneous rapid screening tests for HIV and related coinfections for at risk populations, POC screening strategies stand to become more complex in the near future. To report outcomes and metrics, we need to standardize and emphasize precise reporting, documentation, and measurement in publications. In this paper, we have presented a framework that defines a few of the commonly reported IROs beyond accuracy in research that involves POCs in syphilis. To improve evidence on metrics, a methodological framework that standardizes measures and parameters and ensures standardized collection, reporting, and documentation of data beyond accuracy must be emphasized. This move will greatly improve the quality of outcomes reporting in POC diagnostics and allow for enhanced documentation beyond traditional “accuracy outcomes.”

6. Summary

Implementation research outcomes documented in point-of-care (POC) diagnostic evaluations for syphilis are poorly defined, evaluated, and reported across studies. We defined operational definitions and proposed a framework to help evaluate the reporting of these outcomes. This framework for syphilis POC diagnostics research will facilitate standardized collection, documentation, synthesis of outcomes, and their inclusion in policy initiatives.

Conflict of Interests

The authors have no conflict of interests to report.

Acknowledgments

This work was supported by a Knowledge Synthesis Grant from the Canadian Institutes for Health Research (CIHR, KRS 102067). Dr. Pant Pai acknowledges the current support of the CIHR New Investigator Award 2010 and CIHR Operating Grant CIHR HBF 103210. The authors also acknowledge the data abstraction and searches performed by Sushmita Shivkumar and Rohit Vijh.

References

World Health Organization, Global Prevalence and Incidence of Selected Curable Sexually Transmitted Infections Overview and Estimates, World Health Organization, Geneva, Switzerland, 2001.

R. W. Peeling and A. Ronald, “Diagnostic challenges of sexually transmitted infections in resource-limited settings,” Future Microbiology, vol. 4, no. 10, pp. 1271–1282, 2009.

R. W. Peeling and H. Ye, “Diagnostic tools for preventing and managing maternal and congenital syphilis: an overview,” Bulletin of the World Health Organization, vol. 82, no. 6, pp. 439–446, 2004.

A. Herring, R. Ballard, D. Mabey, and R. W. Peeling, “Evaluation of rapid diagnostic tests: syphilis,” Nature Reviews Microbiology, vol. 4, no. 12, supplement, pp. S33–S40, 2006.

D. L. Kasper and T. R. Harrison, Harrison's Principles of Internal Medicine, McGraw-Hill Medical Publishing Division, New York, NY, USA, 16th edition, 2005.

R. W. Peeling, D. Mabey, A. Herring, and E. W. Hook III, “Why do we need quality-assured diagnostic tests for sexually transmitted infections?” Nature Reviews Microbiology, vol. 4, no. 12, pp. 909–921, 2006.

S. I. Egglestone and A. J. Turner, “Serological diagnosis of syphilis. PHLS syphilis serology working group,” Communicable Disease and Public Health, vol. 3, no. 3, pp. 158–162, 2000.

R. W. Peeling and D. C. Mabey, “Syphilis,” Nature Reviews Microbiology, vol. 2, no. 6, pp. 448–449, 2004.

J. D. Tucker, J. Bu, L. B. Brown, Y. Yin, X. Chen, and M. S. Cohen, “Accelerating worldwide syphilis screening through rapid testing: a systematic review,” The Lancet Infectious Diseases, vol. 10, no. 6, pp. 381–386, 2010.

Y. Jafari, R. W. Peeling, S. Shivkumar, C. Claessens, L. Joseph, and N. Pant Pai, “Are Treponema pallidum specific rapid and point-of-care tests for syphilis accurate enough for screening in resource limited settings? Evidence from a meta-analysis,” PLoS ONE, vol. 8, no. 2, Article ID e54695, 2013.

H. J. Schünemann, A. D. Oxman, J. Brozek et al., “GRADE: grading quality of evidence and strength of recommendations for diagnostic tests and strategies,” The British Medical Journal, vol. 336, no. 7653, pp. 1106–1110, 2008.

M. M. G. Leeflang, Y. J. Debets-Ossenkopp, C. E. Visser, and P. M. M. Bossuyt, “Meta-analysis of diagnostic test accuracy,” Clinical Infectious Diseases, vol. 43, no. 9, p. 1220, 2006.

L. Thabane, J. Ma, R. Chu et al., “A tutorial on pilot studies: the what, why and how,” BMC Medical Research Methodology, vol. 10, article 1, 2010.

K. J. Rothman, S. Greenland, and T. L. Lash, Modern Epidemiology, Wolters Kluwer Health/Lippincott Williams & Wilkins, Philadelphia, Pa, USA, 3rd edition, 2008.

PRISMA: Trasparent Reporting of Systematic Reviews and Meta-Analyses, 2014, http://www.prisma-statement.org

M. Sabidó, A. S. Benzaken, Ê. J. A. De Rodrigues, and P. Mayaud, “Rapid point-of-care diagnostic test for syphilis in high-risk populations, Manaus, Brazil,” Emerging Infectious Diseases, vol. 15, no. 4, pp. 647–649, 2009.

R. N. Bronzan, D. C. Mwesigwa-Kayongo, D. Narkunas et al., “Onsite rapid antenatal syphilis screening with an immunochromatographic strip improves case detection and treatment in rural South African clinics,” Sexually Transmitted Diseases, vol. 34, no. 7, supplement, pp. S55–S60, 2007.

D. Lee, C. Fairley, R. Cummings, M. Bush, T. Read, and M. Chen, “Men who have sex with men prefer rapid testing for syphilis and may test more frequently using it,” Sexually Transmitted Diseases, vol. 37, no. 9, pp. 557–558, 2010.

A. S. Benzaken, M. Sabidó, E. G. Galban et al., “Field evaluation of the performance and testing costs of a rapid point-of-care test for syphilis in a red-light district of Manaus, Brazil,” Sexually Transmitted Infections, vol. 84, no. 4, pp. 297–302, 2008.

N. P. Pai, J. Kurji, A. Singam et al., “Simultaneous triple point-of-care testing for HIV, syphilis and hepatitis B virus to prevent mother-to-child transmission in India,” International Journal of STD and AIDS, vol. 23, no. 5, pp. 319–324, 2012.

S. Dayan, A. Tekin, R. Tekin et al., “HBsAg, anti-HCV, anti-HIV 1/2 and syphilis seroprevalence in healthy volunteer blood donors in southeastern Anatolia,” Journal of Infection in Developing Countries, vol. 7, no. 9, pp. 665–669, 2013.

D. C. Mabey, K. A. Sollis, H. A. Kelly, et al., “Point-of-care tests to strengthen health systems and save newborn lives: the case of syphilis,” PLoS Medicine, vol. 9, no. 6, Article ID e1001233, 2012.

X. S. Chen, Y. P. Yin, C. Shen, et al., “Rapid syphilis testing uptake for female sex workers at sex venues in Southern China: implications for expanding syphilis screening,” PLoS ONE, vol. 7, no. 12, Article ID e52579, 2012.

F. M. Callegari, L. F. Pinto-Neto, C. J. Medeiros, C. B. Scopel, K. Page, and A. E. Miranda, “Syphilis and HIV co-infection in patients who attend an AIDS outpatient clinic in Vitoria, Brazil,” AIDS and Behavior, vol. 18, supplement 1, pp. 104–109, 2014.

X. S. Chen, Y. P. Yin, G. G. Liu et al., “Outreach syphilis testing services by different health providers to female sex workers in southern China,” PLoS ONE, vol. 8, no. 4, Article ID e60626, 2013.

S. G. García, F. Tinajeros, R. Revollo et al., “Demonstrating public health at work: a demonstration project of congenital syphilis prevention efforts in Bolivia,” Sexually Transmitted Diseases, vol. 34, no. 7, supplement, pp. S37–S41, 2007.

S. Gupte, C. Daly, V. Agarwal, S. B. Gaikwad, and B. George, “Introduction of rapid tests for large-scale syphilis screening among female, male, and transgender sex workers in Mumbai, India,” Sexually Transmitted Diseases, vol. 38, no. 6, pp. 499–502, 2011.

K. Manavi, G. Williams, and R. Newton, “The uptake of HIV and syphilis testing in a nurse-delivered service during Gay Pride events,” International Journal of STD and AIDS, vol. 23, no. 12, pp. 887–889, 2012.

S. Mishra, B. Naik, B. Venugopal et al., “Syphilis screening among female sex workers in Bangalore, India: comparison of point-of-care testing and traditional serological approaches,” Sexually Transmitted Infections, vol. 86, no. 3, pp. 193–198, 2010.

P. J. Read, V. Knight, C. Bourne et al., “Community event-based outreach screening for syphilis and other sexually transmissible infections among gay men in Sydney, Australia,” Sexual Health, vol. 10, no. 4, pp. 357–362, 2013.

N. Seguy, M. Denniston, W. Hladik et al., “HIV and syphilis infection among gold and diamond miners—Guyana, 2004,” West Indian Medical Journal, vol. 57, no. 5, pp. 444–449, 2008.

S. Strasser, E. Bitarakwate, M. Gill et al., “Introduction of rapid syphilis testing within prevention of mother-to-child transmission of HIV programs in Uganda and Zambia: a field acceptability and feasibility study,” Journal of Acquired Immune Deficiency Syndromes, vol. 61, no. 3, pp. e40–e46, 2012.

J. D. Tucker, L. Yang, B. Yang et al., “A twin response to twin epidemics: integrated HIV/syphilis testing at STI clinics in South China,” Journal of Acquired Immune Deficiency Syndromes, vol. 57, no. 5, pp. e106–e111, 2011.

P. E. Campos, A. L. Buffardi, M. Chiappe et al., “Utility of the determine syphilis TP rapid test in commercial sex venues in Peru,” Sexually Transmitted Infections, vol. 82, supplement 5, pp. v22–v25, 2006.

A. S. Benzaken, E. G. García, J. C. G. Sardinha, J. C. Dutra Jr., and R. Peeling, “Rapid tests for diagnosing syphilis: validation in an STD clinic in the Amazon Region, Brazil,” Cadernos de Saude Publica, vol. 23, no. 3, pp. S456–S464, 2007.

N. R. Dlamini, R. Phili, and C. Connolly, “Evaluation of rapid syphilis tests in KwaZulu-Natal,” Journal of Clinical Laboratory Analysis, vol. 28, no. 1, pp. 77–81, 2014.

L. Juárez-Figueroa, F. Uribe-Salas, S. García-Cisneros, M. Olamendi-Portugal, and C. J. Conde-Glez, “Evaluation of a rapid strip and a particle agglutination tests for syphilis diagnosis,” Diagnostic Microbiology and Infectious Disease, vol. 59, no. 2, pp. 123–126, 2007.

B. Munkhuu, T. Liabsuetrakul, E. McNeil, and R. Janchiv, “Feasibility of one-stop antenatal syphilis screening in ulaanbaatar, mongolia: women and providers perspectives,” Southeast Asian Journal of Tropical Medicine and Public Health, vol. 40, no. 4, pp. 861–870, 2009.

P. J. Garcia, C. P. Cárcamo, M. Chiappe et al., “Rapid syphilis tests as catalysts for health systems strengthening: a case study from Peru,” PLoS ONE, vol. 8, no. 6, Article ID e66905, 2013.

B. Munkhuu, T. Liabsuetrakul, V. Chongsuvivatwong, E. McNeil, and R. Janchiv, “One-stop service for antenatal syphilis screening and prevention of congenital syphilis in Ulaanbaatar, Mongolia: a cluster randomized trial,” Sexually Transmitted Diseases, vol. 36, no. 11, pp. 714–720, 2009.

A. E. Miranda, E. R. Filho, C. R. Trindade et al., “Prevalence of syphilis and HIV using rapid tests among parturients attended in public maternity hospitals in Vitória, State of Espírito Santo,” Revista da Sociedade Brasileira de Medicina Tropical, vol. 42, no. 4, pp. 386–391, 2009.

M. R. Parthasarathy, P. Narayanan, A. Das, A. Gurung, P. Prabhakar, and T. Wi, “Integrating syphilis screening in a large-scale HIV prevention program for key populations: the Avahan experience from India,” Journal of Infection in Developing Countries, vol. 7, no. 6, pp. 484–488, 2013.

P. W. Smit, D. Mabey, J. Changalucha, et al., “The trade-off between accuracy and accessibility of syphilis screening assays,” PLoS ONE, vol. 8, no. 9, Article ID e75327, 2013.

L. G. Yang, J. D. Tucker, F. Y. Liu, et al., “Syphilis screening among 27,150 pregnant women in South Chinese rural areas using point-of-care tests,” PLoS ONE, vol. 8, no. 8, Article ID e72149, 2013.

E. S. Amadi, C. E. Ononiwu, N. Aballa, S. A. Oladimeji, F. A. Aneke, and D. C. Ugwuoru, “Prevalence of syphilis among patients in a dental clinic in Enugu, Nigeria,” Pakistan Journal of Medical Sciences, vol. 26, no. 1, pp. 59–61, 2010.

M. Elhadi, A. Elbadawi, S. Abdelrahman, et al., “Integrated bio-behavioural HIV surveillance surveys among female sex workers in Sudan, 2011-2012,” Sexually Transmitted Infections, vol. 89, supplement 3, pp. iii17–iii22, 2013.

I. Hurtado, I. Alastrue, P. García de Olalla, D. Albiach, M. Martín, and S. Pérez-Hoyos, “Preventive intervention in venues for interaction used by men who have sex with men,” Gaceta Sanitaria, vol. 24, no. 1, pp. 78–80, 2010.

M. Lahuerta, M. Sabidó, F. Giardina et al., “Comparison of users of an HIV/syphilis screening community-based mobile van and traditional voluntary counselling and testing sites in Guatemala,” Sexually Transmitted Infections, vol. 87, no. 2, pp. 136–140, 2011.

I. A. Onwuezobe, E. A. Ochang, A. Umoiyoho, E. A. Bassey, and E. M. Umoffia, “Prevalence of syphilis seropositivity in antenatal clinic clients in a teaching hospital in South-South region of Nigeria,” Asian Pacific Journal of Tropical Disease, vol. 1, no. 1, pp. 21–23, 2011.

R. Revollo, F. Tinajeros, C. Hilari et al., “Maternal and congenital syphilis in four provinces in Bolivia,” Salud Publica de Mexico, vol. 49, no. 6, pp. 422–428, 2007.

C. S. Todd, M. Ahmadzai, F. Atiqzai et al., “Seroprevalence and correlates of HIV, syphilis, and hepatitis B and C virus among intrapartum patients in Kabul, Afghanistan,” BMC Infectious Diseases, vol. 8, article 119, 2008.

C. S. Todd, A. Nasir, M. R. Stanekzai et al., “Prevalence and correlates of HIV, syphilis, and hepatitis B and C infection and harm reduction program use among male injecting drug users in Kabul, Afghanistan: a cross-sectional assessment,” Harm Reduction Journal, vol. 8, article 22, 2011.

International Initiative for Impact Evaluation, 2012, http://www.3ieimpact.org/en/

G. H. Guyatt, A. D. Oxman, H. J. Schünemann, P. Tugwell, and A. Knottnerus, “GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology,” Journal of Clinical Epidemiology, vol. 64, no. 4, pp. 380–382, 2011.

C. Barbui, T. Dua, M. van Ommeren et al., “Challenges in developing evidence-based recommendations using the grade approach: the case of mental, neurological, and substance use disorders,” PLoS Medicine, vol. 7, no. 8, Article ID e1000322, 2010.

The World Bank, Country and Lending Groups, 2013, http://data.worldbank.org/about/country-classifications/country-and-lending-groups.