We evaluated a low-cost virological failure assay (VFA) on plasma and dried blood spot (DBS) specimens from HIV-1 infected patients attending an HIV clinic in Harare. The results were compared to the performance of the ultrasensitive heat-denatured p24 assay (p24). The COBAS AmpliPrep/COBAS TaqMan HIV-1 test, version 2.0, served as the gold standard. Using a cutoff of 5,000 copies/mL, the plasma VFA had a sensitivity of 94.5% and specificity of 92.7% and was largely superior to the VFA on DBS (sensitivity = 61.9%; specificity = 99.0%) or to the p24 (sensitivity = 54.3%; specificity = 82.3%) when tested on 302 HIV treated and untreated patients. However, among the 202 long-term ART-exposed patients, the sensitivity of the VFA decreased to 72.7% and to 35.7% using a threshold of 5,000 and 1,000 RNA copies/mL, respectively. We show that the VFA (either on plasma or on DBS) and the p24 are not reliable to monitor long-term treated, HIV-1 infected patients. Moreover, achieving acceptable assay sensitivity using DBS proved technically difficult in a less-experienced laboratory. Importantly, the high level of virological suppression (93%) indicated that quality care focused on treatment adherence limits virological failure even when PCR-based viral load monitoring is not available.
In 2011, 34 million people were estimated to be living with HIV/AIDS of whom 69% reside in sub-Saharan Africa [
Despite the unquestionable success of internationally funded ART access programs, accumulating evidence suggests that sustaining patients on treatment and assuring quality of care is a formidable next challenge. In particular, the monitoring of the ART response based on the current clinicoimmunological parameters may be associated with prolonged virological failure, accumulation of HIV drug resistance (HIVDR) mutations, and inappropriate switching to second line treatment [
Patient plasma HIV-1 viral load has been demonstrated to be the most sensitive and reliable marker of ART failure [
Despite this recommendation, the majority of clinical settings in sub-Saharan Africa cannot afford routine or even targeted VL monitoring because of the high price and complexity of current commercial PCR-based assays. The four FDA-approved viral load assays from Roche, Siemens, Abbott, and Biomérieux typically cost between US$ 40 and 125/test with prices varying as a function of the region, volume of samples, and negotiation with the supplier [
Several initiatives have been taken to provide alternatives for VL testing in resource limited settings. These include the measurement of indirect markers of viral replication such as the expression of the activation marker CD38 on CD8 lymphocytes by flow cytometry [
In light of the above, our team developed the Affordable Resistance Test for Africa (ARTA) virological failure assay (VFA):
So far, the VFA has been evaluated on purposefully selected panels of HIV-1 infected plasma and in laboratory-controlled conditions [
Performance outcomes, benefits, and operational challenges of the alternative assays in this clinical setting are discussed.
Newlands Clinic is a family centered, nurse based HIV care and treatment center based in Harare, Zimbabwe, which was founded in 2004. It is a part of the coordinated public-private partnership between the Ministry of Health and Child Care and provides access to care and treatment to over 4000 HIV-1 infected patients from marginalized communities within urban and periurban Harare and Chitungwiza. Patient care follows the national HIV treatment guidelines of Zimbabwe [
Matched plasma and DBS were collected at single time points from 202 HIV-1 infected patients participating in a long-term observational cohort study [
Seven mL of EDTA blood was drawn from all patients. Fifty
For each patient, two aliquots of plasma were shipped on dry ice to the reference laboratory of Witwatersrand University, Johannesburg, South Africa, where reference VL determination was performed (see the following), according to international quality standards and procedures. The remaining plasma aliquots were kept on site for VFA and p24 testing (see the following).
p24 antigen concentration was measured on all plasma specimens using the Ultrasensitive p24 Ag ELISA kit (Perkin-Elmer Life Sciences, Boston, MA), following the procedure previously described [
The optimal cutoff to define virological failure was determined using the ROC curve (see the following).
VLref was measured using the COBAS AmpliPrep/COBAS TaqMan HIV-1 test, version 2.0 (Roche Molecular Diagnostic Systems, Branchburg, NJ), as per manufacturer’s instructions. The assay has a detection limit of 20 RNA copies/mL.
The VFA is based on real-time PCR targeting the long terminal repeat domain (LTR) of HIV-1 [
Nucleic acid extractions from DBS were performed using an initial off-board lysis step, using two DBS (estimated 50
Previous evaluations [
Three aliquots from a well-characterized plasma specimen, with VL > 7
Viral load determination of clinical samples was performed as previously described [
The calculation of the VFA cost included the price of the equipment, reagents (including shipment), consumables, and labour. The cost of the VFA in Zimbabwe was compared to the cost of the VFA as performed in the reference laboratory in South Africa.
In treated patients, immunological failure was defined as CD4 count measured at month 36: (1) equal or lower than baseline values; (2) lower than 100 cells/uL; or (3) lower than 50% of treatment peak value (measured at month 12 or month 24), according to the WHO guidelines for ART monitoring [
Undetectable viral load results were given a value representing the average between 0 and the lower detection limit for each test: VLref = 10 copies/mL, VFA on plasma = 125 copies/mL, and VFA on DBS = 500 copies/mL. VLref, VFA, and p24 results were log-transformed prior to quantitative analysis to approach normal distribution.
Data analysis was performed using both the previous (5,000 RNA copies/mL) and current (1,000 RNA copies/mL) cutoff values to define virological failure according to WHO guidelines [
Data were analyzed using SPSS version 20.0 (IBM Corporation, Armonk, NY) and Graph Pad Prism version 6.00 for Windows (GraphPad Software, San Diego CA).
One hundred ART-naive and 202 ART patients treated for at least 36 months were included in the study. There were no significant differences in age and gender between the two patient groups (Table
Characteristics of the study population.
ART-naive ( |
ART-treated ( |
| |
---|---|---|---|
Age (years) | NS | ||
18–29 | 19 (19) | 12 (5.9)¥ | |
30–49 | 74 (74) | 153 (75.7) | |
>50 | 7 (7) | 35 (17.3) | |
Gender | NS | ||
Male | 32 (32) | 72 (35.6) | |
Female | 68 (68) | 130 (64.3) | |
HIV-1 subtypes | Not available | all subtype C | |
Hemoglobin (g/dL, mean, [min, max]) | 12.4 [5.9–17.30] | 12.9 [7.40–16.9]* | 0.055 ( |
CD4 count (cells/mL) |
| ||
≤200 cells/mL | 17 (17.1) | 18 |
|
251–350 cells/mL | 23 (23.2) | 58 (29.2) | |
351–500 cells/mL | 28 (28.2) | 62 (31.3) | |
>500 cells/mL | 31 (31.3) | 56 (28.3) | |
Immunological failure | Not applicable | 38 (18.8) | |
Positive VL/virological failure | |||
Ref VL > 5000 cp/mL | 81 (81) | 11 (5.4) |
|
Ref VL > 1000 cp/mL | 92 (92) | 14 (6.9) |
|
Positive p24 Ag (VLp24 > 3 pg/mL) | 51 (51) | 36 (17.8)* |
|
Among the 100 ART-naive patients, 81 had a VLref > 5,000 copies/mL and 92 a VLref > 1,000 copies/mL. Among the ART-treated patients these figures were 11 (5.4%) and 14 (6.9%), respectively.
The standard curve was built from testing serial dilution of one well-characterized plasma aliquots (data not shown). Mean Ct-values and SD corresponding to 5,000 and 1,000 RNA copies/mL were extrapolated from the equations of three individual standard curves. Thresholds Ct-values defined as [mean ± 2SD] were calculated and equaled to
A total of 302 specimens had VLref results. Three hundred one plasma specimens were tested with the p24 assay and 300 with the VFA on plasma. Two hundred ninety-nine DBS specimens were tested with the VFA.
Area Under the ROC Curve (AUC, data not shown) indicated that the capacity to predict virological failure as per VLref was the highest in the plasma VFA (AUC = 0.980 and 0.981 when using a threshold of, resp., 5,000 or 1,000 RNA copies/mL as per VLref) followed by the DBS VFA (AUC = 0.910, using the threshold of 5,000 RNA copies/mL only).
Overall, the percentage of samples correctly classified was the highest with plasma VFA, (93.3% using 5,000 RNA copies/mL and 93.0% using 1,000 RNA copies/mL, as thresholds, Table
Performance of the plasma VFA, DBS VFA, and p24 to identify virological failure using VLref as the reference assay.
All patients (
Number tested | Correctly classified | Misclassified | Undercalled | Overcalled | Sensitivity | Specificity | |
---|---|---|---|---|---|---|---|
VFA plasma (CO = 5,000 cp/mL) | 300 | 280 (93.3%) | 20 (6.6%) | 5/20 | 15/20 | 94.5% | 92.7% |
VFA plasma (CO = 1,000 cp/mL) | 300 | 279 (93.0%) | 21 (7%) | 21/21 | 0/21 | 80.1% | 100% |
VFA DBS (CO = 5,000 cp/mL) | 299 | 262 (87.6%) | 37 (12.3%) | 35/37 | 2/37 | 61.9% | 99.0 |
p24 (CO = 5,000 cp/mL) | 301 | 222 (73.5%) | 79 (26.2%) | 42/79 | 37/79 | 54.3% | 82.3% |
p24 (CO = 1,000 cp/mL) | 301 | 216 (71.7%) | 85 (28.2%) | 52/85 | 33/85 | 49.0% | 83.0% |
Treated patients (
Number tested | Correctly classified | Misclassified | Undercalled | Overcalled | Sensitivity | Specificity | |
---|---|---|---|---|---|---|---|
VFA plasma (CO = 5,000 cp/mL) | 200 | 194 (97.0%) | 6 (3.0%) | 3/6 | 3/6 | 72.7% | 98.0% |
VFA plasma (CO = 1,000 cp/mL) | 200 | 191 (95.5%) | 9 (4.5%) | 9/9 | 0/9 | 35.7% | 100% |
VFA DBS (CO = 5,000 cp/mL) | 199 | 189 (95.0%) | 10 (5.0%) | 10/10 | 0/10 | 9.0% | 100% |
p24 (CO = 5,000 cp/mL) | 201 | 166 (82.5%) | 35 (17.5%) | 5/35 | 30/35 | 54.7% | 84.2% |
p24 (CO = 1,000 cp/mL) | 201 | 163 (81.0%) | 48 (19.0%) | 8/38 | 30/38 | 42.8% | 83.9% |
The AUC of the p24 assay was 0.714 using the threshold of 1,000 RNA copies/mL and 0.715 using the threshold of 5,000 copies/mL, reflecting the lower diagnostic capacity of this assay as compared to the VFA (data not shown). The Youden index indicated that the optimal cutoff to identify virological failure using the p24 assay was 3 pg/mL for the threshold of 5,000 RNA copies/mL and 2.65 pg/mL for the threshold of 1,000 RNA copies/mL as per VLref. These two cutoff values were used for the calculation of sensitivities and specificities. The p24 assay demonstrated a poor capacity to identify VLref > 1,000 RNA copies/mL (sensitivity = 54.3%) or VLref > 5,000 copies/mL (sensitivity = 49.0%) with more than a quarter of the samples being misclassified using either threshold (Table
Although the VFA is not designed for quantitative VL determination, we conducted a Bland Altman analysis to further explore the agreement between measurements with plasma VFA and the VLref (Figure
Bland-Altman analysis of plasma using the VLref as the gold standard.
Using a threshold of 5,000 RNA copies/mL, the plasma VFA misclassified 5 samples as false negative and 15 samples as false positive (see Table
Characteristics of correctly classified versus misclassified virological failures as per the VLref and using a cut-off of 1000 copies/mL (
Correctly classified |
Misclassified |
| |
---|---|---|---|
Treated patients | 5 (5.8%) | 9 (42.8%) |
|
ART-naive | 80 (94.1%) | 12 (57.1%) | |
Male/female ratio | 0.63 | 0.50 | 0.642¥ |
Age (years) | 36.5 | 37.8 | 0.590
|
CD4 count (mean cells/uL)* | 364 | 411 | 0.367
|
Hb (mean g/dL)** | 12.43 | 12.76 | 0.557
|
VLref (mean |
4.90 | 3.82 |
|
Conversely, among a total of 210 samples with VLref ≤ 5,000 RNA copies/mL, VLref values were significantly higher in the 15 samples classified as false positive using the plasma VFA as compared to the 195 correctly classified samples (true negative, VLref = 3.14
In order to explore the utility of the three assays beyond the identification of virological failures, we compared levels of viral load as per VLref and plasma VFA as well as levels of p24 concentration between groups of patients categorized as a function of WHO-defined immunological failure. Both VLref and plasma VFA quantitative measurements were significantly higher in patients with immunological failure (
The cost of the VFA in plasma was calculated based on one run of 19 plasma samples, including positive and negative controls. Labor costs were based on an average of 2.5 hours spent by one laboratory technician per test run. Overall, the cost of the VFA in Zimbabwe (US$ 30.31) was very similar to South Africa (US$ 28.5, Table
Assay costing.
Cost per samples | Wits laboratory |
Newlands clinic |
---|---|---|
Labour cost | 0.2 | 1.34 |
Fixed instrument expense | 0.3 | 0.74 |
Reagent + consumable cost | 28 | 28.23 |
Total cost |
|
|
This study evaluated the performance of an alternative, open-platform, low-cost VL assay (VFA) in plasma and DBS as compared to the p24 assay, using the commercial viral load assay from Roche (CAP/CTM) as the gold standard for the identification of virological failure in a group of treated and ART-naive HIV-1 infected patients. The evaluation was done in a local setting and included samples from patients treated for at least 36 months, in whom the performance of an alternative viral load assay is most relevant to study.
The data indicate that the plasma VFA showed the best performance in identifying virological failures as compared to the DBS VFA and the p24 assay, using a cutoff of either 5,000 or 1,000 RNA copies/mL as per VLref. AUC, sensitivities, and specificities of the plasma VFA were comparable to previous reports on similar panels of samples [
Overall, the p24 assay had a poor capacity to classify samples into virological or nonvirological failure, regardless of the cutoff used. More than a quarter of the samples tested were misclassified mostly due to overcalling of the VLref (false positive). Despite initial observations indicating that heat-denatured p24 is a good alternative to HIV-1 RNA load [
The sensitivity of VFA in plasma and DBS decreased substantially in the group of long-term treated patients. A plausible explanation is that VL associated with virological failure might be relatively lower than those measured in ARV-naive individuals, with greater odds of being classified as false positives by the VFA. Previous evaluations of the VFA were conducted on panels of samples artificially composed to cover very high to low levels of viral load, regardless of ART exposure [
The present findings do not support the use of the VFA (either on plasma or on DBS) or the p24 for the reliable monitoring of long-term virological response to ART. To date, no consensus has emerged on acceptable rates of misclassification for the field use of alternative VL assays. However, it is our feeling that missing more than 20% of patients failing their treatment is unacceptable, given the fact that false positive patients are not eligible for a retesting before the next (bi-)annual visit [
Interestingly and in contrast with VLref and VFA, there was no association between higher p24 levels and more advanced disease stages as defined by CD4 count. Although intrapatient longitudinal changes of p24 concentration have been shown to have some value in identifying patients at risk of disease progression [
Our observations indicate that the VFA technology transfer to a less-sophisticated laboratory was feasible in this setting. Procurement of reagents and consumables is one of the main barriers limiting adequate operation of medical laboratories in sub-Saharan Africa. Hence, careful consideration and planning are needed when implementing a new assay in the field; especially when this assay is based on an open platform. In this case, instruments and reagents were procured relatively fast and at a reasonable price, albeit with significant logistical and administrative assistance from the reference laboratory in South Africa. Importantly, this report indicates that the price of US$ 30.31/test is only 12% cheaper than the discounted NucliSENS EasyQ HIV-1 v1.2 previously reported for a reference laboratory in South Africa [
The duration of the training was brief (two weeks), since the operator had received previous training in molecular biology techniques, which significantly alleviated the learning curve. At the end of the training, it took 2.5 hours for the laboratory technician to process one test run of 19 samples, which is the average time required for this type of assay.
The high level of viral suppression rate at month 36 on treatment of PASER patients (93%) was achieved in the absence of any sensitive virological monitoring and underscores the excellent clinical practice provided by the site. In addition, the high percentage of patient retention (82%) and the previously reported low level of acquired HIV-DR at month 12 [
This report demonstrates the feasibility of implementing a PCR-based VL assay in a less-experienced, low throughput laboratory, with the plasma VFA results comparable to those obtained in accredited reference laboratory in South Africa and in The Netherlands. Adequate performance of the VFA on DBS was however not achieved in this setting, indicating that using this type of specimen for molecular assays requires more technical proficiency as compared to liquid plasma.
This study indicates that neither the VFA nor the p24 are reliable tools to identify virological failure to ART in long-term treated patients, who are mostly virologically suppressed. The routine use of affordable VL testing is largely advocated to preserve the cost-efficiency of ART programmes in resource limited settings [
The good clinical outcome of treated patients at this site suggests that despite the absence of conventional virological monitoring, quality clinical care, focused on patient support for treatment adherence, contributes to limit virological failure, thereby preventing the emergence of HIV drug resistance.
The authors declare that there is no conflict of interests regarding the publication of this paper.
The authors thank Dr. Rob Schuurman and Dr. Susan C. Aitken (UMCU, Utrecht, The Netherlands) for their useful comments on the laboratory data analysis component and for providing control reagents and samples; Mr. Bram Prins (AIGHD, Amsterdam, The Netherlands) for providing support in data management; and Mr. John Dekker (PharmAccess, Amsterdam, The Netherlands) for his logistical assistance in the procurement of laboratory instruments. They would also like to acknowledge Dr. Sandra Bote, Dr. Cordelia Kunzekwenyika, Evelyn Matanga, Farai Rusinga, and Rudo Dzapasi (Newlands Clinic, Harare, Zimbabwe) who were involved in enrolling the patients and administering and signing of consent forms. This work was funded by the Dutch Aidsfonds (Grant no. 2006083).