A Critical Evaluation of Carotid Duplex Scanning in the Diagnosis of Significant Carotid Artery Occlusive Disease

1Department of Surgery, The Ohio State University Wexner Medical Center, 376 West 10th Avenue, Prior Hall 7th Floor, Columbus, OH 43210, USA 2Vascular Laboratory, The Ohio State University Wexner Medical Center, 376 West 10th Avenue, Prior Hall 7th Floor, Columbus, OH 43210, USA 3Division of Vascular Disease and Surgery, Department of Surgery, The Ohio State University Wexner Medical Center, 376 West 10th Avenue, Prior Hall 7th Floor, Columbus, OH 43210, USA


Introduction
Over 795,000 people are afflicted by stoke every year in the United States [1] of which 15-20% are due to extracranial large vessel atherosclerosis [2].The internal carotid artery is the source of over 75% of the strokes due to extracranial large vessel disease [3].Carotid endarterectomy has been shown to be effective at preventing stroke in both symptomatic and asymptomatic carotid stenosis [4,5].Imaging of internal carotid artery stenosis is necessary for interventions including surgery and although multiple modalities are available, carotid duplex (CD) scanning has become the method of choice [6,7].
CD has become the primary method of evaluating patients with symptoms related to the cerebrovascular system and those who are asymptomatic but have risk factors or physical examination findings suggestive of extracranial arterial disease [8].While the accuracy reported has generally improved, the variability in equipment, technology, and physician reader capabilities as well as test protocols and criteria for diagnosing hemodynamically significant disease requires periodic reassessment.Contrast angiography has historically been the gold standard to compare with CD, but utilization of this imaging modality has decreased in patients with significant carotid disease due to potential complications and cost; therefore other contrast imaging modalities must be utilized.There has also been a gradual shift away from confirming significant disease by CD in patients who are candidates for intervention by contrast arteriography due to potential complications and cost [9].Therefore, the validation of the accuracy of CD has become even more critical because of the elimination of the intermediary step of a contrast study [6].In an era of cost containment and eliminating even a small risk of contrast induced complications, quality assurance on a regular basis to confirm overall accuracy, sensitivity, and specificity of CD is vital in each VL.
We have previously reported our accuracy with color coded Doppler ultrasound and more recently using duplex scanning in 2010 [10,11].Certainly the technology has changed dramatically since our first report.To continue to offer the best care for our patients we feel it is vital that we continuously reassess our CD in comparison to contrast imaging.The objective of this study was to review our CD examinations, compare them to current "gold standards, " and arrive at the best criteria for diagnosing extracranial carotid disease using standard statistical methods.

Methods
All CD scan results were prospectively maintained in our vascular lab Vascubase software (Consensus Medical Corporation Inc., Canada).Carotid duplex scans of patients over the age of 18 years from June 1, 2008, to December 31, 2013, at The Ohio State University Wexner Medical Center were available for evaluation.A total of 134 scans were included from this time period if a "gold standard" comparison study had been conducted within 30 days of the CD.Angiographic studies that were considered for comparison included contrast arteriography, computed tomographic arteriography, and magnetic resonance angiography.Exclusion criteria included CD that had missing segments, had missing velocities, or were not able to reliably measure percentage stenosis.Scans of patients who had arteriograms performed outside our institution or beyond thirty days after CD or patients with restenosis after any intervention or previous intervention were also excluded from consideration.Our previous lab validation used the existing criteria for grading internal carotid stenosis that are shown in Table 1 and which we chose to use again for this study [11].
Carotid duplex ultrasonography examinations were performed using several scanners including Logiq 9E (General Electric, Milwaukee, Wisconsin), Philips iU22 (Philips Biomedical Systems, Bothell, Washington), and Zonare (Zonare Inc., Mountain View, California) scanners.A standard 7.5-mHz linear array transducer was used with a 5-mHz Doppler frequency.Twelve separate registered vascular sonographers performed the tests, which were read by five vascular surgeons in an Intersocietal Commission for the Accreditation of Laboratories (ICAL) accredited vascular laboratory at The Ohio State University Heart & Vascular Center.A standard protocol was used at the two hospitals (Ross Heart Hospital and Ohio State University East) with both longitudinal and transverse views of the ICA, common carotid artery (CCA), and the external carotid artery bilaterally.Both gray scale and color images were obtained and spectral Doppler flow was recorded for physician interpretation later.Our protocol also included aligning the cursor parallel to the vessel wall, obtaining waveforms using a small sample volume ideally placed in the center of the blood flow, and attempting to keep the Doppler angle at 60 degrees or less.Peak ICA systolic velocities (PSV), end diastolic ICA velocities (EDV), and internal to common carotid artery PSV ratios (ICA/CCA ratio) were then recorded.
Correlation with a gold standard, in this case arterial digital subtraction arteriography, computed tomographic arteriography, and magnetic resonance angiography, was performed in a blinded fashion by the senior author as a regular quality assurance activity and data was entered into a database (Vascubase, Consensus Medical Corporation Inc., Canada).For arteriography, the diameter of the lumen at the most severely stenotic site of the ICA was measured with calipers to a fraction of a millimeter and used to calculate percentage diameter stenosis based on North American Symptomatic Carotid Endarterectomy Trial (NASCET) criteria [6].

Statistical Analysis. Microsoft Excel (MS Office 2010)
program was used to collect all data and create scatter graphs for the institutional database and two-by-two tables for >50% and >70% stenosis criteria were created (Tables 2(a) and 3(a)).In Figure 1, the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and overall accuracy (OA) were then calculated (Table 4).
Receiver operating characteristic (ROC) curves were constructed with the Vascubase database on those who had complete velocity data with exclusion of totally occluded arteries or where incomplete data existed.Receiver operating characteristic (ROC) curves were constructed to assess the accuracy of ultrasonographic criteria for detecting >50% stenosis and >70% angiographic defined stenosis.The overall accuracy of CD was expressed in terms of the area under the ROC curve (AUROC), ranging from 0.5 (poor) to 1.0 (perfect).Vascubase program was used to generate the fitted curves and the areas under the curves.

Results
Seven eligible internal carotid arteries were totally occluded leaving a total of 124 internal carotid arteries with recorded velocities, a complete CD, and corresponding arteriography or other comparative study within 30 days at our institution and were compiled in our database.Scatter plots were constructed for the peak systolic and end diastolic velocities versus percentage of stenosis based on gold standard angiography.Stenosis based upon arteriographic modalities ranged from 0 to 99% (complete occlusion was excluded).The median stenosis was 79.8% (SD 26.4%).The PSV ranged from 23 to 638 cm/s and had a median of 315 cm/s (SD 159.1 cm/s).The EDV ranged from 0 to 320 cm/s with a median of   Using ROC curve analysis for the 50-69% category, the best criterion for PSV was 197 cm/sec with a sensitivity of 88%, specificity of 93%, NPV of 73%, and PPV 97% (AUROC 0.953 [95% CI 0.909-0.996]).The best criterion for EDV in the 50-69% category was 69 cm/sec with a sensitivity of 81%,  a PSV of 269 cm/sec with a sensitivity of 95% and a specificity of 90% with a NPV of 93% and a PPV of 93% (AUROC 0.931 [95% CI 0.879-0.983]).In the 70-99% category, the best fit was an EDV of 78 cm/sec with a sensitivity of 90%, specificity of 86%, NPV 86%, and PPV of 90% (AUROC 0.923 [0.872-0.973]).

Discussion
The measurement of hemodynamic parameters with CD has become the standard in reporting diameter stenosis rather than using angiographic examinations, but the PSV and EDV must be continuously validated and compared with a "gold standard" such as standard angiography or other imaging studies to assess its accuracy.Our accuracy using our previously validated criteria (Table 1) for grading carotid stenosis by CD is outlined in Tables 2 and 3.In addition, we performed ROC analysis and our findings are shown in Figures 2 and 3.
There are several statistical measures to evaluate the clinical value of CD testing.Sensitivity of a test indicates the ability of a test to detect disease and therefore it is the proportion of the test identified as positive in patients where there is disease present.Specificity on the other hand refers to the ability of a diagnostic test to indicate the absence of disease and therefore it is that proportion identified as negative when there is no disease present.There is always a trade-off between sensitivity and specificity when analyzing a diagnostic test as they are generally inversely proportional to one another.Overall accuracy of a test is the statistical analysis that allows the comparison of the interplay of sensitivity and specificity.Compared to our previous report, there are a few minor changes and it is commonly accepted that there will be some differences over a period of time even when similar personnel or equipment is used [12].If the CD is used as a screening test then a high degree of sensitivity and NPV is needed.In other words, we are willing to accept a few more false positive studies.However, in most cases where we are relying on CD alone without any confirmatory contrast studies, good specificity and PPV are necessary.Our analysis shows a specificity of 88% and a PPV of 92.5% using our existing criteria for >70% stenosis [13].
A receiver operating characteristic (ROC) curve is plotting the sensitivity of the test versus 1 − specificity on a graph.Theoretically, if both sensitivity and specificity of a test were to be equal to 1, the test would be perfect.In such a case, the ROC curve would start at the 0 point on both the -axis and the -axis and then go up the -axis to 0 (specificity) and 1 (sensitivity) and then move horizontally along the -axis to the 1 point (both specificity and sensitivity) (Figures 2 and 3).It is therefore evident that any test with a ROC curve as close to this "ideal" would be excellent.Next, one can quantify the exact performance of the test, in this case CD, by calculation of the "area under the curve" or AUROC.In the ideal test AUROC would be equal to 1 and as our results demonstrate that, although neither the PSV nor the EDV for the 50-69% or the 70-99% category is perfect, the PSV for both categories is a better measure to use.When compared to our previous validation, our AUROC has improved with PSV for 50-69% increasing from 0.86 to 0.95.The EDV for 50-69% increased from 0.86 to 0.95.For 70-99%, the ESV AUROC increased from 0.74 to 0.93 and increased from 0.74 to 0.92 for EDV.
In a 10% random sample of the ICAL database, an attempt was made to review variability of diagnostic criteria among 152 accredited VLs.The criteria used by the VL were Society of Radiologists in Ultrasound (SRU) based criteria (27%) [14], Bluth based criteria (23%) [15], University of Washington criteria (21%) [16], hybrid or unreferenced criteria (20.4%), and home-grown criteria in 3.9% [17].A panel of radiologists, vascular surgeons, and other vascular disease specialists reviewed the literature and published their recommendations regarding standardization of criteria for interpretation of CD tests.The SRU criteria recommended a PSV > 125 cm/sec and <230 cm/sec for indicating a 50-69% and >230 cm/sec for a >70% diameter stenosis, respectively.They also observed that additional parameters such as EDV > 40 cm/s and an ICA/CCA PSV ratio of >2 were helpful in estimating a 50-69% stenosis.Similarly, in estimating 70-99% stenosis adjunctive parameters such as an ECV > 100 m/sec and an ICA : CCA ratio of >4 could be used [14].
In a recent report, AbuRahma and colleagues tested the SRU criteria in comparing CD and angiographic findings [8].They contended that although the criteria for diagnosing a >70% stenosis were appropriate using the SRU criteria there was room for improvement in the 50-69% category.Raising the threshold of the PSV from 125 cm/sec to 140 cm/sec improved the overall accuracy from 85% to 92%, sensitivity from 93% to 94%, and specificity from 68% to 92%.In addition, they did not find the EDV to add further value to diagnostic accuracy.
The "gold standard" or the modalities for correlation with the CD for accreditation purposes vary greatly between VLs.In a 2012 survey of 809 respondents of whom 78% were technical directors of VL, CT angiography was used in 86.1%, MR angiography in 76.1%, digital subtraction arteriography in 69.7%, and surgical findings in 63.1%.Importantly, VLs used multiple modalities and 30.2% used three modalities and 37.7% used four modalities for correlation purposes.Two-thirds of respondents opined that there should be a single set of diagnostic criteria for determining degree of carotid stenosis by CD [16].This has led to the IAC recently issuing a "white paper" and recommending that VLs use SRU Consensus Conference Criteria for carotid interpretation.However, the ICAL also states that "Facilities which have rigorously internally validated their own criteria may continue to use these criteria at the present time" [18].
Based upon our analysis of our existing criteria and ROC analysis, the data support our decision to use our own internally validated criteria rather than the consensus criteria.Every VL must perform regular quality assurance and periodically validate their own internal criteria or the consensus criteria to make sure that referring physicians can rely on the results to make decisions in the best interests of the patients.

Figure 1 :
Figure 1: Scatter plots of internal carotid peak systolic velocity (PSV, (a)) and internal carotid end diastolic velocity (EDV, (b)) compared to percentage of stenosis based on angiographic imaging.

Figure 2 :
Figure 2: Receiver operating characteristic curves illustrating the accuracy with which our CD criteria are able to detect angiographic stenosis of 50% or greater.(a) Peak systolic velocity.(b) End diastolic velocity.

Figure 3 :
Figure 3: Receiver operating characteristic curves illustrating the accuracy with which our CD criteria are able to detect angiographic stenosis of 70% or greater.(a) Peak systolic velocity.(b) End diastolic velocity.

Table 1 :
Duplex ultrasound criteria for internal carotid artery stenosis.