18F-Fluorodeoxyglucose Positron Emission Tomography (FDG-PET), usually combined with low-dose CT (FDG-PET/CT), may be used to detect vascular wall inflammation [
Several criteria for the qualitative (visual) assessment of FDG-PET for the detection of large artery involvement in GCA have been introduced, ranging from “increased” circumferential 18F-FDG uptake (i.e., not further specified) in a segment of the arterial wall to equal or more intense vascular wall uptake than liver uptake [
It has been recommended that only those with specific expertise and experience should assess vascular wall 18F-FDG uptake [
Overview of articles reporting imaging findings in GCA/large-vessel vasculitis and assessment of observer agreement of visual assessment. (n.r.: not reported, 18F-FDG-PET: 18F-fluorodeoxyglucose positron emission tomography, MRI: magnetic resonance imaging, CT: computed tomography, and US: ultrasound).
Article (year of publication) | Imaging modality | Number of observers | Observer agreement |
---|---|---|---|
Blockmans et al., 2000 [ |
18F-FDG-PET | 4 (2 teams) | n.r. |
Blockmans et al., 2006 [ |
2 | n.r. | |
Walter et al., 2005 [ |
2 | 93% (28/30) | |
|
|||
Henes et al., 2008 [ |
|
At least 2 | n.r. |
Lehmann et al.,
2011 [ |
2 | 85% (Cohen’s kappa 0.7) | |
Papathanasiou et al., 2012 [ |
2 | n.r. | |
Fuchs et al., 2012 [ |
3 | n.r. | |
|
|||
Brodmann et al., 2004 [ |
18F-FDG-PET and US | 1 | n.r. |
|
|||
Meller et al., 2003 [ |
18F-FDG-PET and MRI | 2 | n.r. |
Scheel et al., 2004 [ |
2 | n.r. | |
Both et al., 2008 [ |
2 | n.r. | |
|
|||
Agard et al., 2008 [ |
CT | 1 | n.r. |
Marie et al., 2009 [ |
1 | n.r. | |
Prieto-González et al., 2012 [ |
2 | 98% | |
|
|||
Schmidt et al., 2002 [ |
US | 2 | 89% |
Schmidt et al., 2008 [ |
? | n.r. | |
DE et al., 2009 [ |
29 | 0.847 (kappa) | |
|
|||
Narvaez et al., 2005 [ |
MRI | n.r. 2 | n.r. 0.73 (kappa) |
31 FDG-PET/CT scans were selected from the databases of the Department of Nuclear Medicine and PET Research of the VU University Medical Center (VUMC) and the Department of Nuclear Medicine and Molecular Imaging of the University Medical Center Groningen (UMCG). These FDG-PET/CT scans were performed in clinical practice in order to (1) determine the cause of inflammation of unknown origin (
The first two groups were selected as a high incidence of large-vessel vasculitis was expected. A diagnosis of temporal arteritis (according to the ACR criteria for GCA) [
Patient characteristics, total and ordered by group.
Total ( |
Inflammation of unknown origin ( |
Temporal arteritis ( |
Polymyalgia rheumatica ( |
Control group ( | |
---|---|---|---|---|---|
Age (years)* | 70 (12) | 73 (13) | 67 (10) | 73 (7) | 62 (13) |
Sex (female)• | 65% | 58% | 100% | 57% | 50% |
ESR (mm/h)* | 70 (32) | 79 (27) | 64 (42) | 58 (30) | Unknown |
BMI (kg/m2)* | 23,4 (7,3) | 23,7 (8,6) | 23,3 (4,6) | 21,6 (9,9) | 25 (1,7) |
A Philips Gemini TOF (VUMC) and a Siemens Biograph (UMCG) PET/CT scanner (Philips Medical Systems, Eindhoven, Netherland and Siemens Medical Systems, Knoxville, TN) were used. A standardised protocol according to European Association of Nuclear Medicine (EANM) guidelines was used for the acquisition of scans [
Four observers used three distinct methods to visually (qualitatively) assess 18F-FDG vascular wall uptake (vascular uptake). These methods were applied in a consecutive manner in all scans and were consistently used by the observers. The level of experience varied among the observers, that is, 12, 8, 5 (three nuclear medicine physicians), and 2 years (one general physician working as a researcher in the field of PET/CT and large-vessel inflammation).
First, vascular uptake was assessed without using predefined criteria and was therefore based on first impression. This method was selected as it is often used in clinical practice, due to the absence of established observer criteria (qualitative and quantitative), such as comparing vascular uptake with uptake in other organs (e.g., the liver). Vascular wall uptake was scored as 1: normal, 2: atherosclerosis, or 3: large-vessel vasculitis (method (I)). The term large-vessel vasculitis was used as temporal artery involvement cannot be assessed using PET/CT due to a relatively low resolution of the scan and potential spill over from adjacent (physiological) brain 18F-FDG uptake [
Subsequently, the intensity of vascular uptake was systematically compared with the intensity of 18F-FDG liver uptake and scored as 0: absent, 1: less intense, 2: equally intense, or 3: more intense (method (II)).
The arterial segments that were studied in these first two methods included carotid, vertebral, subclavian, iliac, and femoral arteries, the aortic arch, and the ascending, descending, and abdominal aorta.
In the third method, femoral artery uptake was chosen as reference since the femoral artery is rarely involved in GCA and has a high incidence of atherosclerosis [
The distribution pattern, either focal (in all segments <2 cm) or diffuse (at least one segment comprising more than 2 cm of contiguous vascular wall uptake), and presence of arterial calcification on low-dose CT were also scored.
After the first reading, a consensus meeting was held with the goal to clarify causes of disagreement.
Finally, interobserver agreement was calculated using definitions of large-vessel vasculitis for each of the applied methods. These definitions were first impression (no predefined criteria), diffuse vascular wall uptake, diffuse vascular wall uptake, diffuse uptake,
To date, there is no clinical reference standard for a diagnosis of large artery involvement in GCA (either with or without temporal artery involvement). Furthermore, there is no consensus on which imaging modality or imaging criteria should be used. Therefore, in order to compare FDG-PET/CT results to a clinical diagnosis of large artery involvement in GCA, we defined the latter as temporal arteritis according to ACR criteria [ inflammation (i.e., elevated ESR) of unknown origin, not fulfilling ACR criteria for GCA, accompanied by FDG-PET/CT results that were unanimously classified as large-vessel vasculitis by all observers for at least two of the methods applied.
In both groups, a good clinical response to immunosuppressive therapy (prednisone), defined as rapid resolution of signs and symptoms, accompanied by normalisation of inflammatory parameters, was mandatory for the diagnosis. Additionally, no other diagnosis was allowed to have been established during a follow-up period of at least 3 months.
This clinical diagnosis was used to determine sensitivity and specificity for the 4 FDG-PET/CT scoring methods.
Based on the results of the first four observers, the definitions of large-vessel vasculitis that showed high levels of agreement between observers and with the clinical diagnosis were used for external validation. The first four observers are employees at a university hospital and have specific interest and experience in large-vessel vasculitis and PET/CT reporting. Therefore, two community hospital nuclear medicine physicians were asked to score the same scans by these definitions. Again, interobserver agreement was determined.
Levels of agreement were quantified using Fleiss’ kappa (
After all scans were analysed, a remarkably high disagreement (i.e., 1/3 of all cases) was noticed when vascular uptake was compared to liver uptake. In a subgroup of patients vascular uptake was scored as less intense than liver uptake by some of the observers, whereas it was scored equally intense by other observers. During the consensus meeting, the observers concluded that in some cases vascular wall uptake was considered to be less intense than liver uptake based on their first impression that large-vessel vasculitis was not present. Therefore, a reanalysis of these scans was performed. Furthermore, the observers agreed that vascular uptake was only consistent with vasculitis if it showed a diffuse uptake pattern. The presence of calcification was ignored as it was present in all patients on low-dose CT scans.
Table
Average number of vasculitis PET/CT scores (individual observer scores), Fleiss’ kappa, sensitivity, and specificity (95%-CI) according to the different methods applied.
Method | Average number of vasculitides |
Fleiss’ kappa | Sensitivity* | Specificity* |
---|---|---|---|---|
(I) First impression | 9 (6, 9, 10, 11) | 0,68 | 92% (52–98%) | 90% (70–97%) |
|
||||
(IIa) Diffuse uptake, |
16 (14, 16, 17, 18) | 0,78 | 100% (61–100%) | 60% (39–78%) |
|
||||
(IIb) Diffuse uptake, |
7 (7, 7, 7, 8) | 0,96 | 100% (61–100%) | 98% (82–100%) |
|
||||
(III) Diffuse uptake, |
7 (6, 6, 6, 10) | 0,81 | 80% (41–94%) | 96% (79–99%) |
18F-FDG PET/CT scans showing: (a) Maximum intensity projection (MIP) image: scored as large-vessel vasculitis by all observers according to all methods, (b) coronal image: 18F-FDG uptake in descending aorta (arrow) scored as equal to liver uptake (arrowhead) by 2 observers and lower than liver uptake by 2 other observers, none of the observers scored higher than liver or femoral artery uptake, (c) MIP image of a PMR patient that was scored negative for large-vessel vasculitis by all observers. (Cerebral and urinary tract 18-18F-FDG uptake are physiological).
All observers agreed in 21 of 30 cases (70%) when assessing scans according to method (I). The corresponding Fleiss’ kappa was 0,68 (0,72 without the results of the least experienced observer).
There was agreement between all observers in 19 of 30 patients (63%) when scoring according to method (IIa), and kappa was 0,78 (0,81 without results of the least experienced observer) (Table
Finally, for method (III), all observers agreed in 27 of 30 cases (90%) with a resulting kappa of 0,81 (1 without the results of the least experienced observer).
In 4 patients (13%), we were unable to obtain sufficient follow-up data to ascertain a clinical diagnosis. One patient (from the group of inflammation of unknown origin who was suspected of large-vessel vasculitis after PET/CT) died the week after the scan was performed; autopsy was not performed. Three patients were lost to follow-up. In the remaining 27 patients, a clinical diagnosis of large-vessel vasculitis was established in 6 (22%) according to the previously mentioned criteria. The average sensitivity and specificity for all definitions are shown in Table
Of 6 patients with temporal arteritis, 4 had large artery involvement on FDG-PET/CT according to all observers. Two of these four patients had a negative temporal artery biopsy, whereas one biopsy was inconclusive (not arterial tissue). The two patients with a negative FDG-PET/CT scan, both, had a positive temporal artery biopsy. One of the 7 patients with PMR had large-vessel vasculitis on FDG-PET/CT when using definition (IIa), whereas no large-vessel vasculitis was present using definitions (IIb) and (III).
Table
Average number of vasculitis PET/CT scores (individual observer scores), Cohen’s kappa, sensitivity, and specificity (95%-CI) according to the different methods applied.
Method | Average vasculitis score |
Cohen’s kappa | Sensitivity* | Specificity* |
---|---|---|---|---|
(I) First impression | 10 (9, 11) | 0,85 | 100% (61–100%) | 88% (67–96%) |
|
||||
(IIb) Diffuse uptake, |
6 (5, 7) | 0,79 | 83% (46–95%) | 100% (84–100%) |
|
||||
(III) Diffuse uptake, |
7 (7, 7) | 0,63 | 83% (44–97%) | 93% (73–98%) |
Our study suggests a preference for standardized observer criteria for the assessment of large-vessel vasculitis on 18F-FDG-PET/CT images. Among dedicated and experienced observers, assessment of diffuse vascular uptake that exceeds liver uptake provides the highest observer agreement. Additionally, sensitivity and specificity appear to be superior, although the results show considerable overlap in the confidence intervals, at least partly due to the limited number of patients in this study. Among less experienced observers, agreement was more or less similar for standardized criteria (i.e., comparing vascular uptake to liver uptake) versus “first impression.” Altogether, standardization of imaging criteria is equal to or superior to using first impression. In addition, standardization of imaging criteria will undoubtedly facilitate communication, both in clinical practice and in science, bearing in mind the increasing use of FDG-PET/CT in clinical practice and hence the assessment of vascular 18F-FDG uptake by observers with varying degrees of experience.
High interobserver agreement was suggested in two previous studies which also compared vascular uptake with liver uptake [
Our study has some strengths and limitations. One of the strengths is that observer agreement was studied in a group of observers with a varying degree of experience, enhancing the generalizability of the results. Additionally, external validation by nonacademical nuclear medicine physicians confirmed the results, which is important as many studies addressing this topic are performed solely in an academic setting. In clinical practice, FDG-PET/CT scans will be assessed by nuclear medicine physicians or radiologists with a varying amount of clinical experience. The clinical reference standard that we constructed is also a strength of this study. Although we do realise that the definitions we used to establish a clinical diagnosis of large-vessel vasculitis are not universally accepted diagnostic criteria, we at least made a serious attempt to relate imaging tests to a clinical diagnosis. Our proposed criteria incorporate the most important characteristics of GCA (clinical signs and symptoms, inflammation and rapid response to steroids) and a consensus among multiple observers regarding imaging characteristics of vasculitis in large arteries. This approach enabled us to establish that changing the cut-off value for 18F-FDG uptake (higher than liver uptake as opposed to equal to or higher) did not seem to affect sensitivity of the clinical diagnosis but increased specificity. Another strength is the application of different criteria in a single study to investigate whether one definition possesses both high interobserver agreement and diagnostic accuracy. In the current study, vascular wall uptake was also compared with femoral artery uptake, as we have experienced that, in the elderly (i.e., patients over 50 years of age), the femoral artery invariably displays 18F-FDG uptake. It remains to be elucidated whether this results from atherosclerotic plaque inflammation, which is known to be common in the elderly [
A limitation is the absence of a true reference standard for the diagnosis of large-vessel vasculitis, which would need to be histopathological evidence. Obviously, our clinical diagnosis comprises the test under study which may introduce bias. However, as a “practical gold standard” to establish large-vessel vasculitis is currently not present, we believe that this approach may be a first step towards establishing such a gold standard. We are inclined to think that the criteria we used for a patient to be classified as a large-vessel vasculitis patient were appropriate.
The limited sample size may also be considered a limitation. Although we included patients with inflammation of unknown origin and apparently healthy patients (after follow-up for malignancy) as controls, which further enhances the external validity of our study, the limited size of all groups warrants corroboration in a larger group of similar patients. Finally, it has been suggested that not two but four characteristics might differentiate vasculitis from atherosclerotic plaque inflammation [
In conclusion, predefined standardized criteria (comparing vascular uptake to liver uptake) have high interobserver agreement and probably have good diagnostic accuracy for large-vessel vasculitis. All patients with a clinical diagnosis of large-vessel vasculitis displayed diffuse vascular wall uptake higher than liver uptake. We recommend that observers consider scans with these characteristics to be consistent with large-vessel vasculitis when using modern PET/CT scanners. In case of irregular liver uptake, femoral artery uptake may be used as an alternative reference standard, bearing in mind that sensitivity might be slightly lower if the femoral artery is involved in the disease process. Finally, these results may not apply to patients that used steroids prior to the FDG-PET/CT scan. Future studies need to address the effect of steroids on vascular uptake (i.e., establishing time interval between start of steroids and resolution of characteristics on FDG-PET/CT) and the potential value of semiquantitative methods (SUVs).
The authors declare that there is no conflict of interests regarding the publication of this paper.