The value of thoracic computed tomography scans in clinical diagnosis : A prospective study

1Respiratory Division; 2Department of Radiology; 3Centre for Clinical Epidemiology and Evaluation, University of British Columbia and Vancouver General Hospital, Vancouver Coastal Research Institute, Vancouver, British Columbia Correspondence: Dr John Mayo, Department of Radiology, Vancouver General Hospital, 899 West 12th Avenue, Vancouver, British Columbia V5Z 1M9. Telephone 604-875-4111 ext 63193, fax 604-875-5498, e-mail john.mayo@vch.ca MO Turner, JR Mayo, NL Müller, M Schulzer, JM FitzGerald. The value of thoracic computed tomography scans in clinical diagnosis: A prospective study. Can Respir J 2006;13(6):311-316.

©2006 Pulsus Group Inc.All rights reserved ORIGINAL ARTICLE I mprovements in computed tomography (CT) technology and the wide dissemination of scanners have led to a marked increase in CT imaging for chest disease in the past 20 years.Although the fundamental physical advantage of CT over chest radiography is easily measured (1), the effect of CT information on clinical practice is more difficult to quantify (2,3).As with all diagnostic tests, thoracic CT is used to decrease diagnostic uncertainty or confusion.In certain clinical situations, CT results may be definitive, showing either the presence or absence of a suspected disease state (eg, pulmonary embolism) with a high degree of confidence (2)(3)(4)(5)(6).However, in other situations, CT is used even though it only provides suggestive diagnostic information, and firm diagnosis requires the integration of imaging results with clinical data, laboratory tests and pathological findings (7,8).
Ideally, a diagnostic test gives an accurate, definitive answer about the presence or absence of a disease state in all patients.In practice, however, this is rarely achieved due to variations in test results and conditions and disease severity.The impact of diagnostic tests, such as thoracic CT scans, on patient management is also variable if the results do not change the clinical diagnosis or if optimal therapy has already been instituted.Finally, inadequate communication between the radiologist and the referring clinician can result in failure to appreciate the implications of diagnostic imaging results.
Although there have been recommendations to assess the clinical impact of thoracic CT in patient management (9), few studies have addressed this aspect of clinical care (10).The wide dissemination and accepted use of CT imaging has made rigorous evaluation of the incremental benefits attributable to the CT scan by randomized trials difficult (11)(12)(13).In our review of the literature, we were unable to find a study that measured changes in diagnosis and patient management (clinical utility) following thoracic CT examinations.Measuring the size of these effects is important to optimize waiting times, make rational decisions regarding the allocation of imaging resources and evaluate the risk-benefit ratio of the radiation exposure associated with CT imaging (14,15).We measured the clinical utility and waiting times of thoracic CT examinations in 454 patients using clinician questionnaires conducted before and after each test.Some of these results have been previously reported in the form of an abstract (16).

Study design
Two outcomes were studied: changes in the working or clinical diagnosis before and after thoracic CT scans, and waiting times for CT scans.These outcomes were measured by three questionnairestwo completed by the patient's clinician and the other filled out by the radiologist interpreting the thoracic CT scan.The experimental design followed the methods recommended by Guyatt et al (17) for before-and-after studies.Ethical approval was obtained from the local institutional review board.

Setting
The present study was performed on consecutive patients receiving thoracic CT scans over a six-month period at Vancouver General Hospital (Vancouver, British Columbia) -a 700-bed tertiary care teaching centre equipped with two state-of-the-art single-track spiral CT scanners.The hospital is the regional centre for thoracic surgery and serves a referral population of approximately two million people.

Procedure
On receipt of a CT scan request form, the research coordinator delivered a pre-CT scan questionnaire to the patient's attending or most responsible clinician.The completed pre-CT questionnaires were collected before the CT scan was performed.The CT scan requests were prioritized by one of two chest radiologists (JRM and NLM) or by the radiologist on-call at night or on weekends for urgent or emergency scans.Scan waiting times were based on the written clinical information on the requisition augmented by personal contact between the referring clinician and the staff chest radiologist or on-call radiologist.Scan times were categorized using the priority scale described in Table 1.A staff radiologist interpreted the CT examination and a handwritten radiology report was completed.The radiology report and the post-CT scan questionnaire were then delivered to either the ward or the attending clinician's office.Cases that did not have a completed pre-CT scan questionnaire before the completion of the CT scan were excluded, as well as cases with any pre-or post-CT scan questionnaires completed by resident house staff.Patients were classified as either inpatients or outpatients.
Before commencing data collection, the overall objectives and sample questionnaires were discussed at two interdisciplinary meetings attended by radiologists, thoracic surgeons and respiratory physicians.Prototype questionnaires for common clinical diagnoses (sarcoidosis, lung cancer and pulmonary fibrosis) were presented.The pre-and post-CT scan forms were modified based on the feedback received at these rounds.
To measure the diagnostic impact of the CT scan, clinicians were asked to record the initial (pre-CT scan) clinical diagnosis and the probability of this diagnosis (using a scale of 0% to 100%), and the estimated impact of the CT scan on both the diagnosis and subsequent management.The clinician-estimated impact of the thoracic CT scan on diagnosis and treatment was measured using a five-point scale, ranging from very large (score of five) to none (score of one).Clinicians indicated the scan priority, ranging from emergency (within 8 h) to elective (within four to eight weeks) (Table 1).

Radiology questionnaire
Immediately following the CT scan, the attending radiologist and thoracic radiology fellow reviewed the images in conjunction with the clinical information from the radiology requisition.The radiologist questionnaire was completed, indicating whether the scan was normal or abnormal.If the scan was abnormal, relevant findings and the differential diagnosis were recorded.The radiologist's confidence in the first-choice diagnosis was classified using a three-point scale (high, moderate or low).
Radiologists were asked whether additional clinical information not found on the accompanying radiology requisition was available at the time of scan interpretation, and using a three-point scale ('yes', 'no' or 'uncertain'), whether the waiting time was appropriate.The completed radiologist and post-CT scan clinician questionnaires were then returned to the referring clinician or the ward.

Clinician post-CT scan questionnaire
The initial clinical diagnosis from the pre-CT scan questionnaire was transcribed onto the post-CT scan questionnaire.After receiving the CT scan results, clinicians indicated the revised probability of the initial clinical diagnosis on a scale from 0% to 100%, and recorded the post-CT scan clinical diagnosis.Clinicians estimated the impact of the CT scan on the revised diagnosis and treatment plan using the same five-point ordinal scale as above.They were asked whether any clinically relevant abnormalities were found using CT that were not identified using chest radiography, and if any were identified, they were asked to assess the impact of these findings.Finally, clinicians noted whether patient waiting times were satisfactory.

Data classification and measurements
The The change in a clinician's estimated probability of the clinical diagnosis was calculated by subtracting the estimates of the pre-and post-CT scan questionnaires.The validity of these probability estimates was assessed by comparing the initial clinical diagnosis with the post-CT diagnosis, and only included 70% (320 of 454) of responses.Probability estimates that did not reflect the change in diagnosis or clinical information following imaging were judged to be invalid after consensus discussion.An example of an invalid response was a pre-CT scan questionnaire of a 65-year-old man with a long smoking history that listed lung cancer as the diagnosis with a 70% probability.After finding a large spiculated mass, nodal enlargement and an adrenal mass on a CT scan, the post-CT clinical diagnosis remained lung cancer with a 70% probability.The unchanging 70% probability of lung cancer following the CT scan findings led the observers to conclude that the probability estimate was invalid.There was no difference between inpatients and outpatients for the proportion of valid responses (P=0.17).The waiting time for the CT scan was determined by calculating the difference between the receipt of the requisition in radiology booking and the day the CT scan was performed.

Statistical analysis
Results are expressed as means ± SD, and all probability estimates are expressed as absolute values.Associations between categorical variables (eg, proportions of CT scans in different specialties versus inpatients or outpatients) were examined using χ 2 analyses of contingency tables.Significant cells within the tables were identified by corresponding adjusted standardized deviates.The effects of various factors (eg, different diagnoses and normal/abnormal scans) on continuous outcomes (eg, changes in probabilities of initial working diagnoses) were examined by ANOVA.Interaction effects were also assessed, and multiple comparisons were carried out using Duncan's and Tukey's multiple comparison methods.To assess the robustness of the parametric assumptions, the analyses were repeated using the corresponding nonparametric methods (Mann-Whitney and Kruskal-Wallis).
Associations between continuous variables (eg, days waiting versus changes in probabilities) were assessed by regression analyses.Analysis of covariance was used when dealing with both continuous and categorical explanatory variables.The effects of continuous variables on dichotomous outcomes (eg, normal versus abnormal scans) were analyzed using logistic regression.

RESULTS
There were 1555 body CT scans performed during the study period: CT scans of the thorax were performed in 631 patients, while the thorax and an adjacent body region, usually the abdomen, were scanned in 101 patients.Complete pre-and post-CT scan questionnaires were available for 62% (454 of 732) of patients.Outpatients accounted for 80% (361 of 454) and inpatients for 20% (93 of 454) of scans.The thoracic CT scans were classified as being for diagnostic purposes in 75% (340 of 454) of cases, for staging in 11% (51 of 454), to assess the progression of disease in 10% (49 of 454), for confirmation of a chest radiograph abnormality in 2% (11 of 454) and other in 1% (three of 454).The distribution of ordering clinicians by specialty is outlined in Table 2. Respirologists were more likely to order scans for diagnostic purposes and thoracic surgeons for staging (P<0.05).The three most common diagnostic categories for the initial working diagnosis were cancer, nodule and interstitial lung disease, accounting for 51% of scans (Table 3).Cancer was the   most common initial working diagnosis in outpatient CT scans.Inpatients were more likely to be investigated for infections, pulmonary embolism and pleural diseases (overall P=0.002).

Change in clinical diagnosis following thoracic CT scan
The clinical diagnosis changed in 48% (216 of 454) of cases following the CT scan (Table 3).When the working diagnosis changed, the scan was normal in 25% (112 of 454) and provided findings that indicated a different diagnosis in 23% (106 of 454).The CT scan supported the initial working diagnosis in 52% (234 of 454).There was a change in the clinical diagnosis in 58% (196 of 340) of cases that were performed specifically for diagnostic reasons.The change in a clinician's estimated probability for the clinical diagnosis between pre-and post-CT scan questionnaires was greatest for those CT scans that were classified as normal (mean 43.9%±25.3%),compared with those classified as having a different diagnosis (mean 36.3%±24.4%)and those with the same diagnosis (mean 26.3%±21.4%).These differences were highly significant by ANOVA (P=0.0001).The inclusion and analysis of all probability estimates (valid and invalid) did not alter the order or significance of the above results.The change in the estimated probability of the clinical diagnosis following the CT scan was significantly greater (P=0.004) for outpatients (mean 36.0%±24.6%)than for inpatients (mean 26.0%±22.8%).In comparison with the chest radiograph, clinicians indicated a large or very large impact of additional information from the CT scan in 56% (254 of 454) of cases.

Normal compared with abnormal CT studies
Twenty-five per cent of CT scans were reported as normal.The working diagnosis with the largest percentage of normal scans was pulmonary vascular abnormality (68%; P=0.0001), and in this group, the most common clinical suspicion was pulmonary embolism.
No association was found between clinician estimates of diagnostic and treatment impact using ordinal scales, scan prioritization or the probability of identifying abnormalities not seen on chest x-ray and the likelihood of a normal CT scan.Additionally, there was no significant difference between normal and abnormal CT scan results and clinicians' estimates of diagnostic impact (P=0.085).The clinicians' estimated priority for the CT scan did not discriminate between changes in diagnosis; however, CT scans that led to a different diagnosis were less likely to be elective (P=0.045).

Impact of CT scans for inpatients versus outpatients
There was no difference between physician specialties for the proportions of CT scans performed for inpatients and outpatients (P=0.15)(Table 2).There were no significant differences (P>0.05) between inpatients and outpatients in change of working diagnosis, post-CT scan impact on diagnosis or treatment, and the impact of CT compared with chest radiography.

Scan priority
Clinicians' priority assignment was highly correlated with waiting time for the CT scan (Figure 1).Overall, the waiting time diminished by 7.89 days for each unit increase in prioritization.With respect to scan prioritization, there were significant differences (P<0.05) when comparing outpatients with inpatients; outpatients were more likely to be classified as priority 1 or 2 (61.6% versus 18.3%, respectively) and less likely to be classified as priority 4 or 5 (11% versus 35.5%, respectively).
On the post-CT scan questionnaire, the ordering clinician reported that waiting time was inappropriately long for 4% (17 of 454) of CT scan patients.The radiologists disagreed with this assessment in 12 of the 17 cases.The clinician had provided additional information to indicate a high priority in only one case.The majority of ordering physicians were respirologists and most cases were outpatients (15 of 17).Cancer was the working diagnosis in 12 cases; two cases presented with hemoptysis; and there was one case each of pulmonary fibrosis, airways disease and pulmonary embolism.The scan was normal in five cases, the diagnosis changed in five cases and the pretest diagnosis was confirmed in seven cases.

DISCUSSION
In the present study, we found that thoracic CT scans had a large impact on diagnosis and management, changing the clinical diagnosis in 48% of patients.The change in clinical diagnosis resulted from a normal scan in 25% of cases and CT findings indicated a different clinical diagnosis in 23%.The CT scans that were normal had the greatest measured change in the estimated diagnostic probability of the initial diagnosis.The largest proportion of normal scans was in the pulmonary vascular disease category, in which scans were mainly performed in patients with suspected pulmonary embolism.These negative scans provide valuable reassurance and may limit therapeutic evaluation or further diagnostic evaluation.Additional value from negative diagnostic tests, such as biochemical markers used to assess chest pain in the emergency department, is demonstrated by early discharge and risk stratification (18).The finding of a significant number of normal scans in our study may have large cost implications because they likely limit further investigations, and thus, influence the overall costeffectiveness of CT (19,20).However, a full assessment of this question is beyond the scope of the present investigation.
We found that the majority of CT scans were performed for diagnostic purposes, as opposed to treatment assessment.The enhanced imaging provided by CT compared with chest radiography is highlighted by the finding of a large or very large impact of additional information from the CT scan in 56% of cases.Outpatient CT scans showed a significantly (P=0.004)greater change in the estimated probability of the clinical

Clinician assigned priority
Figure 1) Waiting time (mean days ± SD) to obtain a thoracic computed tomography scan versus clinician-assigned scan priority using a five-point scale from elective (priority 1) to urgent (priority 5).There was a high correlation between assigned priority and waiting time (R 2 =0.9517) diagnosis compared with inpatients.This change was expected because disease processes are generally less advanced in outpatients than in inpatients.However, changes in the initial clinical diagnosis were also observed for inpatients, indicating that CT scans impact the clinical diagnosis in both patient groups.Nonsurgical referrals were more common for the investigation and diagnosis of suspected abnormalities, and surgeons mainly used CT to stage or follow the progression of disease.These data concur with the role of a respirologist, who is usually asked to investigate an abnormal chest x-ray or symptom complex and then refers the patient for surgery, if intervention is required.
The waiting time between the request and the completion of the CT scan reflects clinical practice within the governmentfunded Canadian health care system.While it cannot be considered optimal, our finding that only 4% (17 or 454) of scans were thought by the referring clinician to be inappropriately delayed is reassuring.Waiting times ranged from a mean of 5.2 days (median of 1.5 days) for pulmonary vascular disease to 29.4 days for interstitial lung disease.The mean waiting time for patients investigated for cancer or lung nodules was 12 to 14 days.There was a significant relationship between the scan priority specified by the clinician and the waiting time for both outpatients and inpatients.As expected, inpatients had significantly (P<0.05)shorter waiting times.The strong negative correlation with increased priority probably explains the low percentage (4%) of cases in which the clinician indicated dissatisfaction with the timing of the scan.However, in the majority of these cases, the radiologist thought that the scans were appropriately scheduled.
Additional clinical information provided to the radiologist and captured on our questionnaire was identified in only one of the 17 cases that the ordering physician reported to be inappropriately delayed.Past studies have recommended greater radiologist involvement in consultative roles (21) because increased clinician input to the radiologist has the potential to expedite CT scans and provide greater diagnostic impact, especially in complicated and symptomatic patients.Our data suggest that this did not occur in a formal sense during the time of the study.However, our data are reassuring, in that there is a strong correlation between clinician-assigned priority and decreased waiting times, despite little formal consultation with radiologists.Therefore, our results do not support a general policy requiring radiology consultation before thoracic CT scanning.It is quite possible that 'curbstone consultations' between clinicians and radiologists occurred that were not captured by our survey techniques (22).
The weaknesses of the present study are primarily associated with study design.Randomized controlled trials are the optimal way to measure the impact of an intervention, but once a diagnostic technology is widely disseminated, these studies become more difficult and expensive to implement, and are limited in scope (11,12).Although we made every effort to obtain completed questionnaires before and after the CT scans, we depended on the goodwill and interest of busy clinicians.We did not require completion of questionnaires before booking the CT scan.The reported impact and change in probability reported by clinicians may not represent the full impact of the CT scan results on patient care and management strategies.Even after presentations to facilitate questionnaire completion, some confusion remained regarding the revised probability of the initial diagnosis on the post-CT scan questionnaire, accounting for the fact that only 70% of probability responses were considered valid.
The present study was completed using single-detector scanners.Now that multidetector-row CT scanners are widely disseminated, a criticism may be that our results may underestimate the impact of thoracic CT.Although the multidetectorrow CT scanners result in higher detection rates for peripheral pulmonary emboli (23) and small pulmonary nodules (24,25), they would have a lower impact for other conditions evaluated in the present study.Therefore, we believe that current multidetector-row scanners would not substantially change the magnitude or direction of our results.However, it is important to note that the economic and outcome impacts of detecting large numbers of indeterminate small nodules using multidetectorrow CT scanners requires further research, which is ongoing in our department (24,25).
The strengths of the present study are that we evaluated consecutive patients referred for thoracic CT imaging at a tertiary care hospital.We asked clinicians directly involved in management and treatment decisions to assess the impact of information obtained from the CT scan.Because diagnostic tests are ordered in the context of a synthesis of medical history, physical examination and other test results, we believe that the reported change in probabilities incorporates this aspect of the diagnostic process (26) and, therefore, represents a reasonable measure of the utility of the CT scan.Our approach to assess the impact of imaging information is complementary to other reports evaluating the impact of diagnostic imaging for abdominal pain and in psychiatric patients (27,28).

CONCLUSIONS
The present study shows that thoracic CT scans have a major impact on a working diagnosis and provide superior diagnostic imaging compared with the first line of investigation, chest radiography.The current system of scan prioritization in the Canadian health care system appears to appropriately match scarce imaging resources to clinicians' perceptions of patient need.Further research is required to assess the impact of current multidetector-row CT scanners that are being deployed in large numbers.We and others (29) note that the incremental cost-effectiveness of these different imaging strategies also needs to be assessed in future studies, particularly when diagnoses require repetitive follow-up CT imaging.

ACKNOWLEDGEMENTS:
The authors thank Mr Edwin Mak for statistical assistance and all the clinicians who participated so willingly.
FUNDING: The present study was funded by the British Columbia Lung Association.

TABLE 1
Computed tomography scan impact on diagnosis and treatment, and the physician assignment of computed tomography scan priority using an ordinal scale