Spirometry in primary care : An analysis of spirometry test quality in a regional primary care asthma program

1University of Western Ontario, St Joseph’s Health Care, London; 2University of Windsor; 3Asthma Research Group, Windsor, Ontario Correspondence and reprints: Dr Christopher J Licskai, University of Western Ontario, St Joseph’s Health Care, 268 Grosvenor Street, London, Ontario N6A 4V2. Telephone 519-646-6405, fax 519-646-6064, e-mail clicskai@uwo.ca Evidence-based clinical practice guidelines recommend spirometry for the diagnosis and management of asthma (1-7). Spirometry can be easily adopted into primary care practice (8), has been demonstrated to improve diagnostic accuracy (9,10) and improves the concordance of clinical practice with evidence-based guidelines (11). The National Lung Health Education Program consensus statement (8) recommended the implementation of office spirometry in the primary care setting, and technological advances have made high-quality spirometers available at a relatively low cost. Despite these facilitating factors, there has been limited uptake of this asthma guideline recommendation (12,13) and there continues to be limited access to spirometry in primary care (14). While there are many challenges to the implementation of spirometry testing in primary care, an essential first step is to confirm that quality testing can be achieved in this setting. Spirometry quality standards published by the American Thoracic Society (ATS) (15,16) and the European Respiratory Society (ERS) (17) are accepted internationally as the standard for test quality. It has been demonstrated that these standards are achievable by health care providers in a variety of clinical settings. Experienced pulmonary function technologists obtained spirometry results that met quality benchmarks in 90% of patients tested in a pulmonary function laboratory (18) and in a large field study (19). After only one-half to two days of training, research personnel were able to meet or exceed spirometry quality standards in 79% of children and adults tested in asthmarelated clinical trials (20). Emergency department staff achieved modified ATS quality criteria in 74% of acutely ill adult and adolescent patients presenting with asthma-related dyspnea (21). In contrast, several studies in primary care have reported disappointing quality results, with <40% of tests being technically adequate (22-26). In the present study, we evaluated the quality of spirometry performed by asthma educators with limited spirometry training in a regional primary care-based asthma program (RAP) and compared their performance with experienced pulmonary function technicians at two regional reference sites. originAl Article

Spirometry in primary care: An analysis of spirometry test quality in a regional primary care asthma program Christopher J Licskai MD 1 , Todd W Sands PhD 2 , Lisa Paolatto MSc 3 , Ivan Nicoletti RRT 3 , Madonna Ferrone RRT E vidence-based clinical practice guidelines recommend spirometry for the diagnosis and management of asthma (1)(2)(3)(4)(5)(6)(7).Spirometry can be easily adopted into primary care practice (8), has been demonstrated to improve diagnostic accuracy (9,10) and improves the concordance of clinical practice with evidence-based guidelines (11).The National Lung Health Education Program consensus statement (8) recommended the implementation of office spirometry in the primary care setting, and technological advances have made high-quality spirometers available at a relatively low cost.Despite these facilitating factors, there has been limited uptake of this asthma guideline recommendation (12,13) and there continues to be limited access to spirometry in primary care (14).While there are many challenges to the implementation of spirometry testing in primary care, an essential first step is to confirm that quality testing can be achieved in this setting.
Spirometry quality standards published by the American Thoracic Society (ATS) (15,16) and the European Respiratory Society (ERS) (17) are accepted internationally as the standard for test quality.It has been demonstrated that these standards are achievable by health care providers in a variety of clinical settings.Experienced pulmonary function technologists obtained spirometry results that met quality benchmarks in 90% of patients tested in a pulmonary function laboratory (18) and in a large field study (19).After only one-half to two days of training, research personnel were able to meet or exceed spirometry quality standards in 79% of children and adults tested in asthmarelated clinical trials (20).Emergency department staff achieved modified ATS quality criteria in 74% of acutely ill adult and adolescent patients presenting with asthma-related dyspnea (21).In contrast, several studies in primary care have reported disappointing quality results, with <40% of tests being technically adequate (22)(23)(24)(25)(26).
In the present study, we evaluated the quality of spirometry performed by asthma educators with limited spirometry training in a regional primary care-based asthma program (RAP) and compared their performance with experienced pulmonary function technicians at two regional reference sites.

MeThodS
Study design and participants Reference sites: Asthma patients from two pulmonary function laboratories in the region were used as regional quality reference standards.Spirometry measurements were completed by experienced pulmonary function technicians who worked full-or part-time in this capacity.The reference sites were The Family Medical Centre (FMC, Windsor, Ontario), a primary care group practice where spirometry was performed as a satellite of a local pulmonary function laboratory, and The London Health Sciences Centre (LHSC, London, Ontario), a university teaching hospital pulmonary function laboratory.evaluation sites: Spirometry testing was completed as a component of a RAP between October 2004 and November 2006 (27).In the RAP, asthma educators travelled to primary care sites on assigned days and performed spirometry.Ten health care providers who were asthma educators with the following professional designations: registered nurse (n=1), registered respiratory therapist (n=5) and pharmacist (n=4), performed spirometry in 19 primary care sites across the region.None of the participating asthma educators had regular experience or in-depth training on performing spirometry before the project.Importantly, these asthma educators received general instruction on the principles of spirometry as a component of their course curriculum but were not trained to perform testing before the present study.Respiratory therapists receive instruction on performing spirometry as a component of their course curriculum.Eighty per cent (eight of 10) of the educators in the present study had no work experience with performing spirometry whatsoever, and two had occasionally performed bedside spirometry in the hospital before the present study.equipment RAP: All asthma educators used the Jaeger Masterscope spirometer version 4.1 (Jaeger-Toennis, Germany).This model consists of a handheld pneumotachograph with laptop integration.The FMC reference site used a Sensormedics rolling-drum spirometer with V-max Version 05-2A (Sensormedics, USA) and LHSC used a Sensormedics rollingdrum spirometer with V-max version 12-1A.Spirometry test results and quality data were extracted from proprietary software and the quality outputs (error codes) were verified by analysis of actual forced expiratory time, back extrapolation volume, forced expiratory volume in 1 s (FEV 1 ) and forced vital capacity (FVC) measurements.Error code analysis was the only source of data for end of test flow criteria.

Asthma educator training
Asthma educators were trained to perform spirometry by an experienced pulmonary function technician during two 2 h workshop sessions, and supervised in up to six patient sessions; thereafter, the trainer was available as a resource person.Specific workshop and onsite training objectives included: in-servicing on the spirometry equipment including assessing automated test performance quality feedback and performing regular quality control (eg, calibration); general principles of spirometry, measurement values (FEV 1 , FVC, FEV 1 /FVC ratio), predicted normal values, reversibility criteria and contraindications to spirometry; spirometry technique and patient coaching, recognizing and correcting common patient performance and equipment problems; and the 1994 ATS spirometry quality criteria for acceptability and reproducibility.There was no audit and feedback process during the study.The Masterscope spirometer provided automated, in-session quality feedback on test acceptability and reproducibility.

Quality goals ATS 1994:
The ATS 1994 quality benchmarks in effect at the time were used for the primary analysis in the present study (16).Acceptable curves were defined as those that met the following three criteria: back extrapolation volume <5% of FVC or <0.150 L; forced expiratory time >6 s; and/or end of test criteria for flow of <0.025 L/s for >1 s.A minimum of three acceptable curves were required in each spirometry session.Reproducibility was defined as two acceptable curves with an FEV 1 and an FVC within 0.200 L.

ATS/eRS 2003:
A secondary analysis was performed using ATS/ERS criteria published subsequently (17).The criteria for acceptability were the same as the ATS 1994 standards.The criteria for reproducibility were more stringent: reproducibility was defined as two acceptable curves with an FEV 1 and an FVC within 0.150 L.

Statistical analysis
Statistical evaluation was completed using SAS version 9.1 (SAS Institute Inc, USA).Subject characteristics were measured at the session level without identifying individual patients or adjusting for multiple observations per subject.Between-group comparisons of age categories and sex were performed using c 2 tests for comparing proportions.Between-group comparisons of actual age, height and weight were performed using Tukey's test for ANOVA with multiple comparisons.To address the possibility that the variances across groups were not equal, the comparisons were repeated using a Kruskal-Wallis test for comparing groups and the Wilcoxon two-sample test for pair-wise comparisons.The results were similar and, therefore, reported based on Tukey's test.
Acceptability criteria were analyzed at the flow-volume loop (FVL) or trial level.For continuous end points, mixed-model ANOVA was used, while for dichotomous end points, the generalized estimating equations algorithm was used to adjust for the effect of multiple observations per session (ie, clustering).Reproducibility for FEV 1 and FVC criteria were analyzed at the session level using ANOVA for continuous end points and c 2 tests for dichotomous end points.P<0.05 was considered to be statistically significant.To preserve an overall alpha level of 0.05 in performing pair-wise comparisons, a Bonferroni correction was made in which dichotomous variables were involved with differences deemed significant if <0.017, and Tukey's multiple comparisons test for continuous variables.Exact P values are not available for these comparisons using this method of analysis and, therefore, pair-wise comparisons were reported as either P<0.05 or not significant.

ethics review
The present study was approved by the Office of Research Ethics at The University of Western Ontario (London, Ontario) and the Research Ethics Board of Hotel-Dieu Grace Hospital REB# 02-SE-015.Informed consent was obtained from all subjects.

Spirometry measurements
A total of 12,815 FVLs were evaluated for quality.RAP: Four-hundred seventy-two (472) individual spirometry sessions were identified, with a total of 1606 FVLs.Asthma educators performed a mean of 47 spirometry test sessions, median 28 and range one to 121 sessions.Reference sites: The FMC dataset included 4013 asthma-related FVLs in 573 sessions and the LHSC included 7196 FVLs in 1151 sessions.

Subject characteristics
Demographic data were extracted from the respective spirometry software programs on all subjects.LHSC patients were older than RAP and FMC patients, with a mean (± SD) age of 48.6±19 years versus RAP (41.3±23.9years) and FMC (38.3±21.4years).There were more children in both community settings than in the academic pulmonary function laboratory (LHSC) (Table 1).Additional clinical data were available by electronic chart abstraction in 93% (437 of 472) of subjects from the RAP.The majority of RAP subjects were on asthma controller therapy (81.5% [356 of 437]) and were nonsmokers (87% [380 of 437]).RAP subjects used a mean of 0.60±1.26doses of beta 2agonist per day (Table 1).No additional clinical data were available on subjects from the community reference sites.

Individual acceptability criteria:
The RAP met start of test criteria (back extrapolation volume) in 1473 of 1606 (91.8%)FVLs, forced expiratory time criterion in 1133 of 1606 (70.6%) and end of test criteria flow criteria in 1112 of 1606 (69.2%).Compared with the regional reference sites, the RAP had numerically similar results for back extrapolation volume, met forced expiratory time criteria more often than FMC (2341 of 4013 [58.3%];P<0.05) but less often than LHSC (6124 of 7196 [85.1%];P<0.05), and met end of test criteria flow criteria more often than both FMC (1645 of 4013 [41.0%];P<0.05) and LHSC (4360 of 7196 [60.6%];P<0.05).After adjusting forced expiratory time for age, the RAP had a higher proportion of children than LHSC (Table 1), forced expiratory time criteria were met in the RAP in 1211 of 1606 (75.4%)FVLs.

Sessional acceptability criteria
The RAP had a greater proportion of patient sessions with a minimum of three FVLs that met acceptability criteria (392 of 472 [83.1%]) versus the regional primary care comparator FMC (392 of 573 [68.4%];P<0.05) but a smaller proportion of acceptable sessions than the academic pulmonary function laboratory LHSC (1069 of 1151 [92.9%];P<0.05).

Reproducibility criteria
Of 472 RAP patient sessions, there were three sessions in which only one FVL was available; therefore, reproducibility was assessed on 469 sessions.The RAP spirometry results were highly reproducible, meeting the ATS quality standard for both FEV 1 and FVC in 428 of 469 (91.3%) sessions, slightly less reproducible than the FMC and the LHSC results: 564 of 573 (98.4%) and 1131 of 1151 (98.3%), respectively (P<0.001) (Table 3).There was more measurement variability in both the FEV 1 and FVC in the RAP group compared with the regional reference sites.RAP spirometry met the stricter reproducibility criteria of the 2003 ATS/ERS quality standard in 395 of 469 (84.2%) patient sessions.

overall sessional quality: Acceptability and reproducibility
The RAP achieved all ATS acceptability and reproducibility quality criteria in 361 of 472 sessions (76.5%), more often than the primary care comparator FMC (388 of 573 [67.7%]) and less often than the academic pulmonary function laboratory (LHSC) (1050 of 1151 [91.2%];P<0.001) (Table 4).Using the more rigorous ATS/ERS criteria, the RAP met all criteria in 337 of 472 (71.4%) sessions.

dISCuSSIoN
The present study demonstrated that high-quality testing can be achieved in primary care practice by health professionals with limited spirometry training and experience operating within a simple quality control model.We also confirm the work of Enright et al (18), who demonstrated that ATS/ERS spirometry quality standards are achievable in 90% of patients tested in an academic pulmonary function laboratory.
The ATS ( 16) and the ATS/ERS (17) performance thresholds have been set such that >90% of patients can meet the requirements within five manoeuvres if coached by a technician with good training, motivation and experience (8).It is clear from our work and from the work of others (18) that this standard can be met by experienced technicians in a pulmonary function laboratory.In the present study, 10 asthma educators performed 472 spirometry tests in a RAP and achieved ATS quality benchmarks in 77%.Our results are consistent with four primary care studies demonstrating high rates of technical adequacy in spirometry performed by personnel with limited training (11,(28)(29)(30).The following rates of technical adequacy were reported: 71% by Yawn et al (11) on 368 tests in 12 primary care practices; 76% by Walters et al (28) in 531 tests completed by two trained nurses; 78% by Zanconato et al (29) in 109 tests completed by 10 pediatricians; 92% by Bednarek et al (30) in 1960 tests completed by two nurses; and 79% by Enright et al (20) in 9355 tests completed by research personnel.Collectively, these results suggest that an achievable target range for technical adequacy in 'first tier' primary care testing is 75% to 90%.
While evaluating the impact of a more permissive spirometry quality standard for primary care is beyond the scope of this discussion, to partially address this question, we considered whether the 111 (23%) technically inadequate studies were normal.We identified that 91 of 111 (82%) had an FEV 1 in the normal range (>80% predicted).In the present analysis, only 20 of 472 (4%) tests were both technically inadequate and had an FEV 1 <80% predicted.Arguably, a technically inadequate study with a normal FEV 1 is clinically valuable when, for example, the objective is to evaluate for a diagnosis of chronic obstructive pulmonary disease or in the follow-up of individuals with airways disease.Whereas the high prevalence of normal testing can be helpful in primary care, it also presents a diagnostic challenge in asthma.In a population of asthmatic patients predominantly from primary care, Aaron et al (31) found that only 16% of subjects had a diagnosis of asthma confirmed by spirometry and that 72% required a methacholine challenge test.We suggest that methacholine challenge testing be considered to confirm a diagnosis of asthma when spirometric measurements are normal.In a primary care population in which the frequency of normal testing is high, this subanalysis suggests that a 77% rate of technical adequacy may be acceptable.
The quality model in our study included delegating spirometry measurements to an individual committed to performing spirometry, providing time-limited workshop and hands-on training, and using spirometers that provided automated quality feedback.Our quality model was similar to that used in the conduct of research trials (19,20), and to two recent primary care studies in which "visiting trained nurses" performed spirometry with high rates of technical adequacy (28,30).To our knowledge, our model is the first to use asthma educators -a group of health professionals that understand spirometry values -and who know how to integrate spirometry results into clinical care and are highly motivated to obtain spirometry measurements.Similar to other health care professionals that may be tasked with performing spirometry in primary care (eg, registered nurses, pharmacists and physicians), asthma educators have general knowledge about spirometry but are not trained on test performance.In our educator group, only two of 10 educators had any previous work experience performing testing: both were respiratory therapists and both had minimal previous experience.While we cannot exclude the possibility that previous training and experience influenced our quality outcomes, we believe the effect to be small.In Canada, asthma and chronic obstructive pulmonary disease educators are nationally certified.Our study supports an interdisciplinary management model in which certified educators perform spirometry in primary care.We acknowledge that a model using asthma educators may limit the generalizability of our findings but emphasize that we engaged providers from a cross section of primary care health disciplines, and that the core quality elements of our model are simple to execute, adaptable and demonstrated to be effective in primary care practice (11,(28)(29)(30).
There are other limitations to the present study.The ATS 1994 standards have been replaced by the ATS/ERS 2003 standards.Our asthma educators were guided by the ATS 1994 quality criteria and were not instructed on the more stringent ATS/ERS 2003 standard; however, a secondary analysis using this standard demonstrated compliance in 71% of sessions.We expect that if we had trained our educators using the current standard and had programmed automated spirometer quality feedback based on these criteria, our ATS/ERS rates of technical adequacy would have been higher.Also, in the reference site analysis, we were limited to identifying patients with asthma based on diagnostic information recorded on the spirometer and, therefore, may have included patients who did not have asthma.In addition, there were demographic differences between the reference sites and the RAP that may have independently influenced spirometry quality.Acknowledging these limitations, we note that the quality data from our regional comparator sites are consistent with rates reported in the literature (18,19).
Health care providers with limited spirometry training and experience can obtain ATS/ERS quality testing in primary care practices.The demonstration that quality testing is achievable in primary care is an essential step toward the broader implementation of spirometry testing, which would have a positive impact on the diagnosis and management of chronic obstructive pulmonary disease and asthma.Our analysis suggests that a target range for technical adequacy of between 75% and 90% is achievable.A formal evaluation of the impact of an adapted and more permissive primary care standard is recommended.A quality model defined by providing a time-limited workshop and hands-on training, a motivated mobile health care provider committed to spirometry testing and supported by a spirometer that provides automated quality feedback, should be prospectively evaluated in future primary care studies.

ACkNoWLedgeMeNTS:
The authors acknowledge and thank the following individuals for their contribution to this project: Ms Kathy Colledge for assistance with preparation of the manuscript; medical writer, Flora Krasnoshtein MSc, for her editorial assistance with the manuscript; Mr Larry Stitt from the University of Western Ontario for biostatistical support; Mr Jason West from the University of Windsor -Centre for Smart Community Innovation for his research assistance; and Mr John Prost and Mr Murray Beaton for assistance with data extraction; Dr Rob McFadden and Ms Nancy Garvey for their helpful input on the manuscript.
AuThoRS CoNTRIBuTIoNS: CL, TS, LP, MF and IN contributed to the conception and design of the study, and to the analysis and interpretation of data.CL is the guarantor for this work.All authors were involved in critically revising the article for important intellectual content and gave final approval of the version to be published.

FuNdINg:
This study was funded by the Government of Ontario.The project sponsor had no role in the study design, the collection, analysis and interpretation of data; or writing the manuscript; and no role in the decision to submit the manuscript for publication.

TabLe 1 Subject characteristics (patient session level)
1 Forced expiratory volume in 1 s; FMC Family Medical Centre (Windsor, Ontario); LHSC London Health Sciences Centre (London, Ontario); NS Not statistically significant; RAP Regional primary care asthma program; vs Versus