Comparison of the Diagnostic Performance of Five Clinical Questionnaires for Chronic Obstructive Pulmonary Disease

Background Chronic obstructive pulmonary disease (COPD) remains one of the most prevalent pathologies in the world and is among the leading causes of mortality and morbidity, partially due to underdiagnosis. The use of clinical questionnaires to identify high-risk individuals to take them to further diagnostic procedures has emerged as a strategy to address this problem. Objective To compare the performance of the COULD IT BE COPD, CDQ, COPD-PS, LFQ, and PUMA questionnaires for COPD diagnosis. Methods A cross-sectional study was carried out on subjects who underwent spirometry in the third-level center. Data were collected between January 2015 and March 2020. Bivariate analysis was performed between the study variables and the presence of COPD. The area under the receiver operating characteristics curve (AUC-ROC), sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (LR+), and negative likelihood ratio (LR−) for each questionnaire were calculated. The AUC-ROCs were compared with the DeLong test, considering a p value <0.05 statistically significant. Results 681 subjects met the inclusion criteria and were taken to the final analysis. The prevalence of COPD was 27.5% (187/681). The mean age of the subjects was 65.9 years (SD ± 11.79); 46.3% (315/681) were female, and 83.6% (569/681) reported respiratory symptoms. Statistically significant relationship was found for COPD diagnosis with male sex, older age, respiratory symptoms, and exposure to wood smoke (p value <0.05). The AUC-ROCs of the questionnaires were between 0.581 and 0.681. The COULD IT BE COPD questionnaire had a lower discriminatory capacity AUC-ROC of 0.581, concerning the other scores (DeLong test, p = 0.0002). Conclusion The CDQ, COPD-PS, LFQ, PUMA, and COULD IT BE COPD questionnaires have acceptable performance for the diagnosis of COPD together with low sensitivity and specificity. Therefore, its use must be complemented with other diagnostic tests or techniques such as pulmonary function tests.


Introduction
Chronic obstructive pulmonary disease (COPD) remains one of the most prevalent pathologies in the world, with 212.3 million cases reported in 2019 [1].Despite the improvement in diagnosis and treatment in the last decades, it continues to be in the top etiologies of morbidity and mortality.In the population above 50 and 75 years old, COPD occupies the 3rd and 4th most frequent cause of disability-adjusted life years (DALYs) and the third most common cause of death [1,2].Although statistical information about the morbidity and mortality of COPD in the Latin American population is scarce, the most comprehensive studies show prevalence ranging from 8.9 to 19.4%, with a high rate of underdiagnosis of 87.4% that resembles worldwide literature [3][4][5].
Te universally high rate of COPD underdiagnosis is related to less severe forms of the disease, low educational level, and poor access to pulmonary function tests; these issues seem to be more pronounced in low-resource settings [6].Tere is limited information on the health burden of COPD underdiagnosis; nevertheless, the available evidence shows that lung-function decline befalls predominantly in the early stages of the disease [7] and that a late diagnosis impacts the quality of life and clinical outcomes of these individuals [8].Given the abovementioned issues, multiple case-fnding strategies have been developed for the diagnosis of COPD and most of them include the use of clinical questionnaires in the process [9,10].
One of the most extensively studied questionnaires is the CDQ published in 2006 by Price et al.Te researchers developed a score based on eight questions that included risk factors, symptoms, and anthropometric variables.It reported good discriminatory capacity for COPD diagnosis with an AUC-ROC of 0.816 [11].Tese fndings are not replicated in other populations according to a recent systematic review published in 2016, where the reported AUC-ROC was between 0.65 and 0.71 [12].More recently, the COPD-PS and LFQ questionnaires were developed based on national surveys conducted in the United States; these investigations found a good discriminatory capacity for COPD diagnosis with a reported AUC-ROC of 0.81 and 0.72 [13,14].Both questionnaires were validated in other populations, showing reductions in their performance capacity with an AUC-ROC of 0.748 in the COPD-PS and 0.652 in the LFQ [15,16].
Available questionnaires include risk factors and symptoms that allow identifying high-risk individuals to take them to further diagnostic procedures.Estimates show that using these scores could translate into a reduction in the number of spirometries necessary to detect cases of COPD, which is relevant when this resource is limited [17].Among the most used scores in primary care are the COULD IT BE COPD, CDQ, COPD-PS, LFQ, and PUMA [12,14,17,18].Few of these scores have been compared in other studies showing mixed results [19][20][21].None of these studies have included the application of these fve questionnaires at the same time in a single population.In addition, the diagnostic performance of the questionnaires changes according to where they are applied, whereby this study aims to compare the diagnostic performance of these fve questionnaires in our population.

Materials and Methods
A cross-sectional study was carried out on subjects who underwent spirometry pre-and post-beta-2-agonist administration in the third-level center Clínica Universidad de La Sabana (Chia, Cundinamarca, Colombia).Data were collected between January 2015 and March 2020.

Selection Criteria.
Inclusion criteria included age over 40 years, individuals scheduled to perform spirometry regardless of medical indication, informed consent signature for voluntary participation in the study, native Spanish language speaker, and availability of time to complete the fve questionnaires.Subjects whose spirometry did not meet the quality criteria according to the guidelines of the American Toracic Society (ATS), submitted incomplete questionnaires, and with a limitation for communication or understanding the questionnaire or spirometry technique were excluded [22].

Variables.
Te data obtained included identifcation variables, sex, height, weight, race, education level, presence of respiratory symptoms, age at the onset of respiratory symptoms, history of smoking, year package index, exposure to wood smoke, passive smoking, and prior history of COPD.Te questionnaires were administered and pre-and post-beta-2-agonist (B2) spirometry was performed, measuring the forced vital capacity (FVC), forced expiratory volume in the frst second (FEV1), and the FEV1/FVC ratio, as well as the percentage change of FVC and FEV1.Te spirometry was performed with prior calibration of the equipment by the same train and qualifed personnel of the pulmonary function laboratory.COPD was defned as the presence of fxed airfow obstruction defned by the American Toracic Society as the FEV1/FVC ratio <0.7 after bronchodilator administration [22].

Sample Size.
Estimation of the sample size for the comparison of areas under the receiver operating characteristics curve (AUC-ROC) was made with the method proposed by Hanley and McNeil [23].Te assumed ratio between no COPD and COPD individuals was 3 : 1, considering the prevalence of COPD in Colombia (26.4%) found by the PUMA study performed in Latin America [24].Te expected AUC-ROC of the questionnaires was taken from the original development studies that reported values of 0.81 and 0.72 [13,14].For a confdence interval of 95% and power of 95%, a minimum of 660 individuals were required.

Statistical Analysis.
Te data were obtained automatically through an electronic collection form, which was automatically entered into an Excel spreadsheet for later verifcation of their values by the research group for the search for transcription and correction errors.Te database was analyzed in the licensed statistical program STATA 14 and SPSS 25.Qualitative variables were reported in frequencies and percentages, and quantitative variables were summarized in the mean and standard deviation.Univariate analysis was performed between the study variables and the presence or absence of COPD using the chi-square test for qualitative variables and Student's t-test for quantitative variables.Using the scores obtained from the fve questionnaires, AUC-ROC, sensitivity, specifcity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (LR+), and negative likelihood ratio (LR− ) for the best cutof point according to Youden's index were determined.Te AUC-ROCs of the diferent scores were compared with the DeLong test.For the estimates, a p value <0.05 was considered signifcant.

Results
A total of 2199 potentially eligible patients were admitted during the study period.681 subjects met inclusion criteria and were taken to the fnal analysis.Figure 1 shows the fowchart for the inclusion of the subjects in the study.Te prevalence of COPD was 27.5% (187/681).

Discussion
In this study, we compared the performance characteristics of fve diferent questionnaires for COPD diagnosis at the same time in a single population.Te main fnding was that the discriminatory capacity of the CDQ, COPD-PS, LFQ, and PUMA was comparable with an AUC-ROC between 0.646 and 0.681.Comparatively, the COULD IT BE COPD questionnaire had limited discriminatory capacity with respect to the other available scores, with an AUC-ROC of 0.581.Other fndings included the determination of a prevalence of 27.5% of COPD in our population and its relationship with older age, male sex, fewer years of academic training, respiratory symptoms, and exposure to wood smoke.Te complexity and costs associated with lung function tests have limited their widespread use, which has led to underdiagnosis and suboptimal management of COPD, especially in primary care practice [6,25].To address this issue, in the last two decades, researchers developed clinical questionnaires to identify high-risk individuals to take them to spirometry and demonstrate airfow obstruction distinctive to this disease.Te broad spectrum of clinical presentation, severity, and risk factors associated with COPD is a challenge to develop a universally efective screening questionnaire; therefore, available questionnaires are designed from diferent populations [11,13,14,18,24].
In 2005, Calverley and collaborators developed one of the frst clinical questionnaires for COPD based on the population derived from the Tird National Health and Nutrition Examination Survey (NHANES III) [18].Tis fvequestion questionnaire (COULD IT BE COPD) demonstrated good discriminatory capacity for predicting airfow obstruction compatible with COPD with a sensitivity of 85% and NPV of 88%.Tese results contrast with the fndings of our study, where the sensitivity and NPV dropped to 58.8% and 77.4%, and the discriminatory capacity was poor with an AUC-ROC of 0.581.Te limited diagnostic performance observed in our study of the COULD IT BE COPD questionnaire could be explained by the noninclusion among the questions of key risk factors identifed in our population, such as male sex, stratifed age, exposure to wood smoke, and educational level [4].

Canadian Respiratory Journal 3
A prior study that assessed the LFQ questionnaire in our population showed similar performance compared to the original development study with an AUC-ROC of 0.71 [26].At the moment, only one questionnaire has been developed and validated based on a Latin American population which corresponds to the PUMA score.Te original development study in 2016 reported good diagnostic performance with an AUC-ROC of 0.71.Despite the fact that the population on which this questionnaire was developed corresponded to hospitalized subjects, the discriminatory capacity was similar in the external validity studies carried out on primary care and the general population [24,27].Te available evidence of the performance characteristics of the questionnaires described above shows a clear trend towards a decrease in its discriminative capacities for COPD when evaluated in other populations.Te results of our study mostly resemble the fndings of the performance characteristics of these questionnaires found in the external validation studies.Te diference concerning the original development studies is probably due the variability in the prevalence of disease and distribution of risk factors.Tese diferences originate in the selection of population and inclusion criteria used in each of these investigations; the most relevant is that most external validations were performed on individuals who attended primary care and not in the general population.
Few comparison studies of multiple clinical questionnaires for COPD diagnosis are available in the literature; none of them were performed in Latin America [19][20][21].In 2016, Sogbetun and collaborators compared fve clinical questionnaires on 383 subjects in primary care in the United States, including COPD-PS and LFQ.[20].Tey found a similar discriminatory capacity between the questionnaires with an AUC-ROC of 0.623 for the COPD-PS and 0.655 for the LFQ, which is consistent with our results.Te similarity of the discriminative capacities of the questionnaires found in both studies is explained by the resemblance between some of the population features.Tese include the   In 2022, a study comparing multiple questionnaires that included the CDQ, COPD-PS, and LFQ was published by Zhou et al. [21].Te researchers found a comparable diagnostic accuracy among the three questionnaires in a Chinese primary care population with AUC-ROCs between 0.653 and 0.719 and negative predictive values above 85%, which resemble the results of our study.Although the study had a higher prevalence of COPD (36.3%), the distribution of risk factors was similar, with the participants being 45.8% men and 38.8% with smoking history.Te available evidence seems to show that the performance of the most comprehensive questionnaires that include and stratify risk factors such as age, the intensity of exposure to smoking, and gender is similar in various populations.Taking into account that the application of the questionnaires evaluated in our study could be interchangeable based on their similar discriminatory capacity for COPD, the intrinsic characteristics of each questionnaire (number of questions, clarity, amount of time to carry it out, etc.) should be evaluated to choose the one with the feasible applicability in the primary care.
Our study has some limitations that must be noted for interpreting its fndings.First, it is a single-center study that limits the external validation of its results.In addition, a higher prevalence of COPD compared to current data and a high number of subjects undergoing pulmonary function tests included in our study impact the predictive values and performance found.Nevertheless, the internal validity strengthens by the application of the fve questionnaires at the same time in a single population.However, the principal strength of the questionnaires corresponds to their NPV, which would favor their use in populations with a lower prevalence of COPD.It is beyond the scope of this study to propose risk factors for COPD that could improve the discriminatory capacity of the questionnaires, but the noninclusion of risk factors such as exposure to wood smoke and the educational level could have impacted their performance.It is desirable to carry out studies in this direction, as well as measure the cost-efectiveness of screening strategies in our population.

Conclusion
Te CDQ, COPD-PS, LFQ, PUMA, and COULD IT BE COPD questionnaires have acceptable performance for the diagnosis of COPD, together with low sensitivity and specifcity.Terefore, its use must be complemented with other diagnostic tests or techniques such as pulmonary function tests.

Figure 1 :
Figure 1: Flowchart showing the selection of subjects and reasons for exclusion from statistical data analysis.COPD: chronic obstructive pulmonary disease.

Figure 2 :
Figure 2: Comparison of the diagnostic performance for COPD among COULD IT BE COPD, CDQ, COPD-PS, LFQ, and PUMA questionnaires.(a) Comparison between the fve questionnaires.(b) Comparison between questionnaires excluding COULD IT BE COPD.AUC: area under the receiver operating characteristics curve.* p value result of the DeLong test.

2
Canadian Respiratory Journal 2.5.Ethical Considerations.Tis study was approved by the Research Committee of the Universidad de La Sabana and the Institutional Ethics Committee of the Clínica Universidad de La Sabana, as registered on Act Minute no.20180928 of September 28, 2022.

Table 1 :
General characteristics of the population.: chronic obstructive pulmonary disease, SD: standard deviation, and IPA: index package/year.Te p value prior diagnosis of COPD, n (%) in Table 1 is <0.001.Te signifcance level of the bold values is <0.05. COPD

Table 2 :
General pulmonary function characteristics of study population.

Table 3 :
Performance of COPD diagnostic questionnaires in study population.