Prognostic Accuracy of qSOFA and SIRS for Mortality in the Emergency Department: A Meta-Analysis and Systematic Review of Prospective Studies

Objective This meta-analysis aimed to determine the prognostic performance of quick sequential organ failure assessment (qSOFA) score in comparison to systemic inflammatory response syndrome (SIRS) in predicting in-hospital mortality in the emergency department (ED) patients. Methods Eligible studies comparing the performance of qSOFA and SIRS in predicting in-hospital death of ED patients were identified from searching PubMed, Embase, and Cochrane. Raw data were collected, and the pooled sensitivity and specificity were calculated for qSOFA and SIRS. The summary receiver operating curve was also plotted to calculate the area under the curve. Results A total of 16 prospective studies with 35,756 patients and 2,285 deaths were included. The pooled sensitivity was 0.43 (95% CI: 0.32–0.54) and 0.8 (95% CI: 0.73–0.86) for qSOFA and SIRS, respectively. The pooled specificity was 0.89 (95% CI: 0.84–0.93) and 0.39 (95% CI: 0.3–0.5) for qSOFA and SIRS, respectively. The area under the summary receiver operating curve was 0.76 (95% CI: 0.72–0.8) and 0.67 (95% CI: 0.62–0.72) for qSOFA and SIRS, respectively. A significant heterogeneity was observed for both qSOFA and SIRS studies. Conclusion The present meta-analysis suggested that qSOFA had a higher specificity but a lower sensitivity as compared with SIRS in predicting in-hospital mortality in the ED patients. qSOFA appeared to be a more concise and simple way to recognize patients at high risk for death. However, the use of SIRS in the ED cannot be completely replaced since the sensitivity of qSOFA was relatively lower.


Introduction
Sepsis is a leading cause of critical morbidity and mortality in the emergency department (ED) worldwide [1,2], accounting for approximately 10% of intensive care unit (ICU) admission and 10-20% of in-hospital death [3][4][5]. However, recognizing patients with suspected infection who are at higher risk of sepsis and subsequent death in the ED setting is still a crucial challenge for clinicians. erefore, an easyto-use, fast, and accurate measure for predicting the prognosis of sepsis might be of great help for ED clinicians to determine whether more intensive monitoring and aggressive treatment should be applied to prevent adverse outcomes and ultimately in-hospital mortality. So far, a few scoring systems, such as systemic inflammatory response syndrome (SIRS) criteria, quick sequential organ failure assessment score (qSOFA), and National Early Warning Score (NEWS), have been devised for patients with suspected sepsis. ese systems are considered as an effective tool for patients with critical illness, especially those in ICU [6]. However, in the ED setting, the performance and applicability of these scoring systems remain undetermined. e first sepsis criterion, SIRS, was published in 1992 by the American College of Chest Physicians/Society of Critical Care Medicine consensus [7]. Accordingly, 4 SIRS criteria were defined as follows: tachypnea (respiratory rate more than 20 breaths per minute), tachycardia (heart rate more than 90 beats per minute), leukocytosis or leukopenia (leukocyte count more than 12,000 per μL or less than 4000 per μL, respectively), and fever or hypothermia (body temperature >38°C or <36°C, respectively). e consensus definition of sepsis required a defined infection and two or more SIRS criteria. ereafter, in a number of studies, SIRS has been found to be of high sensitivity but unsatisfactory specificity for the diagnosis of sepsis and predicting inhospital death [8]. In recent years, sepsis has been redefined by an international task force in e ird International Consensus Definitions for Sepsis and Septic Shock [9]. A new set of criteria, qSOFA, was adopted after the Sepsis-3 recommendations and was defined as follows: systolic arterial blood pressure no higher than 100 mmHg, respiratory rate more than 21 breaths per minute, and altered mental status. Increased qSOFA score was considered to be related with higher probability of mortality. e definition group stated that qSOFA might be a better indicator for in-hospital death as compared with SIRS and suggested using a qSOFA score of ≥2 to recognize patients at high risk for death due to infection, instead of a SIRS score of ≥2. However, the advantages and disadvantages of both traditional and new criteria in specific clinical settings, such as the ED, remain to be further clarified.
Since qSOFA was proposed, there have been emerging clinical investigations comparing the prognostic value of SIRS and qSOFA in predicting mortality of patients with suspected infection in the ED. Quite a few studies revealed a better performance of the new criteria. For instance, Freund et al. prospectively compared the predictive ability of qSOFA with the previous ones in the ED setting and observed a greater prognostic accuracy for in-hospital mortality of qSOFA than did either SIRS or severe sepsis [10].
us, there is still significant controversy regarding the merits and shortcoming of qSOFA and SIRS.
Two years after qSOFA was proposed, Jiang et al. performed a head-to-head meta-analysis to compare qSOFA and SIRS criteria in predicting the mortality of infected patients admitted to the ED [14]. ey included 8 studies with a total of 52,849 patients and observed that qSOFA score ≥2 indicated a higher probability of mortality in ED patients with infections than SIRS score ≥2 (risk ratio � 4.55 versus 2.75) [14]. However, Jiang et al. also stated that their meta-analysis included a relatively small number of studies (n � 8), and there were limited number of prospective studies (n � 4), which might not have enabled a complete evaluation of the predictive ability of qSOFA and SIRS [14]. erefore, with the emerging clinical data on the comparison of qSOFA and SIRS in recent 3 years, we further performed this metaanalysis based on prospective studies. In the present metaanalysis, we aimed to compare the prognostic accuracy of qSOFA and SIRS for mortality in the emergency department including the diagnostic odds ratio (OR), sensitivity, and specificity as well as the area under the summary receiver operating curve (SROC).

Search Strategy and Selection Criteria.
We searched all literature focusing on the performance of qSOFA and SIRS for the mortality of patients visiting ED through November 2021 in the PubMed, Embase, and Cochrane databases. e key words for literature search were as follows: for ED: emergency department, emergency, emergencies, urgent, and emergent; for qSOFA: qSOFA, Sequential Organ Failure Assessment, or Organ Failure Assessment; for SIRS: SIRS or systemic inflammatory response syndrome or inflammatory response syndrome. e literature search was supplemented by checking the reference lists of the literature identified. e literature was managed by using EndNote (version X7). e protocol of this meta-analysis was registered in the International Prospective Register of Systematic Reviews (PROSPERO, registration ID: CRD42021289556). e eligibility of searched literature for inclusion was independently reviewed by two authors, and the divergence in the decision was issued via discussion with the other author. All human-based studies were considered eligible for inclusion if the following criteria were met: (1) prospective study; (2) the outcome was inhospital death; (3) the data of sensitivity or specificity for predicting in-hospital death were reported for both qSOFA and SIRS. e literature was excluded if it was: (1) duplicate literature; (2) review, meta-analysis, or guideline; (3) case report, letter, comment, editorial, protocol, or reply; (4) with less than 20 patients; (5) basic research; (6) topic nonrelevant; (7) not in English; (8) without appropriate data ( Figure 1). To fulfill the aim of the meta-analysis and obtain a homogeneous population, the participants of included studies should also fit the following criteria: (1) consecutively enrolled; (2) patients visiting the emergency department; (3) both qSOFA and SIRS were prospectively obtained at the emergency department; (4) repeat admissions during the study period should be excluded; (5) patients transferred 2 Emergency Medicine International from another hospital with pretreatment should be excluded; (6) age >16 years.

Data Extraction and Definitions.
Raw data were collected by two authors independently. If there existed dissonance of extracted data by the two authors, a third author would preside over a discussion until consensus was achieved. e following data were collected: first author, time of publication, study location, size and age of population, participant selection, number of deaths, and sensitivity and specificity of qSOFA and SIRS for in-hospital death.
e primary outcome was in-hospital death, defined as patients visiting the ED and died with 28 days during treatment in hospital. If mortality was evaluated within both 28 days and other time-span, 28 days was the priority choice. If mortality was estimated only within other time-span such as 7 days, then it was also considered as appropriate data.

Data Synthesis and Statistical Analysis.
e MIDAS module of the STATA software, version 12.0 (Stata Corporation, College Station, TX), and Meta-DiSc 1.4 (XI Cochrane Colloquium, Barcelona, Spain) were used to calculate the overall sensitivity, specificity, and SROC. e data were presented as the pooled estimates with 95% CIs. e heterogeneity was evaluated by the Q and I 2 statistics. e posttest probability was also calculated and depicted through the MIDAS module of the STATA software based on Bayes' theorem. To estimate the possible sources of heterogeneity, subgroup analysis was performed according to the characteristics of included literature such as study location (Europe, Asia, or others), publication time (before 2020 or 2020 and after), and bias risk of articles (low risk, unclear, and high risk). A sensitivity analysis was also conducted by omitting one literature at each analysis.

Assessment of Quality of Studies and Publication Bias.
e quality of study and publication bias were evaluated by RevMan version 5.3 via assessing the risk of bias (patient selection, index test, reference standard, and flow and timing) and applicability concerns (patient selection, index test, and reference standard). e original dataset used for analysis is available in the Supplementary Materials.

Search Results and Study Characteristics.
e flowchart of literature selection is shown in Figure 1. A total of 1,312 and 42 literature were initially identified through database searching and citation searching, respectively. After removal of duplicates and screening by reviewing title, abstract, and full-text, a total of 16 prospective studies [10][11][12][13][15][16][17][18][19][20][21][22][23][24][25][26]   Emergency Medicine International causes. e mean/median age of included population ranged from 30 to 83.6 years. e quality was considered good for all included articles, probably due to the prospective study design ( Figure 2). However, there was possible risk of patient selection bias in the studies by Boillat-Blanco et al. [16], Graham et al. [13], Loritz et al. [20], and Ranzani et al. [22]. e risk of bias was mainly because these studies included nonselective patients or patients of other causes, which were significantly different from the majority population with suspected or confirmed infection.
Taken together, qSOFA had relatively lower sensitivity but higher specificity as compared with SIRS. e performance of qSOFA in estimating in-hospital mortality was better than SIRS with a higher AUC.

Subgroup Analysis.
A subgroup analysis restricted to different study locations, publication time, and bias risk of articles was performed (Table 2 and Supplementary Materials). In all of the subgroups, qSOFA still showed lower sensitivity but higher specificity than SIRS did. Only in the studies (n � 4) performed in the region other than Europe and Asia, the heterogeneity could be reduced to zero, while in all the other subgroups, the heterogeneity remained significant (I 2 > 75%). us, the heterogeneity might mainly result from the different study locations.

Sensitivity Analysis of Sensitivity and Specificity for qSOFA and SIRS.
e sensitivity of qSOFA varied from 0.4 to 0.46, and the sensitivity varied from 0.88 to 0.91 when omitting one study at each analysis (Table 3 and Supplementary Materials). On the other hand, the sensitivity of SIRS ranged from 0.79 to 0.82 and the sensitivity ranged from 0.37 to 0.42 when omitting one study at each analysis. is suggested that the results were relatively stable since no significant variation was observed when each of the studies was omitted.

Discussion
Early detection and intervention of sepsis have been paid increasing attention due to the rising incidence of sepsisrelated death and health expenses in recent years. e      Emergency Medicine International systematic review including only prospective studies comparing the prognostic value of the new sepsis definitions with the previous definitions in predicting in-hospital death in ED patients with suspected infections. As is known, the superiority of prospective study is that the detailed data of qSOFA and SIRS score are collected before the outcome of in-hospital death and therefore cannot be biased by the outcome. us, the results in this meta-analysis of prospective studies might be more accurate and further demonstrated the conclusion.
In certain circumstance, the sensitivity of diagnostic criteria is also important for the detection and mortality of sepsis, especially in the ED. As for the case of sepsis with high mortality, the potential life-threatening risk should be paid more attention than the cost for false positivity (unnecessary antibiotics or unnecessary hospitalizations). A test with high sensitivity is necessary for cases in which missing a potentially severe disease condition may lead to adverse clinical consequences. us, a diagnostic criterion with high sensitivity is also preferred in recognizing of sepsis and predicting in-hospital death [27]. In most previous studies [24][25][26] as well as in the present meta-analysis, SIRS showed a relatively higher sensitivity than qSOFA. erefore, SIRS is also useful in initially screening patients with sepsis and is helpful for avoiding missed diagnosis. On the other hand, a highly specific test is also required since false-positive diagnosis may result in excessive burden on the patients. A highly sensitive bedside diagnostic tool can detect most of the patients with severe disease at an early stage and facilitate early intervention, while a highly specific diagnostic test can help clinicians in preventing overdiagnosis, overexamination, and overtreatment [28]. us, both SIRS and qSOFA criteria contribute to the identification of infected patients at high risk of serious outcomes in the ED. Besides, the qSOFA result could vary rapidly over a short time frame during in-hospital treatment, suggesting that a solitary calculation of qSOFA criteria on first visit might be insufficient. Future studies should investigate the relation of serial measurements of qSOFA and the adverse outcome, which might help more accurately determine the better time window for measuring qSOFA criteria during the ED stay. e linear combination method of these score systems can simplify the bedside evaluation process, but the effect of the terms should not be as simple as 0 or 1. ere are several ways to address the problem such as the ensemble modeling method and the nomogram, which can automatically take the nonlinearity effect of variables into analysis during the modeling process. ese complex but more accurate model methods will be practicable as the electronic technology improves in the near future. Also, more studies are     Emergency Medicine International warranted to further investigate the prognostic effect of these terms for calculating risks. ere are several strengths of this meta-analysis. is meta-analysis only includes prospective studies, thus minimizing the potential risk of reporting bias and missing data which are frequently observed in many retrospective studies of sepsis. e prospective design of the included studies ensures that all patients visiting the ED are consecutively screened and included in the research. Furthermore, to the best of our knowledge, this is the largest meta-analysis of prospective studies comparing the performance of qSOFA and SIRS for patients with suspected infection in the ED setting. However, the heterogeneity is a limitation of this meta-analysis. e heterogeneity can be reduced in the subgroup of study location, suggesting that the heterogeneity mainly originates from the different regions of study populations. Notably, in the former meta-analysis by Jiang et al. [14], the heterogeneity was also significant in the   Emergency Medicine International combined population as well as in subgroups, yet the study location-associated heterogeneity was not evaluated in their analysis. us, further research should pay more attention to the variation of population characteristics in different geographical locations.

Conclusion
In summary, our analysis suggested that both qSOFA score and SIRS score were helpful risk stratification tools which are strongly associated with in-hospital death for patient in the ED with infections. qSOFA appears to be a more concise and effective index to detect patients at high risk for death. However, the sensitivity of qSOFA was relatively lower and the use of SIRS in the ED cannot be completely replaced.
Data Availability e data supporting the findings of this study are included within the article and its supplementary materials. ). e sponsors had no involvement in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.