Cardiology Research and Practice
Volume 2020 (2020), Article ID 2469281, 6 pages
https://doi.org/10.1155/2020/2469281

Research Article

Comparison of the CAMI-NSTEMI and GRACE Risk Model for Predicting In-Hospital Mortality in Chinese Non-ST-Segment Elevation Myocardial Infarction Patients

Peng Wang,1,2 Hongliang Cong,¹ Ying Zhang,¹ and Yujie Liu¹

¹Tianjin Medical University, Tianjin, China
²Department of Cardiology, Tianjin Chest Hospital, Tianjin, China

Correspondence should be addressed to Hongliang Cong; chlwenzhang@126.com

Received 24 April 2020; Revised 13 June 2020; Accepted 18 June 2020; Published 24 July 2020

Academic Editor: Robert Chen

Copyright © 2020 Peng Wang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Introduction. The ability of risk models to predict in-hospital mortality and the influence on downstream therapeutic strategy has not been fully investigated in Chinese Non-ST-segment elevation myocardial infarction (NSTEMI) patients. Thus, we sought to validate and compare the performance of the Global Registry of Acute Coronary Events risk model (GRM) and China Acute Myocardial Infarction risk model (CRM) and investigate impacts of the two models on the selection of downstream therapeutic strategies among these patients. Methods. We identified 2587 consecutive patients with NSTEMI. The primary endpoint was in-hospital death. For each patient, the predicted mortality was calculated according to GRM and CRM, respectively. The area under the receiver operating characteristic curve (AUC), Hosmer–Lemeshow (H–L) test, and net reclassification improvement (NRI) were used to assess the performance of models. Results. In-hospital death occurred in 4.89% (126/2587) patients. Compared to GRM, CRM demonstrated a larger AUC (0.809 versus 0.752, ), less discrepancy between observed and predicted mortality (H–L χ²: 22.71 for GRM, and 10.25 for CRM, ), and positive NRI (0.3311, ), resulting in a significant change of downstream therapeutic strategy. Conclusion. In Chinese NSTEMI patients, the CRM provided a more accurate estimation for in-hospital mortality, and application of the CRM instead of the GRM changes the downstream therapeutic strategy remarkably.

1. Introduction

Non-ST-segment elevation myocardial infarction (NSTEMI) is a leading cause of mortality, morbidity, and hospitalization from cardiovascular disease both worldwide and in China, which has a major influence on health economies [1–4]. Unfortunately, lower rather than higher risk NSTEMI patients are more likely to receive more aggressive therapeutic strategies, which is the so-called risk-treatment paradox [5–7]. To diminish the impact of this paradox, the current guidelines consider the risk assessment by the Global Registry of Acute Coronary Events (GRACE) risk model as a fundamental component to select the most appropriate therapeutic strategy for NSTEMI patients [1–3]. However, several observations have suggested that the performance of the GRACE risk model (GRM) was unsatisfactory among Chinese NSTEMI patients [8, 9]. Recently, a novel risk model has been developed for the risk evaluation of NSTEMI patients [8] based on multicenter data from the China Acute Myocardial Infarction (CAMI) registry [10]. To date, the CAMI-NSTEMI risk model (CRM) has not been systematically validated in an external cohort. Moreover, it has not been fully demonstrated how these risk assessment models influenced clinical management. Thus, the present study tends to validate and compare the performance of GRM and CRM and investigate the impacts of the two proposed models on the selection of downstream therapeutic strategies in Chinese NSTEMI patients.

2. Materials and Methods

2.1. Study Participants

The details of inclusion and exclusion criteria for the study participants are illustrated in Figure 1. According to the most recent guidelines, NSTEMI was defined as symptoms of ischemia and detection of an elevation of cardiac troponin values without new persistent ST-segment elevation [1–3]. Three thousand and twenty-five patients with a primary clinical diagnosis of NSTEMI were included. Then, the patients were excluded if they fulfilled one of the following criteria: data missing (217), acute or chronic infectious diseases (116), cancer (68), and acute cerebrovascular disease (37). Among the 217 patients excluded from the analysis for missing data, medical histories for 68% (147/217), results of blood test for 28% (61/217), and clinical presentations for 24% (52/217) were not available. At last, 2587 patients were included in this study and were divided into in-hospital survival and death groups. This observational study complied with the Declaration of Helsinki and was approved by the local Ethics Committees.

Figure 1: The flow diagram. NSTEMI: non-ST-segment elevation myocardial infarction.

2.2. Data Collection and Definitions

The following variables were registered through review of the electronic medical record: age, sex, history of diabetes mellitus, hypertension, hyperlipidemia, smoking, previous revascularization, and previous myocardial infarction. The body mass index (BMI), Killip class, heart rate (HR), systolic blood pressure (SBP), creatinine (Cr), white blood cell count (WBC), cardiac arrest, and ST-segment depression on an 18-lead electrocardiogram at admission were also recorded. Hyperlipidemia was defined as a total cholesterol of at least 220 mg/dl, low-density lipoprotein cholesterol of at least 140 mg/dl, fasting triglycerides of at least 150 mm/dl, or receiving treatment with oral lipid-lowering agents. Diabetes was defined as fasting glucose levels over 7 mmol/l or treatment currently with diet, oral glucose-lowering agents, or insulin. The smoking status included current smoker, nonsmoker, and previous smoker (quit >6 month). The primary endpoint of this study was in-hospital death, defined as all-cause death during hospitalization.

2.3. Risk Assessment by the CRM and GRM

The GRM included 8 independent risk factors: age, Killip class, SBP, ST-segment depression, cardiac arrest, Cr, initial cardiac enzyme findings, and HR [11]. The initial cardiac enzyme findings were positive because all patients have been diagnosed with NSTEMI. According to the GRM and guideline recommendations, we classified patients into 3 risk groups: the low-, medium-, and high-risk group [1]. The CRM identified 11 independent predictors of in-hospital mortality: age, BMI, SBP, Killip class, cardiac arrest, ST-segment depression, Cr, WBC smoking status, previous MI, and previous percutaneous coronary intervention, and all patients were classified into 3 risk groups based on the CRM [8].

2.4. Statistical Analysis

All statistical analyses were carried out by MedCalc (version 15.2.2; MedCalc Software, Mariakerke, Belgium) and R (version 3.2.4; R Foundation for Statistical Computing, Vienna, Austria). A two-tailed value less than 0.05 was considered statistically significant. Continuous variables were compared using Student’s t-tests or Mann–Whitney U-tests as appropriate. Count variables were assessed using the χ²-test or Fisher’s exact test as appropriate. This study conducting validation and comparison of multivariable prediction models strictly followed Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement [12]. To validate and compare the predictive value of the GRM and CRM, we used three characteristics: discrimination, calibration, and classification [13]. Discrimination refers to how well the model differentiates those having an endpoint from those not having. The area under receiver-operator characteristic curve (AUC) was used to quantify the improvement in discrimination [14]. Calibration reflects the extent to which the values predicted by the model agree with the observed values. We used Hosmer–Lemeshow (H–L) tests which divided patients into ten groups according to deciles of mortality and calculated a chi-square statistic (H–L χ²) to assess calibration [15]. As the selection of downstream therapeutic strategy usually bases on the risk classification, we established a reclassification table to evaluate the net reclassification improvement (NRI), determining how correctly a model reclassifies patients into various risk categories compared with another [16].

3. Results

Baseline characteristics of the study cohort are listed in Table 1. Overall, hospital death occurred in 4.89% (126/2587) patients. Compared to the in-hospital survival group, the in-hospital dead group were older, had more male, lower BMI and SBP, and a higher prevalence of diabetes mellitus, hypertension, current smoker, cardiac arrest, high Killip class, and ST-segment depression. Also, HR, Cr, WBC, and troponin T were higher in the in-hospital dead group. All the differences were statistically significant ().

Table 1: Baseline characteristics of patients who died vs. survived.

Characteristic
Total (n = 2587)
In-hospital dead (n = 126)
In-hospital survival (n = 2461)
value

Age
61.91 ± 10.97
72.84 ± 10.69
61.35 ± 9.74
<0.0001

Male
1290 (50)
84 (67)
1206 (49)
0.0001

BMI (kg/m²)
21.24 ± 3.91
21.14 ± 3.23
23.25 ± 3.74
<0.0001

Diabetes mellitus
540 (21)
48 (38)
492 (20)
<0.0001

Hypertension
1462 (57)
84 (67)
1378 (56)
0.0194

Hyperlipidemia
587 (23)
21 (17)
566 (23)
0.1146

Previous revascularization

    PCI
185 (7)
3 (3)
182 (7)
0.0628

    CABG
45 (1.7)
1 (1)
44 (1.8)
0.5846

Previous MI
325 (12)
30 (24)
295 (12)
0.0027

Smoking status

<0.0001

    Current smoker
855 (33)
18 (14)
837 (34)

    Previous smoker
393 (15)
24 (19)
369 (15)

    Nonsmoker
1339 (52)
84 (67)
1255 (51)

HR (beats/min)
78.94 ± 29.47
90.01 ± 27.23
78.37 ± 21.14
<0.0001

SBP (mmHg)
127.70 ± 34.93
109.24 ± 33.73
128.65 ± 22.19
<0.0001

Cr (μmol/L)
85.67 ± 98.64
144.64 ± 93.17
82.65 ± 63.38
<0.0001

WBC (10⁹/L)
8.33 ± 8.07
16.45 ± 7.66
7.91 ± 4.25
<0.0001

Troponin T (ug/l)
0.26 ± 2.24
1.47 ± 4.28
0.20 ± 1.37
<0.0001

Cardiac arrest
58 (2)
9 (7)
49 (2)
0.0007

Killip class

<0.0001

    I
1598 (62)
23 (18)
1575 (64)

    II
463 (18)
20 (16)
443 (18)

    III
329 (13)
34 (27)
295 (12)

    IV
197 (7)
49 (39)
148 (6)

ST-segment depression
1315 (51)
84 (67)
1231 (50)
0.0003

Values are presented as mean ± SD or n (%). BMI: body mass index; CABG: coronary artery bypass grafting; PCI: percutaneous coronary intervention; MI: myocardial infarction; HR: heart rate; SBP: systolic blood pressure; Cr: creatinine; WBC: white blood cell count.

The receiver-operator characteristic curves of 2 models are exhibited in Figure 2. The AUC for the CRM (0.809, 95% confidence interval: 0.789 to 0.829, ) was significantly () larger than that for the GRM (0.752, 95% confidence interval: 0.729 to 0.774, ).

Figure 2: Comparison of 2 models by receiver operating characteristic curves. GRM: GRACE risk model; CRM: CAMI-NSTEMI risk model.

The predicted mortality was compared with the observed mortality in deciles of predicted mortality as illustrated in Figure 3. The GRM manifested a predominance of underestimation, resulting in a poor calibration (H–L χ² = 22.71, ). The differences between the observed and predicted mortality were not evident in H–L calibration plots for CRM, so the calibration of the CRM was good (H–L χ² = 10.25, ).

(a)

(b)

Figure 3: Predicted and observed mortality by deciles of predicted mortality. GRM: GRACE risk model; CRM: CAMI-NSTEMI risk model.

Table 2 shows the reclassification table comparing the CRM to GRM. For the 126 positive patients, compared to the GRM, the CRM correctly reclassified 23 from the medium- to high-risk category, 12 from low to high, and 6 from low to medium, but 2 from medium to low and 2 from high to medium. Of the 2461 negative patients, 174 were correctly reclassified to a lower risk category but 82 to a higher PTP category. As a result, compared to the GRM, the NRI for the CRM was 0.2937 in positive, 0.0374 in negative, and 0.3311 overall (). In other words, the replacement of the GRM by CRM for every 2587/(23 + 12 + 6 + 174)≈13 patients would result in 1 correct reclassification.

Table 2: Reclassification table comparing the CRM to GRM.

Risk category based on the CRM
Total
Reclassification
NRI†

Low
Medium
High
Up
Down

Risk category based on the GRM

Negative patients

0.0333
0.0707
0.3311
<0.0001

    Low
932
37
38
1007

    Medium
156
871
7
1034

    High
6
12
402
420

    Total
1094
920
447
2461

Positive patients‡

0.3254
0.0317

    Low
1
6
12
19

    Medium
2
3
23
28

    High
0
2
77
79

    Total
3
11
112
126

The classification of patients by the CRM was compared to that by the GRM. †NRI = (P (Up|Positive)-P (Down|Positive)]-[ P (Up|Negative)-P (Down|Negative)). ‡A positive patient was defined as a patient who died in hospital. NRI: net reclassification improvement; GRM: GRACE risk model; CRM: CAMI-NSTEMI risk model.

4. Discussion

This observational analysis determined that the CRM provided a more effective prediction for in-hospital mortality in Chinese NESTEMI patients. Compared to the GRM, the CRM demonstrated a larger AUC, less discrepancy between the observed and predicted mortality, and a positive NRI. It was worth noting that the application of the CRM instead of the GRM may pronouncedly change the downstream therapeutic strategy in these NESTEMI patients.

Compared to ST-segment elevation myocardial infarction (STEMI) patients, NSTEMI patients present with more heterogeneous variation in ischemic risk and comorbidities so that the risk classification has been considered as a fundamental component to select the most appropriate therapeutic strategy for NSTEMI patients [1–3, 17]. However, in clinical practice the treatment-risk paradox is widespread [5–7], which is partly due to the suboptimal risk assessment underestimating the ischemic risk [18, 19], especially in China [4, 6, 20]. In conformity with this, according to the H–L calibration plots in the present study, the GRM dramatically underestimated the in-hospital mortality and classified nearly 40% in-hospital death into the low- or medium-risk category, which may cause underuse of invasive strategy for patients at high risk [1, 2]. Two reasons may potentially account for the unfavorable performance when applying GRM to the Chinese NSTEMI patients. First, GRM was developed in patients mainly from America, Europe and Australia twenty years ago [11]. There are significant differences in characteristics and management between these and contemporary Chinese patients [3, 11]. Second, the GRM was developed to assess the risk of patients with acute coronary syndrome, including STEMI, NSTEMI, and unstable angina pectoris [11].

To address the limitations mentioned above, the CRM was developed to predict in-hospital mortality, particularly for Chinese NSTEMI patients, and its diagnostic performance was superior to that of the GRM in two internal validation studies using data from CAMI registry [8, 9]. Our effort further extended this conclusion in an external validation cohort by indicating a larger AUC, positive NRI, and less disagreement between the observed and predicted mortality for the CRM. Moreover, the GRM classified 15% (19/126) in-hospital dead into the low-risk category, for which invasive strategy was not recommend according to current guidelines and 22% (28/126) into the medium-risk category, for which immediate (<2 h) and early (<24 h) invasive strategy were not recommend [1]. Using the CRM instead of the GRM would imply a significant change for downstream therapeutic strategy in the in-hospital dead: 87% (36/47) of these patients would be reclassified into the higher risk category, for which more aggressive strategies were recommended. Thus, application of the CRM instead of the GRM may have the potential to optimize the referral of aggressive intervention and lead to an evident diminution of the treatment-risk paradox in NSTEMI patients.

This study was subjected to the limitations of its single-center, retrospective, and observational design. The indication of clinical management and downstream therapeutic strategy were based on the individual physician decision. Thus, the strategic discordance of downstream management for NSTEMI patients should not be ignored, and the actual impact of applying the CRM was complicated. To further investigate the generalizability and reliability of CRM-guided therapeutic strategy, more pragmatic and cost-effective randomized control trials are needed, such as the UK GRACE Risk Score Intervention Study (UKGRIS) [21].

5. Conclusions

In Chinese NSTEMI patients, the CRM provided a more effective estimation for in-hospital mortality due to the improvement in discrimination, classification, and calibration compared to the GRM. The application of the CRM instead of the GRM could change the diagnostic strategy and the potential to optimize the referral of a more aggressive intervention.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This study was funded by the Key Project of Scientific and Technological Support Plan of Tianjin in 2018 (No.16ZXMJSY00140).

References

M. Roffi, C. Patrono, J.-P. Collet et al., “2015 ESC guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation,” European Heart Journal, vol. 37, no. 3, pp. 267–315, 2016.

E. A. Amsterdam, N. K. Wenger, R. G. Brindis et al., “2014 AHA/ACC guideline for the management of patients with non-ST-elevation acute coronary syndromes: a report of the American college of cardiology/American heart association task force on practice guidelines,” Journal of the American College of Cardiology, vol. 64, no. 24, pp. e139–e228, 2014.

D. Zhao, J. Liu, M. Wang, X. Zhang, and M. Zhou, “Epidemiology of cardiovascular disease in China: current features and implications,” Nature Reviews Cardiology, vol. 16, no. 4, pp. 203–212, 2019.

Q. Zhang, D. Zhao, W. Xie et al., “Recent trends in hospitalization for acute myocardial infarction in beijing: increasing overall burden and a transition from ST-segment elevation to non-ST-segment elevation myocardial infarction in a population-based study,” Medicine (Baltimore), vol. 95, no. 5, Article ID e2677, 2016.

M. Hall, O. J. Bebb, T. B. Dondo et al., “Guideline-indicated treatments and diagnostics, GRACE risk score, and survival for non-ST elevation myocardial infarction,” European Heart Journal, vol. 39, no. 42, pp. 3798–3806, 2018.

Q. Yang, Y. Wang, J. Liu et al., “Invasive management strategies and antithrombotic treatments in patients with non-ST-segment-elevation acute coronary syndrome in China: findings from the improving CCC Project (care for cardiovascular disease in China),” Circulation Cardiovascular Interventions, vol. 10, no. 6, 2017.

A. Saar, T. Marandi, T. Ainla, K. Fischer, M. Blondal, and J. Eha, “The risk-treatment paradox in non-ST-elevation myocardial infarction patients according to their estimated GRACE risk,” International Journal of Cardiology, vol. 272, pp. 26–32, 2018.

R. Fu, C. Song, J. Yang et al., “CAMI-NSTEMI score-China acute myocardial infarction registry-derived novel tool to predict in-hospital death in non-ST segment elevation myocardial infarction patients,” Circulation Journal: Official Journal of the Japanese Circulation Society, vol. 82, no. 7, pp. 1884–1891, 2018.

C. Song, R. Fu, S. Li et al., “Simple risk score based on the China acute myocardial infarction registry for predicting in-hospital mortality among patients with non-ST-segment elevation myocardial infarction: results of a prospective observational cohort study,” BMJ Open, vol. 9, no. 9, Article ID e030772, 2019.

H. Xu, W. Li, J. Yang et al., “The China Acute Myocardial Infarction (CAMI) Registry: a national long-term registry-research-education integrated platform for exploring acute myocardial infarction in China,” American Heart Journal, vol. 175, pp. 193–201.e3, 2016.

C. B. Granger, R. J. Goldberg, O. Dabbous et al., “Predictors of hospital mortality in the global registry of acute coronary events,” Archives of Internal Medicine, vol. 163, no. 19, pp. 2345–2353, 2003.

G. S. Collins, J. B. Reitsma, D. G. Altman, and K. G. M. Moons, “Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. The TRIPOD group,” Circulation, vol. 131, no. 2, pp. 211–219, 2015.

A. C. Alba, T. Agoritsas, M. Walsh et al., “Discrimination and calibration of clinical prediction models: users’ guides to the medical literature,” JAMA, vol. 318, no. 14, pp. 1377–1384, 2017.

E. R. DeLong, D. M. DeLong, and D. L. Clarke-Pearson, “Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach,” Biometrics, vol. 44, no. 3, pp. 837–845, 1988.

A. A. Kramer and J. E. Zimmerman, “Assessing the calibration of mortality benchmarks in critical care: the Hosmer-Lemeshow test revisited,” Critical Care Medicine, vol. 35, no. 9, pp. 2052–2056, 2007.

M. J. Pencina, R. B. D’Agostino Sr., R. B. D’Agostino Jr., and R. S. Vasan, “Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond,” Statistics in Medicine, vol. 27, no. 2, pp. 157–172, 2008.

A. W. Schoenenberger, D. Radovanovic, S. Windecker et al., “Temporal trends in the treatment and outcomes of elderly patients with acute coronary syndrome,” European Heart Journal, vol. 37, no. 16, pp. 1304–1311, 2016.

D. P. Chew, G. Junbo, W. Parsonage et al., “Perceived risk of ischemic and bleeding events in acute coronary syndromes,” Circulation Cardiovascular Quality and Outcomes, vol. 6, no. 3, pp. 299–308, 2013.

I. Ahrens, O. Averkov, E. C. Zuniga et al., “Invasive and antiplatelet treatment of patients with non-ST-segment elevation myocardial infarction: understanding and addressing the global risk-treatment paradox,” Clinical Cardiology, vol. 42, no. 10, pp. 1028–1040, 2019.

W. Leng, J. Yang, X. Fan et al., “Contemporary invasive management and in-hospital outcomes of patients with non-ST-segment elevation myocardial infarction in China: findings from China Acute Myocardial Infarction (CAMI) Registry,” American Heart Journal, vol. 215, pp. 1–11, 2019.

C. C. Everett, K. A. Fox, C. Reynolds et al., “Evaluation of the impact of the GRACE risk score on the management and outcome of patients hospitalised with non-ST elevation acute coronary syndrome in the UK: protocol of the UKGRIS cluster-randomised registry-based trial,” BMJ Open, vol. 9, no. 9, Article ID e032165, 2019.