A Critical Overview of Systematic Reviews of Shenfu Injection for Heart Failure

Objectives Shenfu Injection (SFI) was widely used in the treatment of heart failure (HF) in China. A plethora of systematic reviews/meta-analyses (SRs/MAs) has been conducted in this research area, although with scattered results. The purpose of this overview was to conduct a comprehensive review to summarize and critically evaluate the existing evidence. Methods Digital databases were searched for SRs/MAs up to January 28, 2021. Two authors independently screened the reviews and assessed the methodological quality of included SRs/MAs using Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2). Quality of evidence for outcomes evaluated within the reviews was appraised with the Grading of Recommendation, Assessment, Development, and Evaluation (GRADE). Results Thirteen SRs/MAs met the inclusion criteria. Based on AMSTAR-2, the quality of all SRs/MAs was critically low, because all of them have more than one critical domains that were unmet. Based on GRADE, the evidence quality of 24 outcome measures was low or very low, 27 outcome measures was moderate, and none outcome measure was high. Descriptive analysis showed that SFI was an effective and safe method for HF. Conclusions The use of SFI for the treatment of HF may be clinically effective and safe. However, this conclusion must be interpreted cautiously due to the generally low methodological quality and low evidence quality of the included SRs/MAs. More rigorously designed SRs/MAs and RCTs with high methodological quality are necessary for further proof.


Introduction
Heart failure (HF) is one of the leading causes of human morbidity and mortality worldwide. HF is a complex clinical syndrome with broad pathological processes, exhibiting an unpredictable trajectory and an escalating symptom profile along with time [1]. HF incidence remained stable in recent decades, with almost 26 million people suffered from heart failure around the world [2]. It was reported that approximately 10 per 1000 among those over 65 years of age in the United States [3] and 9 per 1000 of the population aged 35-74 years in China [4] have clinical manifestations HF.
Over the past 30 years, improvements in treatments that consist of some effective medicines, such as diuretics, digoxin, angiotensin-converting enzyme inhibitors (ACEIs), angiotensin receptor blockers (ARBs), and β-blockers, have improved survival and reduced the hospitalization rate in patients with HF. However, it cannot obtain a desired effect own to poor compliance, lower heart rate of patients, and other questions [5]. Considering the above multiple factors, a combination of Chinese herbal injection and western medicine (WM) treatment has already been a supportive measure in the treatment of HF in China. Shenfu Injection (SFI) has been used in treating cardiac diseases for a long time in China; pharmacological studies have suggested that SFI can reduce peripheral circulation resistance and improve microcirculation [6]. A literature search yielded several published systematic reviews (SRs)/meta-analyses (MAs) on SFI for HF, but their quality varied, and the evidence for the effectiveness of SFI is controversial. To comprehensively evaluate the evidence and applicability of the results of SRs/MAs on SFI for HF, we composed an overview.

Eligibility Criteria
2.1.1. Type of Studies. All peer-reviewed, full-reported SRs/MAs based on randomized controlled trials were included. Duplicate reports, studies with the data were inconsistent or incomplete, and unavailable articles were excluded. No language limitation exists.

Types of Participants.
Participants with HF should be confirmed according to any internationally recognized or accepted clinical guidelines. There are no limitations in age, gender, race, or nationality.

Types of Interventions.
The intervention methods were SFI or SFI plus WM (e.g., cardiotonic, diuretic, ACEIs, β-blocker, and so forth); the control groups were treated with WM or blank controls.

Search Strategy.
A systematic search was conducted in 8 databases including PubMed, Embase, the Cochrane Library, the web of science, China National Knowledge Infrastructure, Wanfang Database, Chongqing VIP, and Sino-Med from their establishment to May 17, 2020, with the following search terms: heart failure, shenfu injection, systematic review, and meta-analysis. We conducted an updated search on January 28, 2021, to provide more up-to-date and comprehensive evidence. Besides, we also search systematic review or meta-analysis registration website (https://www .crd.york.ac.uk/PROSPERO/) and checked the reference lists of all relevant SRs identified, and their authors were contacted to identify additional relevant SRs if necessary. Table 1 provides a search strategy for the PubMed database.

Data Collection and Extraction.
Two authors independently screened all potential abstracts and titles of reviews for inclusion, based on the selection criteria. Each review was evaluated independently, and the full texts of all potentially eligible were obtained for assessment to determine whether the review met the inclusion/exclusion criteria.
Any disagreement regarding the possible inclusion/exclusion of any individual review was resolved by discussion with the third reviewer and by a final group consensus.
Two authors independently extracted data from eligible SRs/MAs. From each study, the following specific characteristics were extracted: the first author, year of publication, country, number of trials and participants and their characteristics, quality of the included trials (as reported by the review authors), interventions and comparisons relevant to this overview, outcomes relevant to this overview, quality assessment methods, and the summary estimate of the intervention effects. The corresponding authors were contacted by email for missing information.
2.4. Quality Assessment. Two authors separately evaluated the quality of included SRs/MAs by using Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2) [7]. AMSTAR-2 evaluates the systematic review using 16 distinct criteria, and seven of them are critical items. Each criterion of AMSTAR has 3 choices, namely, "yes," "partial yes," or "no." When no or only 1 noncritical item did not conform, inferring rating overall confidence in the results of the review as high; when more than 1 noncritical item did not conform, inferring rating overall confidence in the results of the review as moderate; when 1 critical item did not conform with noncritical items conforming or not conforming, inferring rating overall confidence in the results of the review as low; and when more than 1 critical items did not conform with noncritical items conforming or not conforming, inferring rating overall confidence in the results of the review as critically low [7].
The evidence quality for each outcome measure was assessed with the Grade of Recommendation, Assessment, Development, and Evaluation (GRADE) [8] by two authors independently. Relevant evidence can be rated down for high risk of bias of included reviews, indirectness, imprecision, inconsistency, and publication bias. The GRADE assesses the certainty of the evidence for each outcome measures by categorizing evidence into "high," "moderate," "low," or "very low" [9]. Any discrepancies were resolved by a final consensus among all reviewers. Descriptive analysis was used for efficacy evaluation.

Study
Characteristics. The characteristics of included SRs/MAs are presented in Table 2. The included reviews were published between 2009 and 2020. Twelve SRs/MAs were published in Chinese [10][11][12][13][14][15][16][17][18][19][20]22], and the remaining 1 [21] was in English. The number of RCTs included in the SRs/MAs varied widely, ranging from 8 to 97 studies, and the sample size varied from 559 to 8272 participants. The intervention of the control groups was WM treatment, for instance, ACEIs, β-blocker, cardiotonic, and diuretic. In the meantime, the treatment groups received SFI on the basis of the control groups. The methodological quality of the original studies using various appraisal tools was assessed in all SRs/MAs as mainly fair or poor.

Methodological
Appraisal. An overview of methodological quality of included SRs/MAs is given in Table 3. All SRs/MAs were regarded as critically low quality. The result of AMSTAR-2 showed that the key factors affecting the quality of the reviews included were item 2 (none review contained an explicit statement that the review methods were established prior to the conduction of the review and justified any significant deviations from the protocol), item 4 (only 3 included studies provide the use of a specific search strategy),     Cardiovascular Therapeutics item 7 (all review authors did not provide a list of excluded studies and justified the exclusions), and item 15 (two reviews did not consider the publication bias when the authors interpreted or discussed the study results).

GRADE Evidence Quality
Classification. The quality of evidence for 46 outcomes in 13 included SRs/MAs is presented in Table 4. Of these outcomes, the quality of evidence was high in 0 (0/51, 0%), moderate in 27 (27/51, 52.9%), low in 18 (18/51, 35.3%), and very low in 6 (6/51, 11.8%). The evidence level of all concerned outcomes was downgraded due to the study limitations within the original trials, inconsistency, imprecision, and the possibility of publication bias. Details regarding downgrades for each GRADE domain by outcome are given in Table 4.

Description of Efficacy
3.5.1. Effectiveness of SFI for HF. We summarized the outcomes from the included SRs/MAs and presented them in Table 4. The evidence in eleven SRs/MAs [11][12][13][14][15][16][17][18][19][20][21][22] [13] revealed that there was a significantly greater reduction in MLHFQ score in the SFI group than in the control group. Readmission rates were reported in 1 review [13]; metaanalysis showed that there was statistical significance between the SFI group and the control group. Mortality rate was reported in 2 reviews [13,21]. Song et al. [21] found that SFI can significantly reduce mortality of patients; however, another review [13] reported no significant difference between the SFI group and the control group.  [11,20,21] reported the following symptoms of side effects including dry mouth, dryness heat, fullness of the head, insomnia, dysphoria, skin itching, tachycardia, feverish dysphoria, flushing of face, tidal fever, dizziness due to low blood pressure, gastrointestinal discomfort, and palpitation.

Discussion
The impairment of HF has been a global public health issue; with the utilization of conjunction between SFI and WM in its treatment, the efficacy of HF has been promoted; meanwhile, more and more relevant SRs/MAs were carried out. Thus, the vast number of SRs/MAs on this topic is concerning, particularly those of low quality which may propagate inaccurate or biased results and conclusions. Under the circumstances, this approach of synthesizing findings of SRs/MAs is better than a high number of SRs/MAs with low quality and unconvincing conclusions, thereby providing a comprehensive evidence-based summary on evident outcomes. In addition, an overview may provide notable information to guide future high-quality RCTs or SRs/MAs.     The confidence interval overlaps less, the heterogeneity test P is very small, and the I 2 is larger. ③ Confidence interval is not narrow enough, or the sample size is small. ④ Fewer studies are included, and there may be greater publication bias. ⑤ Funnel graph asymmetry. 7 Cardiovascular Therapeutics 4.1. Major Study Findings. This is the first overview of SRs/MAs that investigate the effectiveness and safety of SFI for HF. We rigorously appraised the published SRs/MAs with AMSTAR-2 and GRADE. For AMSTAR-2, all included SRs/MAs were judged to be of critically low quality. The key factors affecting the quality of the reviews included were item 2, item 4, item 7, and item 15. Based on GRADE, the quality of evidence was high in 0 (0/51, 0%), moderate in 27 (27/51, 52.9%), low in 18 (18/51, 35.3%), and very low in 6 (6/51, 11.8%). The evidence level of all concerned outcomes was downgraded. The study limitations within the original trials was the most common of the downgrading factors of the evidence level, followed by inconsistency, the possibility of publication bias, and imprecision. Descriptive analysis showed that SFI was an effective and safe method for HF.

Implications for Clinical
Practice. This overview included 51 outcome measures and almost all of which reached positive conclusions; all included SRs/MAs indicated that SFI is effective in treating HF. However, the evidence level of all concerned outcomes was unsatisfactory, indicating that the conclusions of included SRs/MAs may differ from the true results; the lower the quality is the more likely further research would change our confidence in the estimates and the estimates themselves [23].
Additionally, AMSTAR-2 results showed that the methodology quality of all included SRs/MAs was critically low. As we know, high quality of SRs/MAs is crucial to ensure validity, clarity, and accurate comprehension of evidence, while lowquality SRs/MAs are the opposite. Furthermore, authors of most SRs/MAs did not wish to draw definitive conclusions due to the small size of the included trials or their low quality. Therefore, definitive conclusions were impossible to draw from published results; caution should be warranted when recommending SFI as an alternative treatment for HF.

Implications for Further Study.
Based on AMSTAR-2, the key factors affecting the quality of the reviews included were item 2 (none review contained an explicit statement that the review methods were established prior to the conduction of the review and justified any significant deviations from the protocol). The previous study [24] has shown that research protocols help to increase the transparency of the study methods and improve the overall methodological quality of SRs/MAs. Item 4 (only 3 included studies provide the use of a specific search strategy) is likely contributed to generating publication bias and undermined the conclusion's reliability. Item 7 (all review authors did not provide a list of excluded studies and justified the exclusions) is likely contributed to leaving some information missing and undermined the conclusion's reliability. Item 15 (2 reviews did not consider the publication bias when the authors interpreted or discussed the study results) may affect the credibility of the final results. Thus, future SRs/MAs should address these identified shortcomings. Researchers should ensure that the AMSTAR-2 is strictly followed before publication.
Based on GRADE, the evidence level of all concerned outcomes was downgraded. The limitations within the origi-nal trials were the most common of the downgrading factors. Although all the interventions were from RCTs, results showed that there is much room for addressing the bias in random, distributive hiding, or blind during the RCT process; well-designed and implemented RCTs are considered gold standards for evaluating interventions to minimize or avoid bias [25]. Additionally, authors of most SRs/MAs declared that more high-quality RCTs with large-sample size should be carried out.

Strength and Limitations.
As the highest source of evidence, this overview will be beneficial to clinicians in making decisions in opting for methods treating the disease and help researchers to improve the quality of their study. Widely used validated tools to assess methodology (AMSTAR-2) and quality of evidence (GRADE) of included reviews were used; however, valuation of methodological quality and quality of evidence was a subjective process; the accuracy of assessor's assessments cannot be guaranteed.

Conclusion
The use of SFI for the treatment of HF may be clinically effective and safe. However, this conclusion must be interpreted cautiously due to the generally low methodological quality and low evidence quality of the included SRs/MAs. More rigorously designed SRs/MAs and RCTs with high methodological quality are necessary for further proof.