The Quality of Methodological and Reporting in Network Meta-Analysis of Acupuncture and Moxibustion: A Cross-Sectional Survey

Background Acupuncture had long been a primary treatment in the healthcare system of China. In recent years, there were more and more network meta-analyses (NMAs) in the field of acupuncture and moxibustion, but the quality evaluation of NMAs was rare. Objectives The goal of this study was to evaluate the methodological and reporting quality of NMAs and summarize the effects of different treatments of acupuncture and moxibustion. Methods PubMed, Embase, the Cochrane Library, China National Knowledge Infrastructure Database (CNKI), WanFang Database (WF), Chinese Scientific Journal Database (VIP), and Chinese Biomedical Literature Database (CBM) were searched from inception to January 2020 without any language restriction. In addition, the unpublished studies and the references of initially included literature were also retrieved manually. We included all relevant NMAs treated with acupuncture and moxibustion; other therapies such as traditional Chinese medicine and Western medicine may also be included, but at least three types fall under the category of acupuncture in each NMA. Outcome indicators were not limited. We selected AMSTAR2 and PRISMA-NMA to evaluate the methodological and reporting quality of eligible studies, respectively. Results In total, 29 NMAs were included finally, including 12 Chinese references and 17 English references. All eligible studies were published from May 2013 to August 2019. The number of interventions was between 4 and 22. The number of clinical trials included ranged from 10 to 121, with a total of 1098 clinical trials. The NMAs were involved in up to 23 diseases, knee osteoarthritis and primary dysmenorrhea covered with 3 NMAs separately, others focusing on chronic functional constipation, lumbar disc herniation, chronic fatigue syndrome, and the like. The Jadad scale and RoB scale were used as the bias risk assessment tools. Among them, 7 articles adopted the Jadad scale, 22 articles adopted the RoB scale (1 article adopted both the Jadad scale and RoB scale), and only 1 article did not mention the risk assessment tool. The AMSTAR2 methodological evaluation showed that the highest score was 13.5 points and the lowest was 4, with an average of 8.64 and a median of 9.5. According to the quality criteria, only one of them was in high quality, twenty-four were in medium quality, and four were in low quality. The PRISMA-NMA reporting quality evaluation showed that the highest score was 29 points and the lowest was 13.5, with an average of 23.62 and a median of 24.5; severe flaws also existed in some items, especially in “Structured summary,” “Protocol and registration,” “Search,” “Data collection process,” “Data items,” “Additional analyses,” “Risk of bias across studies,” and “Results of additional analyses.” Conclusion The number of NMAs in the field of acupuncture and moxibustion was still in the initial stage. Overall, their methodology and reports were of moderate quality. However, severe flaws also existed in some items. Because the eligible NMAs were limited, the conclusion needed further research to confirm its authenticity and reliability.


Introduction
Acupuncture has a history of more than 4,000 years and is now commonly used in medical care in China. As a traditional oriental therapy, it has been widely used globally [1]. With the development of evidence-based acupuncture and moxibustion, acupuncture and moxibustion is more and more popular in clinics because of its simple treatment, quick effect, and nontoxic side effects. However, there are many kinds of acupuncture and moxibustion intervention methods, and the indications are similar. Traditional metaanalysis can only achieve pairwise direct comparison of intervention measures, but it cannot carry out indirect comparison of intervention measures without a direct comparative study, and let alone comparative analysis of various intervention measures; so, it is difficult to solve this problem.
Network meta-analyses (NMAs) are still named multiple-treatments meta-analysis (MTM) or mixed treatment comparison meta-analyses (MTC). Meta-analysis is an analytical method that evaluates the relative efficiency of treatments and synthesizes evidence using a randomized controlled trial network diagram. is method is based on the combination of traditional direct/head-to-head comparison and indirect comparison for meta-analysis, which can simultaneously compare the efficacy of three or more interventions. It was developed on the basis of classical metaanalysis, which resulted in a consistent and complete set of comparisons based on all available evidence from existing research studies [2][3][4][5].
e NMA provides evidence for clinical drug selection by quantifying different interventions to treat the same disease and ranking the benefits [6]. As an emerging evidence synthesis tool, NMAs are becoming more and more popular, which can make more decisions or choices than classic meta-analysis [7][8][9][10].
In recent years, NMA has made some progress in the field of acupuncture and moxibustion, and the number of publication of NMAs related to acupuncture and moxibustion is on the rise, but its quantity is still small, and its quality still lacks systematic evaluation. is study through retrieving NMAs of acupuncture and moxibustion published a comprehensive understanding of the present situation, and apply AMSTAR2 (a measurement tool to assess the methodological quality of systematic reviews) and PRISMA-NMA (PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations) to assess the methodological and reporting quality of the published NMAs in acupuncture and moxibustion field systematically, in order to offer reference to improve the quality of acupuncture and moxibustion in NMA.

Information Sources.
PubMed, Embase, the Cochrane Library, China National Knowledge Infrastructure Database (CNKI), WanFang Database (WF), Chinese Scientific Journal Database (VIP), and Chinese Biomedical Literature Database (CBM) were searched from inception to January 2020 without any language restriction. In addition, the unpublished studies and the references of initially included literature were also retrieved manually. e comprehensive search strategy for PubMed is presented in Table 1. e retrieval of other electronic databases was similar to PubMed, which adopts the combination of subject words and keywords.

Eligible Criteria.
We included all relevant NMAs treated with acupuncture and moxibustion; other therapies such as traditional Chinese medicine and Western medicine may also be included, but at least three types fall under the category of acupuncture in each NMA. Outcome indicators were not limited, while language limited in Chinese and English.

Study Selection and Data Extraction.
e search was conducted by NoteExpress 5.3.0 literature management software. NoteExpress 5.3.0 automatic duplicate check function was used and combined with manual duplicate check to eliminate the duplicate research. By reading the title and abstract, we excluded studies that obviously do not meet the inclusion criteria; downloaded and read the full text of the remaining studies to judge; and for research with incomplete data report, tried to contact the author and gain the complete data. e three evaluators (TY, XW, and JY) screened and extracted the literatures back to back independently according to the inclusion and exclusion criteria, and cross-checked the results. e included details were as follows: author information, year of publication, sample size, disease, type of study, diagnostic criteria, number of original study, description of interventions, number of interventions, comparators, outcome measures, and risk assessment tools for bias and adverse reactions. When there was any disagreement, it was resolved by the fourth researcher (JX).

Methodological and Reporting Quality Assessment Tools.
ree independent researchers (TY, XW, and JY) evaluated the methodological and reporting quality back to back. e AMSTAR2 tool contained 16 aspects [11]. For each aspect, when the answer was "yes," the score was 1, and when the answer was "no," "cannot answer," or "not applicable," the score was 0. e total score of an NMA was calculated by counting the number of "yes" in 16 items on a scale of 0-16. A score of 12-16 was rated as "high quality," a score of 7-11 was rated as "medium quality," and a score of 0-6 was rated as "low quality." e PRISMA-NMA contained 32 items [12]. For each item, a score of "1" means full compliance, "0.5" means partial compliance, and "0" means noncompliance [13,14]. e total PRISMA-NMA score of an NMA was calculated by accumulating the scores of each item, with a range of 0-32. A score of 26-32 was rated as "high quality," a score of 20-25.5 as rated as "medium quality," and a score of 0-19.5 as rated as "low quality." When there was any disagreement, it was resolved by the fourth researcher (JX).

Data Analysis.
We analyzed the characteristics of included studies through descriptive statistical methods. All the data used were counted in the Excel 2007 spreadsheet. We described the dichotomous data in terms of number and percentage and the continuous variables in terms of median with interquartile range (IQR). And we calculated the number of papers per item, its percentage, and 95% confidence intervals. We summarized the scores according to the quality evaluation tool. AMSTAR2 and PRISMA-NMA scored 16 and 32, respectively. Finally, we calculated the total score through adding a list of each component.

Search
Results. 242 related references through searching seven electronic databases and manual searches were retrieved. After reading the title and abstract, a total of 29 studies were included . e literature screening process is shown in Figure 1.

Study Characteristics.
e characteristics of all included NMAs were documented. After screening, 29 NMAs finally met the inclusion criteria, including 12 Chinese references (including 1 master's thesis) and 17 English references. All eligible studies were published from May 2013 to August 2019.
e study contained 4-22 types of treatments and 10-121 RCTs for 1098 RCTs in total. e NMAs were involved in up to 23 diseases, knee osteoarthritis and primary dysmenorrhea covered with 3 NMAs separately, others focusing on chronic functional constipation, lumbar disc herniation, chronic fatigue syndrome, and the like. e Jadad scale and RoB scale were used as the bias risk assessment tools. Among them, 7 articles adopted the Jadad scale, 22 articles adopted the RoB scale (1 article adopted both the Jadad scale and RoB scale), and only 1 article did not mention the risk assessment tool. e characteristics of the eligible studies are presented in Table 2.

Methodological Quality Assessment.
According to the AMSTAR2 checklist, the median score and IQR of eligible NMAs was 9.5 (6.5-10.75), and the details are presented in Tables 3 and 4. e item with the best degree of compliance was "comprehensive literature search" (100%) ( Table 4). Longitudinal analysis, a good degree of compliance was with item 9 (93.1%), item 11 (79.31%), and items 1, 5, and 6 (75.86%). However, a poor degree of compliance was with item 2 (17.24%) and item 3 (6.9%), and the worst degree of compliance was with item 7 (0%) and item 10 (0%). e details are presented in Tables 3 and 4 and Figure 2.

Summary of Main Findings.
e goal of this crosssectional survey was to evaluate the methodological and reporting quality of NMAs and summarize the effects of different treatments of acupuncture and moxibustion (Table 6). A total of 29 acupuncture NMAs were included in this study. From the perspective of the number and publication time, the development of NMA in acupuncture and moxibustion was still in the initial stage, with a small number, but it had shown a trend of gradual growth. In terms of disease, more than half were chronic pain. In terms of intervention measures, more than half were acupuncture combined therapy. It was not difficult to find that NMAs of acupuncture and moxibustion was still limited to several diseases, and there were still large gaps in many aspects. Evidence-Based Complementary and Alternative Medicine e methodological quality of NMAs was important, so we evaluated the methodological quality of NMAs in acupuncture and moxibustion according to AMSTAR2 tool. e results showed some methodological deficiencies, particularly with regard to item 2, item 3, item 7, and item 10. e highest NMA score for each item was 13.5, the lowest was 2.5, and the median and IQR was 9.5 (6.5-10.75), indicating average methodological quality. e quality evaluation of the report showed that the quality of acupuncture NMAs was generally acceptable, indicating that the NMA researchers had a high level of evidence-based medical knowledge and scientific research literacy. Some items need to be improved, particularly with regard to the structured summary (item 2), protocol and registration (item 5), search (item 8), data collection process (item 10), data items (item 11), additional analyses (item 16), risk of bias across studies (Results section) (item 22), and results of additional analyses (item 23). From the perspective of a single NMA, the highest score was 29, the lowest was only 13.5, and the median and IQR was 9.5 (6.5-10.75), showing that the quality of reports included in the study was of average quality. PRISMA-NMA checklist was helpful to improve the reporting quality of acupuncture NMAs. erefore, it was necessary to improve the comprehensiveness and standardization of the report.

Strengths and Limitations.
First, this was the first study that evaluated the methodological and reporting quality of NMAs comprehensively, which complied with the methodological and reporting guidelines in the field of acupuncture and moxibustion. Even though there were two studies regarding the methodological or reporting quality of NMAs in TCM, one English article excluded acupuncture and moxibustion [44] and one Chinese article included acupuncture and moxibustion literature incompletely [45]. Second, compared with published quality studies of NMAs in acupuncture and moxibustion, this review implemented a more comprehensive and detailed literature retrieval strategy. In addition, the unpublished studies and the references initially were also retrieved manually. As a result, the results were more credible.       [15] Y Zhu et al. [28] N Li et al. [31] Y Mo et al. [32] P .5 Yeh et al. [34] P Tan et al. [36] Y .5 Zhu et al. [38] P Xiong and Chen [39] Evidence-Based Complementary and Alternative Medicine is research also had presented some limitations: first, there was no specific methodological quality assessment tool for NMAs, even though AMSTAR2 was generally used in the quality evaluation of systematic review and meta-analysis.
Second, in this study, the quality of each NMA was quantified by the assignment method, and there were some controversies on whether the weight of each item was consistent. ird, even if a comprehensive literature search strategy was used, there   Item  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15 16 Score Zhang et al. [41] Y Score  20  5  2  15  22  22  0  14  27  0  23  16  16  14  13  11 8.64/9.5 Y, yes (1 point); N, no (0 point); P, partial satisfaction (0.5 point). Item 1, did the research questions and inclusion criteria for the review include the components of PICO? Item 2, did the report of the review contain an explicit statement that the review methods were established prior to the conduct of the review and did the report justify any significant deviations from the protocol? Item 3, did the review authors explain their selection of the study designs for inclusion in the review? Item 4, did the review authors use a comprehensive literature search strategy? Item 5, did the review authors perform study selection in duplicate? Item 6, did the review authors perform data extraction in duplicate? Item 7, did the review authors provide a list of excluded studies and justify the exclusions? Item 8, did the review authors describe the included studies in adequate detail? Item 9, did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review? Item 10, did the review authors report on the sources of funding for the studies included in the review? Item 11, if meta-analysis was performed, did the review authors use appropriate methods for statistical combination of results? Item 12, if meta-analysis was performed, did the review authors assess the potential impact of RoB in individual studies on the results of the meta-analysis or other evidence synthesis? Item 13, did the review authors account for RoB in individual studies when interpreting/ discussing the results of the review? Item 14, did the review authors provide a satisfactory explanation for, and discussion of, any heterogeneity observed in the results of the review? Item 15, if they performed quantitative synthesis, did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review? Item 16, did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review?

Conclusion
e NMAs methodological and report quality related to acupuncture and moxibustion were general, and there was still room for improvement in some aspects. For example, the researchers should design the scheme in advance before carrying out the study, design and carry out the study strictly in accordance with PICOS, and present the network structure, so as to improve the prospective and reliability of the study. Considering the importance of PRISMA-NMA checklist to NMA, we advise that the researchers should strictly follow the PRISMA-NMA checklist when writing a NMA.

Abbreviations
AMSTAR2: A measurement tool to assess the methodological quality of systematic reviews PRISMA-NMA: PRISMA extension statement for reporting of systematic reviews incorporating network meta-analysis of health care interventions NMA: Network meta-analysis.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that they have no conflicts of interest.

Authors' Contributions
JX obtained funds for this study. JX and TY made contributions to the conception and design of the review. FYJ revised the search strategy. TY, JY, and XW searched, extracted, and analyzed data for studies. YT, JY, XW, and JX conducted methodological and reporting quality evaluation. FYJ, HXZ, KL, and LLX assisted. Ting Yuan wrote the original draft. All authors reviewed the manuscript, agreed to all the contents, and agreed the submission.