Quality of Reporting of Randomized Clinical Trials in Tai Chi Interventions—A Systematic Review

Objectives. To evaluate the reporting quality of published randomized clinical trials (RCTs) in the Tai Chi literature following the publication of the CONSORT guidelines in 2001. Data Sources. The OVID MEDLINE and PUBMED databases. Review Methods. To survey the general characteristics of Tai Chi RCTs in the literature, we included any report if (i) it was an original report of the trial; (ii) its design was RCT; (iii) one of the treatments being tested was Tai Chi; and (iv) it was in English. In addition, we assessed the reporting quality of RCTs that were published between 2002 and 2007, using a modified CONSORT checklist of 40 items. The adequate description of Tai Chi interventions in these trials was examined against a 10-item checklist adapted from previous reviews. Results. The search yielded 31 Tai Chi RCTs published from 2002 to 2007 and only 11 for 1992–2001. Among trials published during 2002–2007, the most adequately reported criteria were related to background, participant eligibility and interpretation of the study results. Nonetheless, the most poorly reported items were associated with randomization allocation concealment, implementation of randomization and the definitions of period of recruitment and follow-up. In addition, only 23% of RCTs provided adequate details of Tai Chi intervention used in the trials. Conclusion. The findings in this review indicated that the reporting quality of Tai Chi intervention trials is sub-optimal. Substantial improvement is required to meet the CONSORT guidelines and allow assessment of the quality of evidence. We believe that not only investigators, but also journal editors, reviewers and funding agencies need to follow the CONSORT guidelines to improve the standards of research and strengthen the evidence base for Tai Chi and for complementary and alternative medicine.


Introduction
Randomized controlled trials (RCTs) are generally considered to have the highest level of credibility in determining efficacy of a new treatment, hence, a "gold standard" for evidence-based clinical practice. Many health care professionals make treatment decisions based on reports of RCTs published in peer-reviewed journals.
The randomization process in RCTs not only can control many known and unknown confounding factors, but can also eliminate some bias so that any differences in outcome can be reasonably attributed to the effect of the treatment being tested. Unfortunately, bias can still be introduced into RCTs especially when the trials were poorly conducted. In fact, systematic errors were found in many published RCTs which results in the overestimation of the efficacy of investigational treatments [1,2]. To identify bias, to assess the validity of a study, and to inform decisions, scientific readers need and deserve to know details regarding the conduct of the trial. Therefore, it is crucial for investigators to specify the design, conduct and analysis in study reports for publication.
Understanding the importance of transparency in reporting clinical trails, an international team, including epidemiologists, statisticians and journal editors, developed the Consolidated Standards for Reporting Trials (CONSORT) statement in 1996 [3]. The statement has a checklist of 2 Evidence-Based Complementary and Alternative Medicine items that should be included in a trial report along with a flowchart showing the path of trial participants from enrollment to analysis. The goal of the CONSORT statement was to facilitate critical appraisal and interpretation of RCTs by providing investigators a framework for improvement in reporting studies. In addition, journal reviewers can use the CONSORT statement to assist them in identifying reports with potentially biased results. In 2001, the CONSORT statement, the checklist and the flowchart have all been revised in response to pubic feedback. In addition, extensive explanation and elaboration were provided with the revised statement to make it easier for authors and editors to use [4].
Since its first introduction in 1996 and revision in 2001, CONSORT has been used to assess the reporting quality of RCTs testing different interventions, including pharmacology [5], surgery [6], weight loss [7] and acupuncture [8]. Nonetheless, little is known about the quality of reporting in randomized trials of Tai Chi interventions. Therefore, the primary objective of this review was to evaluate the completeness and transparency of Tai Chi trial reports published in peer-reviewed journals. More specifically, we were aiming to (i) describe the characteristics of published Tai  Trial. In addition, we latter performed a similar search in PUBMED as suggested by the journal editor. To survey the characteristics of all published trials in the Tai Chi literature, we decided that a trial report would be included if it meets the following criteria: (i) the article was an original report of the trial; (ii) the study design was RCT. For this study, a RCT was defined as an experimental study in which study participants were randomly allocated to at least two groups to test the efficacy/effectiveness of interventions [9]; (iii) one of the treatments being tested was Tai Chi; and (iv) the publication was in English. Three authors, J. L., Y. Z. and J. F., then independently reviewed the abstracts and full articles (when necessary) to assess whether randomized trial reports identified in the literature search met our inclusion criteria. Discrepancies in this assessment were resolved through discussion and consensus. Lastly, among all the included Tai Chi RCTs, we identified those published between January 2002 and December 2007 to examine the quality of reporting of these trials.

Data Extraction.
Since the CONSORT statement was not originally designed to be used as a quality assessment instrument, we made the following modification based on the 2001 version of the CONSORT statement.
First, some original items in CONSORT were rewritten so that multiple items were subdivided and listed separately to ensure only one response per item. For instance, item 1 in the original CONSORT statement relating to title and abstract: "How participants were allocated to intervention" was rewritten into two items as the following: (1-1) Does the title identify the report as a RCT? (1-2) Does the abstract have a structured format? This resulted in a checklist including 40 separate items (Table 1). Each item could be assigned a "yes" or "no" to indicate whether the component had been reported as recommended.
Second, item 4 in the original CONSORT statement is described as: "precise details of the interventions intended for each group and how and when they were actually administered". This item is poorly defined and not readily applied to reporting Tai Chi interventions in a trial. In this review, we used a mini-checklist of 10 items to assess the adequate reporting of testing treatments in a Tai Chi study. If more than seven items in this mini checklist were reported by a trial, item 4 in the original CONSORT statement was considered satisfied. This mini-checklist was adapted from previous reviews [10,11] and is shown in Table 2. We used this mini-checklist because there is no consensus in the literature in specifying Tai Chi interventions in a RCT. In addition, we believe that this checklist follows the guidelines for reporting non-pharmacologic treatments recommended by the CONSORT group [12].
One author J. L. reviewed full articles of included trials published between 2002 and 2007 to determine whether each of the 40 items in the modified checklist was reported. In addition, a second reviewer, Y. Z., independently checked a set of 10 articles (33% of all studies) selected using computergenerated "random" numbers. Before the evaluation process, J. L. and other authors studied and discussed the definition of each item in the revised CONSORT statement using the published guideline [4]. Furthermore, J. L. played a leading role in rewriting the checklist used in this review. Using the checklist, the authors evaluated five Tai Chi trials that were published before 2002 together, and then completed an independent evaluation of another three ineligible Tai Chi trials due to their publication date. The results were compared and discrepancies were resolved through discussion and consensus. The above process was to ensure that (i) the new checklist was workable and (ii) the single reviewer shared the same interpretation of the checklist with the other authors.

Statistics Analysis.
We determined the proportion of RCTs reporting each of the 40 items in the modified CONSORT checklist and each of the 10 items in the minichecklist. Only descriptive statistics were used. We did not calculate a total quality of reporting score (summing the dichotomized scores of the 40-item checklist) for each trial in this review, as the individual items on the checklist were not felt to share equal weight.
The second rater, Y. Z., reviewed 10 articles and decided whether each of the 40 items in the modified checklist Do authors report any other analyses performed including subgroup analyses and adjusted analyses, indicating those pre-specified and those exploratory? (e.g., adjusted P, post hoc or a posteriori) 4 Evidence-Based Complementary and Alternative Medicine  was reported. After both raters finished their review, we calculated Cohen's κ-statistics to assess agreement between two reviewers. Good inter-reviewer agreement (κ = 0.8-1) was found for the majority (95%) of the CONSORT items. Fair agreement (κ = 0.6-0.8) was only found for two items (Item 3-2, which states: "Do authors describe settings where the data were collected?" and item 18, which states: "Do authors report who assigned participants to their groups?"). The disagreements were resolved by consensus between two raters.

& Alternative Medicine, Age and Ageing, Archives of Physical
Medicine & Rehabilitation, and so forth.

Clinical Application of Tai Chi.
We found that in the last two decades Tai Chi has been extensively tested for improving balance and preventing falls among older people. The main study outcome in 16 out of a total of 42 RCTs was associated with balance control, muscle strength and/or number of falls. Nonetheless, researchers were also looking into potential application of Tai Chi in other clinical areas, ranging from anti-hypertension to promotion of general fitness. A list of clinical applications that Tai Chi has been tested for in previous trials is presented in Table 3.

The Reporting Quality of RCTs of Tai Chi Published between 2002 and 2007.
A summary of the proportion of RCTs reporting each item in the modified CONSORT checklist is presented in Table 1. In general, the most adequately reported criteria were related to background (item 2), participant eligibility (item 3-1) and interpretation of the study results (item 20-1). In addition, the majority of the reports (90-95%) had well-defined study objectives (item 5) and outcome variables (item 6-1), provided adequate description of statistical methods (item 12), baseline characteristics of study participants (item 15) and a summary of results for study groups (item 17-1). On the other hand, the most poorly reported items were associated with allocation concealment and implementation of randomization (Item 9-1, 9-2, 10-1, 10-2, 10-3). In addition, less than one-fourth (23%) of the trials clearly defined the period of recruitment (item 14-1). Only three trials reported the period of followup of their studies (item 14-2). Finally, 23% of included trial reports were considered to provide adequate details of Tai Chi interventions in their studies, because they reported at least 7 out of 10 items in the mini checklist. The extent to which the mini-checklist criteria were met is presented in Tables 2 and 4. More detailed tables containing full references are available from the authors.

Discussion
In order to present their trials in an open and transparent manner, Tai Chi investigators need to follow the widely accepted CONSORT guidelines. It is important to address this issue for many reasons. First, poorly reported clinical trials make it difficult for other researchers to assess the validity of the results, to replicate the study, and to identify gaps that need to be addressed in the design and reporting of future Tai Chi trials [4]. In addition, inadequate reporting may mislead health-care providers in their treatment decisions  Unfortunately, despite an increasing number of RCTs assessing Tai Chi in the past two decades, the reporting quality of these trials is sub-optimal and substantial improvement is required to meet the CONSORT guidelines. Almost 50% of the trials we reviewed did not satisfy more than half of the criteria in the modified CONSORT checklist and only 23% of RCTs provided adequate details of Tai Chi intervention used in the trials. Moreover, many of the poorly reported criteria have been associated with biased and erroneous interpretations of study findings. Each of these potentially problematic areas, including randomization, intention-totreat analysis, masking, sample size and interventions is discussed below to help scientific readers recognize them when reviewing studies of Tai Chi.

Randomization.
The results from a RCT are considered the most reliable form of scientific evidence because it involves the random allocation of different treatments to study participants. A well-conducted randomization procedure could not only prevent selection bias, but also control known and unknown confounders, thus ensuring the validity of the study results. Unfortunately, human interference could ruin a perfect randomization method and lead to biased and invalid findings. Therefore, the original CON-SORT statement suggests that investigators should provide information regarding who performed randomization and whether allocation sequence was hidden from them, and also how the sequence was generated. This is to help readers assess whether the random assignments were unpredictable, and more importantly, to identify potential bias introduced by the study team. Unfortunately, we found less than half of the trial reports we reviewed provide a description of sequence generation. Reporting of the allocation concealment and randomization implementation was even worse. In more than two-third (70%) of the trials, it was not clear whether the person who allocated interventions knew the sequence or not. This leads a reader to question whether the randomization process was free from human alteration and, in turn, whether the findings were valid.

Intention-to-Treat Analysis.
The goal of intention-totreat analysis is to preserve the benefits of randomization by comparing patients in the treatment groups to which they were originally assigned, not on the treatment eventually administered. The randomization process in a RCT balances potential confounders. Nevertheless, if a confounder is associated with study participation and continuation, then omitting those who withdraw from a trial will reintroduce imbalance on this confounding factor. Thus, the benefit of randomization is compromised if intention-to-treat is not Evidence-Based Complementary and Alternative Medicine 7 the primary analysis method. However, >60% of the Tai Chi RCTs did not mention the use of intention-to-treat analysis. Among them, 10 trials did not even document the numbers of participants in each group included in each analysis, making it very difficult to judge the validity of their findings.

4.3.
Masking. The use of masking (blinding investigators/ participants to treatment status) is aiming to reduce reporting and measurement bias [57]. In reality, it is not feasible to blind the study participants from the interventions assigned to them in a Tai Chi clinical trial. The investigators, however, could and should make every effort to mask the staff who measure outcomes from the group assignment to eliminate measurement bias. Nonetheless, 17 Tai Chi trials (55%) included in this review did not report any blinding effort. Therefore, it is reasonable to suspect that the results of these studies may be biased intentionally or unintentionally. In fact, it has been shown that, on average, RCTs that have not used appropriate levels of blinding show larger treatment effects than blinded trials [1].

Sample
Size. Theoretically, investigators should have a large enough number of subjects in the trials to have a high power of detecting clinical significant difference between interventions, if there is a difference. Ethically, investigators have to have adequate sample size to justify enrolling participants. Unfortunately, more than half (18 studies) of the trials reviewed in the study did not indicate how the sample size was determined. When scientific readers review the results of these trials, they are not able to determine whether the non-significant findings are because Tai Chi was not effective or because the study was underpowered. In addition, even among 13 trials that justified their sample size, five of them did not include attrition in their power calculation. Therefore, these trials were doomed to be underpowered, since the loss of study participants is ubiquitous in RCTs.

4.5.
Interventions. Ideally, the description of interventions in a clinical trial should provide enough details for other researchers to compare and replicate the treatments. In addition, adequate information regarding the interventions in a trial report could help clinicians to introduce the most effective and efficient treatments into practice [12]. Therefore, it is important for investigators to describe their interventions as clearly as possible. However, when one of the treatments used in a trial is Tai Chi, the investigators have to pay extra attention, mainly because Tai Chi has been recognized as a complex exercise with multiple components. Many of these components are hypothesized to be therapeutic, yet remain poorly understood [11,58]. Therefore, the Tai Chi interventions emphasizing different components may have different impact on the estimate of the treatment effect [10]. In this review, we used a 10-item mini-checklist to assess the adequate reporting of Tai Chi interventions in the existing clinical trials. We found no trials that met all the criteria; the majority of the trials (35%) reported only 6 out of 10 items in the checklist. Therefore, many of the Tai Chi RCTs could not provide comprehensive details for the interventions being tested. This finding underscores the necessity of developing a guideline for future investigators to report the Tai Chi protocol used in clinical trials.
Our study also revealed other potentially problematic issues when reporting Tai Chi clinical trials. A participant flow chart is the most effective way to help readers to track the numbers of participants in different phases of a complex RCT. Only 48% of the RCTs we reviewed had a diagram showing participant flow. Since the flow charts play an essential role in understanding why some participants did not receive the intervention as assigned, withdrew from the study, or were not counted in the final analysis, the inclusion of the flow charts in reports about Tai Chi interventions is indispensible. In addition, reporting the periods of recruitment and follow-up, including starting and ending dates, was 23% and 10%, respectively in the Tai Chi clinical trials. Providing this information is useful for other researchers to know the rate at which participants were enrolled in Tai Chi studies, and whether a trial was stopped earlier than expected [4]. Lastly, adverse events have not been well documented in the previous Tai Chi trials; only 39% of the reports provided explicit information on adverse events. It is common that, during a clinical trial, study participants may report unintended or undesirable effects. Although these effects may or may not associate with the interventions, readers still deserve to know this information to weigh the risks and benefits of Tai Chi.

Limitations.
We acknowledge some potential limitations of this review. First, we did not evaluate the reporting quality of Tai Chi clinical trials published before 2002. Previous studies [4,57] have shown that the reporting quality of clinical trials for any disease treatment before the 1996 CONSORT statement was extremely poor and that the publication of CONSORT guidelines was associated with improved reporting quality in studies published in journals both using and not using the CONSORT statement [4,59]. In fact, we found that the average number of items reported in the modified CONSORT checklist is 16.7, 21.8 and 22.0 for trial reports published during 1992-1996, 1997-2001 and 2002-2007, respectively. The poor quality of reporting for Tai Chi trials published between 1992 and 1996 may due to the fact that the CONSORT guidelines were not available to the authors of these trial reports. Therefore, we decided it was only informative to review the Tai Chi trials published after 2002, one year after the publication of the 2001 CONSORT guidelines, since the purpose of our study was to determine how well the Tai Chi RCTs met the new guidelines. Prior to 2002, researchers did not have the strict guidelines for guidance on how to describe their procedures. Second, we only included Tai Chi trial reports found in PUBMED and MEDLINE to ensure that all articles included have undergone similar peer and editorial review ensuring quality control. However, it is important to recognize that Tai Chi studies reported in foreign languages make up a part of Tai Chi literature with an unknown significance. Including foreign language publications may be necessary when the goal of a review is to estimate the effectiveness of Tai Chi interventions. Nevertheless, the primary objective of this 8 Evidence-Based Complementary and Alternative Medicine article was to evaluate the completeness and transparency of Tai Chi trial reports in the literature and not to evaluate the effectiveness/efficacy of Tai Chi interventions.
Tai Chi trial reports are certainly increasing over the years and play an essential role in determining the efficacy and effectiveness of Tai Chi interventions. While we recognize that the RCTs may not constitute the largest proportion of the Tai Chi literature, they are the "gold standard" by which treatment effectiveness is evaluated. Therefore, we decided to focus this review on reports of randomized trials. Lastly, our two reviewers were not blinded to authorship, journal title and other study-related information because it has been shown that masking has little impact on the conclusions of a systematic review [60,61]. In addition, it was almost impossible to be totally blind, as this would have meant reformatting all 31 articles included to mask any mention of the author's work.

Conclusion
As the editors of the Journal o the American Medical Association, Drs. Fontanarosa and Lundberg pointed out in their editorial on alternative medicine in 1998 [62], properly designed and conducted randomized controlled trials are needed to provide solid scientific evidence regarding the safety, efficacy and effectiveness of complimentary and alternative medicine (CAM) interventions. As the number of clinical trials testing CAM interventions is increasing, attention needs to be given to the quality of reporting of clinical trials. Without complete and adequate reporting, it is impossible to assess the methodological rigor of the study, and in turn, the scientific evidence of the CAM intervention being tested. We believe that following the CONSORT guidelines will improve the quality of reporting of a RCT. The essential component of CONSORT is transparency of reporting the design, execution and findings of a study, a principle that should apply equally to other types of research. We also believe that not only investigators but also journal editors, reviewers and funding agencies need to follow the CONSORT clinical trial reporting guidelines to improve the standards of research and the strength of the evidence base for Tai Chi and other research.