Lessons to be learned for gastroenterology from recent issues in clinical trial methodology

Can J Gastroenterol Vol 14 No 4 April 2000 293 This mini-review was prepared from a presentation given at the 26th Pan American Congress of Digestive Diseases, 13th Pan American Congress of Digestive Endoscopy, August 30 to September 2, 1999, Vancouver, British Columbia Research and Telemedicine Committee of the World Organisation of Gastroenterology and Coordination Centre for Clinical Trials and Theoretical Surgery Unit, Department of General and Trauma surgery, Heinrich-Heine-Universtity, Düsseldorf, Germany Correspondence and reprints: Prof Dr C Ohmann, Koordinierungszentrum für Klinische Studien, Moorenstr 5, D-40225 Düsseldorf, Germany. Telephone +49-211-8119700, fax +49-211-811-9702, e-mail ohmannch@uni-duesseldorf.de Received for publication August 31, 1999. Accepted September 1999 MINI-REVIEW

T he present article discusses actual and conflicting issues of clinical trial methodology and realization.General controversial and unsolved problems are exemplified and discussed, and some recommendations are provided that may help to improve the quality of randomized trials in gastroenterology and increase their number.Randomized trials continue to be the most important, but not the only, tool in clinical research.

ONGOING NEED FOR RANDOMIZED TRIALS
In the past decade, the full range of medicine has come under increased scrutiny.Consequently, the efficiency and effectiveness of medical treatment have been frequently questioned, and the need for a solid evidence-based foundation of medical practice is widely recognized.Evidence-based medicine is thereby understood to be the continuous, explicit and judicious use of the current best evidence in making decisions about the care of individual patients (1).Evidencebased medicine integrates individual clinical expertise with the best available external clinical evidence from systematic research.The best available external clinical evidence is clinically relevant research that may come from the basic sciences -especially from patient-centred clinical research (1).The randomized, controlled clinical trial is the most rigorous way to determine whether a cause-effect relationship exists between treatment and outcome (2) and, therefore, provides the strongest clinical evidence.In contrast to other study designs, only adequate randomization achieves comparability among study groups with respect to known and unknown prognostic factors.As has been increasingly recognized, the advantages of new treatments are often only moderate or small (3).The reliable detection or refutation of these moderate differences requires precise trials with negligible bias and small, random error, increasing the need for well conducted, methodologically sound, randomized, controlled clinical trials.
Medical research can be subdivided into three areas: basic clinical research, disease-oriented research and patientoriented research.Randomized trials are the preferred tool for patient-oriented research, and their main role is to enable the transfer of basic research (Figure 1).This frequently leads to new conceptual models of disease, which in turn stimulate sciences (4).In most countries, randomized trials are legally required in order to demonstrate the safety, effectiveness and efficacy of new treatments.Advances in drug design facilitate the creation of new substances and, therefore, considerably increase the need for trials.A telling example is the considerable interest in new treatment for sepsis over the past decade.Various new and promising drugs have been tested that were deemed to be advantageous because of results obtained under laboratory conditions.However, the results of nearly all controlled clinical trials performed have been unable to demonstrate a treatment benefit, and some have even suggested a harmful effect on the outcome (5).While the randomized trial is a reliable guard to keep ineffective therapy out of clinical practice, multiple clinical trials have demonstrated superior treatment due to new therapies.
Exemplary are the trials showing streptokinase treatment for acute myocardial infarction (6), preventive use of medium doses of acetylsalicylic acid in patients at high risk of occlusive vascular disease and hormonal adjuvant treatments for early breast cancer (stage I/II) (3,4).
While the role of clinical trials for the evaluation of clinical treatment is well accepted, many patients see trials as advantageous only for future patients.However, centralized referral or entry to trials is frequently associated with a higher survival rate for all patients participating (7).There is no indication that patients included in trials ever do worse than comparable patients outside of trials.Therefore, it can be suggested that patients in randomized trials generally do well and are likely to receive better care than those treated outside of trials.
Study coverage in many medical areas is appalling.Gastroesophageal reflux disease is one example.Surgical treatment has not been evaluated adequately through randomized trials, adequate comparisons between surgery and medical treatment are lacking, and no trials have evaluated medical or surgical treatments in Barrett's esophagus (8).The need for more clinical trials is undisputed by doctors and recognized by patients.

PROBLEMS IN CARRYING OUT A RANDOMIZED, CONTROLLED CLINICAL TRIAL
While the need for randomized trials is evident, conducting these trials is becoming increasingly complex.Various rules and regulations (eg, International Conference on Harmonization, Good Clinical Practice) guide the conduction of clinical trials, and additional country-specific legal requirements add more details to observe and pitfalls to look out for (9).In addition to these manageable but tedious hazards, the funding is quite often limiting.Studies not related to the licensing of drugs are notoriously difficult to finance.In general, the more that the research applies to patients, the more difficult it is to obtain external funding.This has something to do with the outdated notion that epidemiologically or methodologically oriented studies are intellectually less demanding and, therefore, do not count as true research (10).As a consequence, only 27% of all research funds of the National Institutes of Health in 1996 were partly or completely earmarked for clinical research (11).Despite advances in the past decades and the general move toward evidence-based medicine, there is still a generally negative attitude on the part of patients, doctors and the public toward randomized trials that significantly hinders their conduction.Major arguments by doctors concerning randomized trials are the perceived distortion of the doctor-patient relationship, limitations in clinical decision making through protocols, personal treatment preferences and additional workload.To patients, the process of randomization is not part of medical care, and blinding is disturbing because the important reassurance of treatment is compromised.If there is a treatment preference, participation becomes impossible.In addition, the trial medication can be perceived as experimental, untested and, therefore, dangerous.Patients frequently lack the methodological basis to understand scientific studies, and it is subjectively comprehensible that some feel like guinea pigs.A similar attitude can be observed with regard to the general public, which still has a distorted understanding of the principles of randomized trials and emotionally disapproves of its principles.At the same time, treatment has to be state of the art.Trial conduction is further complicated by the fact that medicine is becoming increasingly more complex.In gastrointestinal cancer, several clinical disciplines, such as internal medicine, surgery, radiology and radiotherapy, are involved, creating intricate logistic and standardization puzzles.
In spite of these obvious obstacles to randomized trials, however, dealing with rules and regulations has been improved through increased knowledge and procedural experience, and the complexity of medical problems is being tackled through organizational changes and increased training of the staff who conduct the trials.The funding problem has been recognized by the relevant authorities, and while money is still inadequate, there are signs of change (12).Doctors, patients and the public are still far from embracing clinical research but are closer because of increased coverage by the media, which have rediscovered medicine and science.
Specific problems in conducting randomized trials are discussed below.The discussion is divided into the following sections: trial design, trial management, informed consent, trial analysis, trial reporting and publication, and the application of study results.Trial design: There are multiple recommendations and standards for the design of randomized trials (13,14).Despite that the library of good methodological advice is huge and that medical journals do their best to inform researchers, the development of research protocols causes frequent methodological problems.Too often, previous research is neglected when a trial is set up, although cumulative synthesis of trial results is scientifically and ethically paramount and achievable.In the early 1980s, a meta-analysis of trials on prophylactic antibiotics in colon surgery showed convincingly that strong evidence of the effectiveness of antibiotic treatment in reducing morbidity and mortality had already existed by the mid-1970s.By neglecting these findings, researchers continued to invite patients to participate in such trials five years later, and reports of trials involving comparison groups receiving no treatment continued to appear throughout the 1980s (15).In retrospect, all of these trials seem to be useless and ethically questionable.When developing a study protocol, it is, therefore, necessary to analyse previous research systematically.This intricate process can be effectively supported by meta-analyses, databases of clinical trials (eg, MEDLINE, Cochrane collaboration) and institutions dedicated to the development of systematic reviews (eg, Cochrane collaboration).
Another frequent problem is inadequate sample size.An extensive review of trials in schizophrenia demonstrated that more than 50% of the schizophrenia trials had 50 or fewer participants.Only 3% of the studies comprised 300 or more patients -the sample size needed to show a 20% difference among groups (16).Sample size calculation is not always easy, and the results may be disputed, but there are some basic rules.Proper sample size calculation requires, among other things, specification of the smallest clinically relevant difference useful to be detected by a trial.The frequency of treatment success in the control group needs to be assessed.The chosen P value and the power of the trial need to be defined (14).If the necessary sample size cannot be achieved, it is ethically and statistically inappropriate to start the trial.
The prospective definition of a hypothesis may be difficult but is absolutely necessary.Once it has been defined, the relevant outcome criteria for the planned intervention and the definition of study population can be tackled.These decisions have a decisive impact on the external validity and acceptance of the trial results.A new discipline called 'outcome analysis' or 'outcome research' concentrates on patient-centred outcome criteria.Traditional outcome criteria are mainly mechanistic and physician focused.They include mortality, complication rate, reoperation rate and the length of hospitalization, among others.Patient-centred criteria focus on quality of life, well being, convalescence period and other criteria (17).Particular care needs to be taken to define the randomization procedure and mechanisms of blinding.All statistical methods must be described prospectively, and outcome measurement must be standardized.Trial management: The scientific validity of a randomized trial is frequently subjected to intense scrutiny, while the practical trial management generally receives little attention (18).However, robust systems and procedures are needed for the day to day operation of clinical trials.They have to be efficient, effective and flexible to ensure the required data quality.Trial teams and/or research institutes with specially trained staff should support the clinical staff to decrease the workload for collaborating groups.
A significant trial management problem is subversion of random allocation.Several trials have shown that an inadequate randomization procedure can severely bias study results.A recent surgical trial showed that randomization using sealed envelopes resulted in significant differences among the study groups with respect to prognostic factors.A centralized telephone system that was consequently initiated yielded comparable study groups (19).Another review compared trials that did not adequately conceal randomization; the effectiveness of the active treatment was higher in trials in which randomization was poorly blinded (20).We do not think, therefore, that the common technique of the sealed envelope system can be recommended.Instead, a distant central randomization service should be used.Distant randomization is much less likely to be compromised and can now be efficiently supported by study support systems via the Internet.Table 1 summarizes the recommendations for the management of clinical trials.Informed consent: One of the major challenges in randomized controlled trials is obtaining informed consent, due to differing perspectives and languages of the physician and patient.As discussed above, physicians are often reluctant to participate in trials due to various reasons related to professional independence and doctor-patient relationship.Others, however, are sometimes too easily convinced to enter patients in trials in spite of well founded treatment preferences or objections.Generally, the major hurdle to obtaining informed consent is communication and getting the correct information (21,22).Recent studies have shown that patients are often not able to understand the explanations of the doctors, and they are not familiar with the concept of randomization because it is so different from day to day clinical care.At the same time, it can be cruel to provide complete information to the patient, considering the stress and the life-threatening situations that they are in (23).Further intricacies arise from the observation that there is a negative association between the amount of information given and the willingness of patients to participate.In a randomized comparison of procedures for obtaining informed consent, the knowledge of treatment side effects and research aspects was statistically significantly improved if total disclosure was used.However, total disclosure led to a significant decrease in the willingness to participate compared with an individual approach (24).To choose the ideal procedure to achieve informed consent, one must consider that self interest is an important motive for participation (25).Most patients are not primarily interested in participating in a study to help future patients but to benefit themselves.
Recommendations have been developed to improve the content and presentation of information to patients.It is es-sential that consent documents are clearly written using short sentences and that attention is paid to layout.The typeface and design must be clear and legible (26,27).Some of the recommendations for obtaining informed consent are summarized in Table 2.
To increase their trust in clinical research, patients need to be informed about the trial results that have been achieved with their collaboration.Further information on trial results leads to understanding of trial methodology and application (28).Documentation of patients who refuse to participate in trials stimulates and facilitates further research in patient recruitment and informed consent.Details on patients not entered point to problem areas in the trial design and illuminate the question of external validity (29,30).If the general trial culture is to be improved, it is necessary to inform the public and patients about trials in general.An interesting initiative is Centre Watch -a clinical trial-listing service for patients interested in participating in clinical trials.Trial analysis: Several standard recommendations for trial analysis (9) have facilitated the conduction of high level clinical research.Despite this statistical guidance, several critical issues persist.The use of confidence intervals is under-represented, although this deficiency has been addressed in several articles (31)(32)(33).Another notorious omission is the presentation of baseline data.An analysis of 80 randomized trials showed that these baseline data were insufficient for 39 trials (49%) (34).In a controlled trial, the process of randomization results in the allocation of patients to treatment purely by chance.This ensures that known and unknown characteristics of patients that may influence outcome are similarly distributed among treatment groups.If this were not the case, the ensuing imbalance could influence outcome and the treatment effect would be uncertain.Baseline imbalance in randomized trials is sometimes referred to as chance bias (35).To allow for this bias and to report clinical trials adequately, known prognostic variables must be described for each treatment group.This is helpful in evaluating the ability to generalize the trial results and may assure the reader that the randomization has been properly conducted.Statistical significance tests are the minimal requirements to detect possible subversion of the allocation procedure (35)   characteristics does not provide an appropriate criterion to assess the effect of any observed imbalance on outcome.Several statistical techniques are available (eg, multivariate analysis) to cope with baseline imbalance.Another problem is related to unduly data-dependent emphasis on particular subgroup results, which can induce large and important bias (3).Repeated significance testing and data-dependent analysis easily induce false positive results.It is, therefore, essential to emphasize the overall result of a trial and not rely on spectacular but biased subgroup analysis.Effects in particular subgroups should not be estimated directly.Rather, subgroup analysis should extrapolate the overall findings of a trial indirectly and semiquantitatively (3).In any case, the analysis of subgroups can generate hypotheses but not test them.
A controversial but relevant issue is intention-to-treat analysis.The primary analysis of a randomized clinical trial should compare patients in their randomly assigned treatment groups (intention-to-treat analysis) (36).However, there is no consensus on this issue.Analysis by intentionto-treat yields statistically valid probabilities, is the most practical analysis and is the only unbiased assessment of clinical trial results.Relying on treatment actually received, however, generally dilutes the measurable treatment effect.In practice, severe differences can occur between intentionto-treat and treatment-received analyses.In the Veterans Administration Coronary Bypass Surgery Study, the intention-to-treat analysis yielded no statistically significant differences between medical and surgical treatment, whereas the analysis according to treatment actually received demonstrated a highly significant difference (37).It has been suggested that the major hazard of intention-to-treat analyses is a lack of common sense.It is arguable whether it is suitable to analyze people as having had surgical treatment when they were assigned to be operated but when the operation was never performed (38).In spite of these contentious cases, intention-to-treat analysis is generally the method of choice.Trial reporting and publication: Despite efforts to improve reporting of trials, some areas of medicine continue to have a poor quality of published results (34,39).Reporting of surgical clinical trials has improved, but general methodological questions poorly answered in the 1980s continued to be poorly answered in the 1990s (39).Editors of surgical journals are, therefore, urged to provide authors with guidelines on how to report clinical trial analysis.In contrast the Consolidation of Standards for Reporting Trials (CONSORT) statement defined an agreed standard of reporting clinical trials, which has been approved by the major medical journals (40).The CONSORT statement incorporates guidelines for presenting a title, abstract, introduction, methods (protocol, assignment, masking), results, participant-flow, follow-up analysis and comments.Compliance to these guidelines ensures adequate reporting of clinical trials.The CONSORT statement, however, cannot address underreporting of clinical trials, which is the major scientific misconduct.It is well known that it is easier to publish statisti-cally significant trials than to publish trials that yield nonsignificant results (41), but under-reporting can bias meta-analysis and lead to publication bias.As a consequence, the results of many trials are lost and the medical profession is misled.Therefore, it is an ethical as much as a scientific duty to publish all clinical trials.To promote publication of the large number of unpublished trials, an 'amnesty for unpublished trials' has been announced by 50 to 100 editors of medical journals, and a call for registering unpublished trials has been initiated (42) and renewed (43).In order to facilitate trial publication, a general register for ongoing clinical trials needs to be set up.All clinical trials performed could be tracked, and personal responsibility for trial publication should be established.At the moment, several initiatives are trying to cope with ongoing clinical trials.Current Controlled Clinical Trials, for example, is a major new database that aims to provide online access to clinical trials.It is as comprehensive as possible and lists ongoing and completed clinical trials in all areas of health care (http://www.controlled-trials.com).The Cochrane Centre in the United Kingdom is even integrating the previously secret projects of pharmaceutical companies in their register (43).Recommendations for reporting and publication are summarized in Table 3.

THE ROLE OF RANDOMIZED TRIALS
IN CLINICAL RESEARCH Despite the various avoidable problems outlined in the last section, some fundamental issues in randomized trials need discussion.Randomized studies may be unnecessary, inappropriate, impossible or inadequate.Dramatic effects of an intervention may make randomization and, indeed, clinical trials superfluous; an example is the use of penicillin in bacterial infections.Trials are inappropriate if the adequate sample size to test the hypothesis adequately cannot be achieved.Examples of trials that are easily inappropriate include studies investigating rare side effects or preventing rare events.A further obstacle for randomized controlled trials is the dependency of the trial result on the subject's active participation, ie, if the clinical outcome depends on the subject's belief and preferences (44).If the trial population studied is restricted, the application of trial designs to future patients is severely limited.While the internal validity of randomized trials is normally achieved, the external validity, that is, the possibility to extrapolate to future patients outside the trial, is often compromised (38).In situations in which the use of randomized trials is limited due to inherent obstacles or because of practical hurdles, alternative designs need to be considered.Prospective observational studies may

Recent issues in clinical trial methodology TABLE 3 Recommendations for reports and publication of clinical trials
Registration of ongoing trials Application of reporting standards (eg, CONSORT) Publication of trials independent of the result CONSORT Consolidation of Standards for Reporting Trials be the appropriate means to continue under these conditions.It must be realized, however, that the degree of evidence severely decreases from randomized, controlled clinical trials to retrospective, uncontrolled clinical studies.In spite of the higher methodological elegance of the randomized, controlled clinical trial, a complementary approach to clinical research is needed.Randomized trials are the preferred standard with the highest evidence, but they need to be supplemented by other study types in order to assess the effectiveness of the full range of treatments in clinical practice.This strategy may help to extrapolate randomized trial results to a larger population and to provide knowledge in cases when randomized trials are not feasible.

CONCLUSIONS
There is a need for more randomized trials, which need to be combined with other types of research for the greatest medical benefit.While the complexity of trial conduction has increased, so have the means of methodological and practical support.Thus, problems occurring in the conduction of clinical trials can be professionally tackled and thereby result in good clinical research.

Figure 1 )
Figure 1) Transfer of knowledge between basic research and routine application