A critical approach to clinical practice guidelines

Clinical practice guidelines (CPG) are systematically developed statements that help health care practitioners and their patients to make appropriate health care decisions in specific clinical circumstances. The development and dissemination of CPG for managing common medical illnesses continue to grow as an industry among health care payers, providers, interest groups and professional organizations. If developed and promoted appropriately, and if acceptable to practitioners, CPG have the potential to improve patient outcomes and enhance the efficiency with which health care is delivered. However, publications labelled ‘CPG’ vary widely in their format, methodological rigour and generalizability to specific practice settings (1). Therefore, as the primary consumers of these documents, health care providers must learn to appraise their quality and interpret their recommendations critically. The rate at which new CPG are published has accelerated significantly over the past decade (Figure 1). In part, this growth reflects the increasing popularity of ‘evidence-based medicine’, and the need to synthesize and disseminate evidence of the effectiveness (and/or harm) of health care interventions. In addition, the development of practice guidelines parallels a more intense scrutiny of the efficiency and economic consequences of health care delivery. In a climate of constrained resources, this movement has identified significant and unexplained variation in practice and a per-


Approche critique envers les lignes directrices de pratique clinique
RÉSUMÉ : Le rythme de publication des lignes directrices de pratique clinique pour la prise en charge d'affections médicales courantes ne cesse de s'accélérer.La diffusion et l'usage appropriés de lignes directrices de pratique de haute qualité peut permettre de synthétiser les données, et d'améliorer l'évolution clinique des patients ainsi que l'efficacité des soins.Cependant, la rigueur méthodologique et la pertinence du nombre croissant de publications libellées « lignes directrices de pratique clinique » varient grandement.Les payeurs et les fournisseurs de soins de santé, ainsi que les conseillers dans ce domaine, doivent apprendre à évaluer de façon critique et à interpréter les recommandations proposées dans ces lignes directrices.Une approche simple et pratique en neuf questions permettant d'évaluer la qualité, la pertinence et l'efficacité des lignes directrices de pratique clinique est présentée.sistence of specific practice patterns that contradict the published evidence.Although many practitioners may fear that guidelines reduce physician autonomy and advocate 'cookbook medicine' (2), CPG might be better characterized as tools to promote optimal clinical practice (3).Irrespective of one's perspective on this issue, the increasing prominence of CPG in the medical landscape means they cannot be dismissed or ignored easily.
The intended goals of CPG can be summarized as follows: to enhance patient outcomes; to reduce unnecessary variations in practice patterns; to improve the cost effectiveness of health care delivery; to synthesize large volumes of scientific information; and to codify optimal practice as an educational tool.Their aim is not and should not be to define practice standards or to impose specific management algorithms.CPG must be flexible and, in particular, must acknowledge the unique nature of each patient and practice setting.Although many users fear that CPG may have legal repercussions (2), current legal opinion offers reassurance that the standard of care for litigation remains that which is acceptable to the broader medical community, rather than that codified by CPG recommendations (4,5).The implementation of CPG in some disciplines appears to have resulted in a reduced number of malpractice claims (6).
To improve health outcomes, the production and dissemination of CPG must pass through five critical stages (7).First, a specific clinical area or question must be identified.Second, the CPG developers must collect, distill and synthesize an accurate and representative summary of current knowledge in that area.Third, CPG promoters must disseminate their recommendations to health care providers and consumers, and foster a positive and receptive attitude toward new management approaches.Fourth, they must convince practitioners who are exposed to the CPG to alter their own practice patterns.Finally, and only if the preceding steps are satisfied, implementation of CPG recommendations may translate to the intended outcomes of improving health and/or the efficiency of health care delivery.This process must be iterative, and the CPG recommendations must incorporate new evidence as it arises.Failure at any of these steps impairs a guideline's overall success.
Because of heterogeneity among publications that are described or indexed as CPG, many organizations have established formal criteria for their development, reporting and critical appraisal.Although the methodological rigour of CPG historically has been low, it appears to be improving over time (1).Standards for CPG production and reporting have been published by the American Gastroenterological Association (8), the American Medical Association (9), the Canadian Medical Association (10) and others.Generally, most of these groups have identified similar concerns, which can be distilled to nine key questions for CPG consumers and producers (Table 1).Each of these is addressed in detail below.However, as a general caveat, the reader should recognize that guidelines, methods and reporting are highly context dependent, and that rigid standards cannot and should not be applied indiscriminately to diverse content areas (11).

QUESTIONS FOR GUIDELINE CONSUMERS Who participated in the CPG development?:
The process used to develop the CPG should be described explicitly and should include representation from all related health care disciplines.Delegates from private industry, government payers, academia, community practice, professional organizations and patient advocacy groups may differ substantially in their priorities and perspectives on any given clinical question.Although value judgments may be inevitable when seeking a consensus among competing interests, these must be recognized as such and made explicit.The specific goals of the CPG should be stated, and the affiliations and potential conflicts of interest of all participants must be identified to allow interpretation of CPG recommendations in the appropriate context.CPG developers are encouraged to have their recommendations reviewed and/or endorsed by the relevant professional organizations, content experts and users' groups, especially if those interests were not involved directly in their development.

How was the evidence used to inform the CPG collected?:
A systematic approach should be used by the guideline developers to identify all pertinent literature published in peer reviewed journals.Evidence used to direct the CPG recom-

506
Can J Gastroenterol Vol 14 No 6 June 2000

Marshall TABLE 1 Key questions for the critical appraisal of clinical practice guidelines (CPG)
Questions for guideline consumers Who participated in the CPG development?How was the evidence to inform the CPG collected?How was the evidence graded and combined?Was the target population for the CPG clearly identified?
Were the date and life expectancy of the CPG specified?Are the CPG flexible?What are the resource implications of CPG implementation?Questions for guideline developers Will the CPG influence practice patterns?Will the CPG improve patient outcomes?
Figure 1) MEDLINE citations per calendar year indexed with the MeSH term "practice guideline".*Incomplete mendations may also be derived from other sources, including expert opinion, decision and analytical models, administrative databases and focus groups.Although the appropriate data sources are determined by their availability and by the nature of the clinical question addressed by the CPG, the strategy used for data retrieval must be clearly identified and should be entirely reproducible.How was the evidence graded and combined?:Although any source of evidence may be evaluated when developing CPG, each has inherent strengths and weaknesses.There exists no single 'correct' approach to the synthesis of discrepant evidence.However, the specific procedures chosen by the CPG developers should be described, and the sources of evidence used to support key recommendations should be acknowledged.A standard system used to grade recommendations based on the quality of supporting evidence is presented in Table 2 (12).In most cases, the results of systematic reviews that pool large, rigorous and unequivocal randomized, controlled trials are given preference over observational evidence from case controlled studies, case series and expert opinion.Randomized, controlled trials provide even stronger evidence if there is little variation or 'heterogeneity' among their results and if the magnitude of the treatment effect is large (ie, the 'number needed to treat' falls below an acceptable threshold).Although the results of clinical trials may not be appropriate to all clinical questions, populations and health care settings, a decision to reject or contradict higher level evidence should be justified.Recommendations that are based solely on expert opinion or Delphi panels in the absence of a higher quality source should be identified explicitly as such.

Was the target population for the CPG clearly identified?:
A CPG that targets a specific health care discipline, subspecialty, practice plan or patient population may not be generalizable.For example, substantial but entirely appropriate differences may exist between primary and tertiary care settings because of referral biases, differential access to technology and divergent patient expectations.Similarly, physicians, nurses and allied health professionals may identify discrepant treatment goals and priorities when evaluating the same clinical scenario.Among individual patients, differences in demographics, comorbid illness and treatment tolerance may make global recommendations irrelevant.Because no guidelines can answer all questions for all scenarios, the CPG must identify the settings to which they apply and those to which they clearly are not relevant.

Were the date and life expectancy of the CPG specified?:
As medical science evolves rapidly, CPG require periodic revision to incorporate new evidence as it arises.Particularly, because there is often a significant delay from data collection to publication, both the date of the CPG's release and the date of the most recent evidence that it considered should be stated clearly.In addition, the CPG should stipulate both a mechanism for the periodic revision of its recommendations and an approximate time horizon for their expiry.Are the CPG flexible?: It is critical that CPG acknowledge the unique nature of each clinical encounter and practice setting, and allow practitioners and their patients to choose other options when appropriate.While CPG cannot be expected to provide an exhaustive menu of management options, clinical circumstance may justify approaches not recommended by the CPG and endorse the role of patient preferences in decision making.By stating explicitly that CPG are flexible, developers may also avoid the misinterpretation of CPG as mandatory protocols or medicolegal standards of practice.Standardized disclaimers have been developed by the Canadian Association of Gastroenterology Practice Affairs Committee (13) and the American College of Gastroenterology Practice Parameters Committee (14) to accompany guideline statements in order to address these concerns.What are the resource implications of CPG implementation?: In an era of constrained resources, consideration should be given to the economic impact of adopting the recommendations made by the CPG.This becomes particularly important if the clinical disorder in question is highly prevalent, if the proposed drugs or technologies are costly or if new infrastructure must be established to provide the services.CPG are not developed as tools for the rationing of health care resources (15), and economic constraints should not be the chief determinant of CPG recommendations.However, CPG cannot advocate a strategy that would be prohibitively expensive to implement widely.When economic concerns are relevant, CPG developers should analyze the costs and health outcomes formally generated by alternative strategies, and estimate the incremental cost effectiveness of adopting the more effective but costly approach.If the CPG developers choose to advocate a less effective strategy because of economic constraints, this should be stated explicitly.

QUESTIONS FOR GUIDELINE DEVELOPERS
Will the CPG influence practice patterns?: Despite the rising prominence of CPG, studies evaluating their impact on practice patterns have yielded mixed and often disappointing  A fundamental challenge for most CPG developers has been the lack of serious attention paid to them by providers.Surveys of Canadian physicians have found that most do not refer routinely to CPG and that those who do so are unlikely to alter their practice patterns as a result (1,17).However, there is good reason to believe that the acceptance and uptake of CPG are improving over time.A 1998 Canadian survey (19) found that 49% of physicians had changed their practice in the previous year as a result of CPG, compared with only 32% of those surveyed in 1994.The observation that young physicians who had 'grown up' with CPG were more likely to use them provides optimism that this trend will continue.
The uptake of CPG can be improved by changes in style and presentation.In their national survey, Hayward et al (21) demonstrated that endorsement of the CPG by a respected professional organization or colleague enhanced their acceptance by practitioners.Uptake of CPG was also influenced by the user-friendliness of their format.Use of clear and unambiguous language improves retention of key messages and avoids misinterpretation.If appropriate to the clinical question at hand, the CPG should provide a concise summary of its main recommendations.A standardized abstract format for the reporting of CPG has been proposed (21).
Among the important attributes of CPG that influence uptake, Rogers (22) used the terms 'triability' and 'observability' to measure whether providers can easily test drive components of the CPG in their own practice, and whether other practices that have implemented the CPG can be observed.The same author also suggested that the adoption of CPG, like that of other innovations, is dependent upon their compatibility with the prevalent beliefs and values of the practitioner.Providers are unlikely to accept CPG that appear to contradict their personal experience.
CPG producers and consumers should be encouraged by the recent advent of online electronic repositories that allow easy and fast access to prescreened, indexed and updated CPG (Table 3).The Agency for Healthcare Research and Quality (formerly the Agency for Health Care Policy and Research) maintains a thorough online database of full text CPG that satisfy prespecified entry criteria for methodology and reporting (23).The Canadian Medical Association maintains the CPG Infobase, which contains over 750 fulltext CPG produced or endorsed by Canadian organizations (24).Will the CPG improve patient outcomes?: Whether CPG improve patient outcomes is a fundamental question for CPG developers and consumers; however, improvements in health outcomes are difficult to measure and demonstrate.In order to improve health, CPG must first meet the challenge of altering practice patterns, and then cross the gulf between efficacy and clinical effectiveness.The efficacy of an intervention is its outcome in the carefully controlled environment of a clinical trial, with selected patients, high compliance and close monitoring.In contrast, clinical effectiveness represents the outcome of an intervention in real world practice, with a less selected population and numerous competing interests.Because few studies are designed to measure effectiveness, CPG recommendations usually are based on the results of efficacy trials, with the implicit assumption that their outcomes can be reproduced in the real world.
The impact of implementing CPG has itself been the subject of a number of controlled trials, with individual patients, physicians or practices randomly assigned to alternative CPG initiatives.Most such studies have measured changes in the process of care rather than patient outcomes, and a recent systematic overview has shown that only a minority of those that studied health benefits yielded significant results (25).However, rigorous randomized trials probably are not well suited to behavioural research and are prone to many of the biases of individual efficacy trials described above.Thus, observational studies of CPG implementation may be more informative (26,27).
Although enthusiasm for CPG may be tempered by their unproven benefits on health outcomes, there is hope that the effectiveness of CPG will continue to improve substantially in the future.This optimism can be attributed to the rising profile of CPG, the widespread recognition of the need to distill large volumes of research data to pragmatic clinical 'bottom lines', a recent emphasis on more rigorous standards for CPG development and a better understanding of effective techniques for their dissemination, marketing and promotion.

SUMMARY
CPG are a useful tool to synthesize evidence and promote optimal clinical practice.Given the increasing prevalence of CPG in the medical literature, health care practitioners should familiarize themselves with the process and outcomes of CPG development.Criteria to evaluate the quality and ef-fectiveness of CPG have been established by many organizations and are synthesized here to a simplified, nine-question approach to critical appraisal.Adherence of CPG to methodological standards is improving over time, as is their acceptance among health care providers.If CPG are to accomplish their goals of improving health while making more efficient use of health care resources, even greater attention must be paid to their creation, dissemination, adoption and re-evaluation.
Can J Gastroenterol Vol 14 No 6 June 2000 507