Funding the new biologics – A health economic critique of the CCOHTA report: Infliximab for the treatment of Crohn’s disease

Presented at a workshop held at the University of Calgary, September 24, 2002. The workshop was supported by an unrestricted grant from Schering Pharmaceuticals, Pointe-Claire, Quebec Centre for Health and Policy Studies (CHAPS), University of Calgary, Calgary, Alberta Correspondence: Dr Craig R Mitton, Centre for Health and Policy Studies (CHAPS), 3330 Hospital Drive Northwest, Calgary, Alberta T2N 4N1. Telephone 403-210-9316, fax 403-210-3818, e-mail crmitton@ucalgary.ca CR Mitton. A health economic critique of the CCOHTA report: Infliximab for the treatment of Crohn’s disease. Can J Gastroenterol 2002;16(12):873-876

The report of the Canadian Coordinating Office for Health Technology Assessment (CCOHTA) on the use of infliximab in the treatment of refractory Crohn's disease stated that the medication did not meet 'conventional standards of cost-effectiveness'.It had several methodological weaknesses, however, including the derivation of the quality-adjusted life-years (QALYs) gained and the interpretation of the incremental cost utility ratios (ICURs).The validity of economic analyses is highly dependent on the underlying assumptions that are made about the implications of health care states and treatments.The authors of the report mapped utilities from three health states, taken from an American study, onto the nine health states that were considered in their economic analysis.The QALYs that were derived might not have been sensitive to small changes in health outcomes.Moreover, the indirect costs of Crohn's disease and its complications were ignored.Therefore, it is possible that the benefits of infliximab therapy were underestimated.The high ICURs that were quoted in the report do not necessarily mean that infliximab is not valuable, because opportunity costs were not considered.Instead of calculating the ICUR, a preferable approach would be to determine the benefit of this therapy, compared with that which could be derived from alternative uses of the same amount of health care resources.A 'balance sheet' approach would allow decision-makers to determine whether the additional cost of infliximab therapy would be justified by the health care gains that it produces.It is inappropriate to assign an arbitrary cut-off point to cost effectiveness, as defined by ICURs, especially when considering new and expensive treatments for severely ill patients who have few other therapeutic alternatives.
Because only a small number of patients would require infliximab, the overall expenditure that would be required to make it available may be manageable.T he purpose of the present paper is to provide a health economic critique of the Canadian Coordinating Office for Health Technology Assessment (CCOHTA) report, Infliximab for the Treatment of Crohn's Disease: A Systematic Review and Cost-utility Analysis (1).Comments will not be provided on the clinical evidence presented in the document; the focus here is primarily on the economic analysis.The authors used a standard Markov decision analytic technique, with realistic values for key cost variables, and tested a range of assumptions in the sensitivity analysis.The primary criticism with this report pertains to the derivation of the quality-adjusted life-years (QALY) measure as an indicator of health benefit and the interpretation of the calculated incremental cost utility ratios (ICURs).Additional concerns include the limited evidence available to 'populate' the decision analysis, thereby resulting in greater uncertainty in the presented results, and the narrow perspective taken in the analysis.

KEY CONCERNS QALY derivation
One key methodological weakness of the CCOHTA study, from a health economic perspective, is in the QALY derivation.The QALY is calculated by multiplying the time spent in a given health state by the valuation of that particular health state.This valuation, or quality adjustment, can be obtained through a number of methods including, as was the case in this study, the standard gamble technique.The authors relied on an American study that had mapped utilities for only three health states.The authors assumed that the utilities for the described health states from the American paper directly corresponded to the health states modeled in the CCOHTA decision analysis.
While utility variation was included in the probabilistic sensitivity analysis, and the authors pointed out that there is more uncertainty on effects than on costs, a key issue here was not sufficiently emphasized.That is, mapping utilities of three health states onto nine states suggests that the derived QALYs, based on the underlying utilities, may not be sensitive enough to detect changes in health benefit.An analogy can be drawn to examining the effectiveness of a given drug on a cohort of patients aged zero to 30 years.One approach would be to simply take the mean value of effectiveness (however measured) for the entire cohort and report this value.A more sensitive approach would be to examine the overall cohort by subgroups, and report results for patients aged less than 10 years, 10 to 20 years, and 20 to 30 years separately.In the same way, the mapping of three measured health states onto the nine health states used in the CCOHTA report is unlikely to lead to a clear picture of more subtle changes in health benefit.
Further, after examining the ICUR results, it is apparent that the relatively high cost per QALY for each of the three scenarios tested against usual care is driven primarily by a very small change in QALYs (eg, from base case to scenario B, the incremental QALY was 0.01524).One could thus raise the question as to whether the QALY measure captured meaningful changes in health gain.Of course, it may in fact be that the health gains are indeed small, but the limited underlying clinical evidence (reported elsewhere in this issue of the Journal) would suggest that this is not the case.While the evidence is not found in the report to adequately answer this question, the study may underestimate the full impact, in terms of health gain as measured with QALYs, of the drug treatment under investigation.If the available clinical evidence did not suggest the effectiveness of infliximab for the treatment of Crohn's disease, it would be difficult to refute the QALY results; as it is, there is a question about whether the QALY measure is picking up meaningful health gain, and as such, interpretation of results based on this measure should be viewed cautiously.

QALY interpretation
A second, more fundamental, issue with this study involves the reporting of results in a cost per QALY manner.Although it is typical to calculate an incremental cost effectiveness ratio (ICER) or ICUR in these types of studies, such an approach actually misrepresents the findings described in the medical literature.Putting the ICUR forward as was done in this report, and referring to a standard threshold for cost effectiveness, might serve to confuse decision-makers, and more importantly, ignores the fundamental economic principle of opportunity cost (2).That is, in some cases, a cost per QALY of $180,000 would be a reasonable buy and, in other cases, where resources are tighter, it may not be.
Quite simply, a low ICUR (or ICER) does not mean that a given treatment as compared with another is cost effective (3).Rather, it is an indicator of how much health care expenditure is required to obtain a stated benefit.As health care spending could be allocated to other services or treatments in the overall system, the question becomes one of efficiency of allocation.Thus, funding decisions should be based only on explicit comparisons with benefits obtained through other (ie, broader) uses of those resources.It is then, upon such broader comparison, that statements regarding cost effectiveness can be made.
The starting point in addressing this issue is to identify the overall incremental budget impact of infliximab treatment in a single jurisdiction (eg, Alberta).While some mention of this is made in the CCOHTA report for the country as a whole, the calculations provided are cursory at best.Because this is the most fundamental issue in the interpretation of the cost per QALY results provided, such an oversight should be seen as a major failure by the authors of the report.Instead of calculating a QALY and then a cost per QALY figure, the costs and benefits should be presented separately in a balance sheet manner (4).This would enable decision-makers to assess whether the additional benefit obtained from the new treatment is worth the extra cost, without being distracted by the apparent cost effectiveness of the treatment, based on a cost per unit of benefit calculation.As argued above, the QALY may not even be the appropriate measure of benefit (due to its potential lack of sensitivity in this situation), thus a range of clinical benefit measures should be presented in conjunction with the incremental cost to obtain the stated benefit.While the QALY has an inherent attractiveness for formulary decisions with its apparent ability to allow broad comparisons across diverse treatment populations, the limitations of this measure, beyond those identified here, have been clearly outlined in the literature (5).This is not to say that QALYs are never appropriate; they are often a useful measure of health benefit, but they are sometimes not appropriate, and thus their use must be weighed carefully.
The authors of the CCOHTA report seem to contradict themselves on these matters in stating, "In its base-case, the economic impact of Infliximab exceeded what is generally considered good value for money."But then, the authors also point out that the: "ICUR alone is not sufficient to determine whether programs should receive funding.It remains up to health care payers, patients and physicians to review the parameters and assumptions of this model, weigh the opportunity costs, consider the alternatives and determine whether Infliximab therapy for [Crohn's disease] makes appropriate use of resources." Are the authors saying that infliximab is not cost effective based on a comparison with an arbitrary cut-off point, or do they recognize that such an approach is inherently flawed, because the economic principle of opportunity cost is ignored?
In essence, making a statement recommending (or not recommending) funding based on an incremental cost effectiveness ratio contravenes the fundamental economic principle of opportunity cost (3).As Auld et al (6) state, "some judgment is required as to whether such extra costs are worth incurring, that is whether more resources should be allocated to that area of care".A more appropriate approach, as advocated in the literature, is to present the costs and benefits alongside of each other, which then allows decision-makers to weigh out resource use in the area under question vis-à-vis alternative means of spending those limited resources.Incremental ratios could feed into the broader decision-making process, but should never be the stand-alone factor on which a funding decision is made.

Other issues
Standard Markov modelling with sensitivity analysis was used in this study.This analysis is limited by the fact that the underlying evidence from the literature is weak and, thus, assumptions were required while constructing the model.That is, the evidence is limited in that few rigorously designed studies are available to inform model construction and ultimate decision-maker assessment.A decision analysis is only as strong as the underlying evidence, as decision analyses do not introduce new data.Rather, their purpose is to summarize or draw together evidence on costs and benefits.Clearly, the results from decision analyses that are based on literature that is not fully developed should be viewed with caution.In the CCOHTA report, it can be noted, for example, that there is a wide range of values identified for the incremental health benefit through the sensitivity analysis.Remodelling would not be beneficial at this stage because a decision analysis can obviously only be based on the available clinical and economic literature.The often-heard call for further primary data certainly applies here.
Another stated limitation with the study is the narrow perspective taken in the analysis.While the authors comment that so-called indirect costs (eg, lost productivity) were not available for inclusion, it could reasonably be expected, noting the apparent clinical benefit of the drug treatment based on the limited literature available, that a meaningful number of days of work lost could be averted.If the data are not currently available to provide insight into this matter, then again, this is clearly an area for further investigation.
Finally, the costs used in the model seemed to be reasonable, but clinical review would be required to determine if all relevant items were included.Clinical assessment is also required to determine whether appropriate transition rates were used, and if the additional assumptions listed in Table 1 were valid.Each of these items would have an effect on the final results.While the methodology appears to be sound from an economic perspective, the key issues requiring clinical review are whether the right costs were included and if the most appropriate underlying assumptions were made.

NEXT STEPS
From an economic perspective, the most important issue to clarify is the number of patients likely to require infliximab in a given year in Alberta.This would allow for a calculation of the incremental budget impact of this drug.Decision-makers could then assess whether the identified benefit from infliximab, over and above usual care, is worth the incremental cost, in comparison to other potential uses of those additional resources.
A rough analysis, requiring verification, would be as follows: • About 200 patients per year in Alberta could receive this drug.
• Estimated incremental cost for the treatment (Scenario B) is $2,762 per patient.

CONCLUSIONS
Noting the concerns presented herein, a number of conclusions are offered.First, because the derivation of QALYs was methodologically weak, basing decisions on the resulting ICURs presented in the report is risky.Second, decision-making bodies should have a full understanding of the meaning of ICURs, noting in particular that no "standard cut-off point" exists.Third, further work should make every effort to capture indirect costs, in particular those related to lost productivity.Finally, further underlying clinical and economic evidence is required to reduce the key assump-tions used in this decision analysis.Decision-makers should be advised of the cost and benefits of the treatment, using a balance sheet approach, and the overall impact on the regional or provincial budget.While QALYs do have an inherent appeal, due caution should be applied when interpreting results based on this measure.
HOT TOPICS IN GASTROENTEROLOGY Funding the new biologics -A health economic critique of the CCOHTA report: Infliximab for the treatment of Crohn's disease Craig R Mitton PhD Presented at a workshop held at the University of Calgary, September 24, 2002.The workshop was supported by an unrestricted grant from Schering Pharmaceuticals, Pointe-Claire, Quebec Centre for Health and Policy Studies (CHAPS), University of Calgary, Calgary, Alberta Correspondence: Dr Craig R Mitton, Centre for Health and Policy Studies (CHAPS), 3330 Hospital Drive Northwest, Calgary, Alberta T2N 4N1.Telephone 403-210-9316, fax 403-210-3818, e-mail crmitton@ucalgary.caCR Mitton.A health economic critique of the CCOHTA report: Infliximab for the treatment of Crohn's disease.Can J Gastroenterol 2002;16(12):873-876