3 The manufacturer's submission
The Appraisal Committee (appendix A) considered evidence submitted by the manufacturer of ipilimumab and a review of this submission by the Evidence Review Group (ERG; appendix B).
3.1 The key evidence for the clinical effectiveness of ipilimumab came from 1 trial (MDX010-20), which assessed the efficacy and safety of ipilimumab in adults with advanced, unresectable stage III or stage IV malignant melanoma who had been previously treated with interleukin-2, dacarbazine, temozolomide or other chemotherapies. This evidence was supported by results from a dose-ranging trial (CA 184-022), and a safety and tolerability trial (CA 184-007).
3.2 The MDX010-20 trial was an international, multicentre, double-blind, 3-armed, randomised, controlled trial. A total of 676 adults with advanced malignant melanoma were randomised to receive ipilimumab 3 mg/kg in combination with an investigational gp100 peptide vaccine ('ipilimumab plus gp100'; n=403), ipilimumab 3 mg/kg in combination with placebo ('ipilimumab alone'; n=137), or gp100 in combination with placebo ('gp100 alone'; n=136) every 3 weeks for 4 cycles. Approximately 38% of patients in the trial were from Europe, with 8% from the UK. The patients were all HLA-A*0201 (human leukocyte antigen serotype group) positive and were generally well balanced for key baseline characteristics. At study entry, nearly all patients (98.2%) had stage IV disease.
3.3 The primary outcome of the MDX010-20 trial was overall survival for people treated with ipilimumab plus gp100 compared with gp100 alone. Secondary outcomes in the trial included overall survival in people treated with ipilimumab plus gp100 compared with ipilimumab alone, best objective response rate, disease control rate, duration of response, progression-free survival, time to progression and health-related quality of life.
3.4 Results from the MDX010-20 trial showed that ipilimumab plus gp100 led to a statistically significant increase in median overall survival by approximately 3.5 months compared with gp100 alone (hazard ratio [HR] 0.68; 95% confidence interval [CI] 0.55 to 0.85; p=0.0004). When ipilimumab alone was compared with gp100 alone, ipilimumab increased median overall survival by approximately 3.7 months (HR 0.66; 95% CI 0.51 to 0.87; p=0.0026). There was no statistically significant difference in median overall survival between people treated with ipilimumab plus gp100 and those treated with ipilimumab alone (HR 1.04; 95% CI 0.83 to 1.30; p=0.7575), which the manufacturer considered was evidence that gp100 did not influence the overall survival outcome when combined with ipilimumab treatment. Approximately 65% of people treated with an ipilimumab-containing regimen received all 4 doses of ipilimumab in line with the licensed regimen, and in this subgroup the median survival in both the gp100 and ipilimumab arms was greater than in those who received fewer than the full 4 doses. The differences in survival gain between the ipilimumab and gp100 arms were also more favourable in people treated with all 4 doses (results provided as academic in confidence). All response-related secondary outcomes (including best objective response rate and progression-free survival) showed positive results for people who received treatment with an ipilimumab-containing regimen compared with people who received gp100 alone.
3.5 The CA 184-022 trial was a double-blind, multicentre, dose-ranging, randomised, controlled trial that included 217 patients with previously treated, treatment-refractory or treatment-intolerant unresectable stage III or stage IV melanoma. They were randomised to receive either ipilimumab 0.3 mg/kg, 3 mg/kg or 10 mg/kg every 3 weeks for 4 cycles followed by maintenance therapy every 3 months. The outcomes included estimated best objective response rate, progression-free survival at 24 weeks, median overall survival and duration of response. The CA 184-007 trial was a double-blind, multicentre, randomised, controlled trial. Patients (n=115) with unresectable stage III or stage IV melanoma who were treatment naive or who had been previously treated were randomised to receive open-label ipilimumab (10 mg/kg at weeks 1, 4, 7 and 10) with either concomitant oral budesonide or placebo. The outcomes included adverse reactions (specifically diarrhoea), best objective response rate, duration of response and overall survival.
3.6 The most common adverse reactions associated with ipilimumab treatment reported in the 3 trials included in the manufacturer's submission resulted from increased or excessive immune activity. They included diarrhoea, rash, pruritus, fatigue, nausea, vomiting, decreased appetite and abdominal pain. These adverse reactions were considered to be generally medically manageable and usually reversible with topical and/or systemic immunosuppressants. Progressive disease was the most frequent reason for death in the MDX010-20 and CA 184-022 studies. There were 14 (2.2%) adverse reactions with an outcome of death in the MDX010-20 trial that related to the study treatments; 8 deaths in the ipilimumab plus gp100 group, 4 in the ipilimumab alone group and 2 in the gp100 alone group. Of those deaths, 7 were associated with immune-related adverse reactions (including colitis, bowel perforation and organ failure): 5 in the ipilimumab plus gp100 group and 2 in the ipilimumab alone group.
3.7 The manufacturer undertook a systematic search and identified 10 economic evaluations in pre-treated or advanced melanoma. None of the studies evaluated ipilimumab. The manufacturer therefore submitted a de novo economic evaluation in which people treated with ipilimumab were compared with those who received best supportive care. There were 4 mutually exclusive states included in the model: baseline disease, non-progressive disease, progressive disease and death. All people were assumed to start in the baseline disease state (after chemotherapy), then at the end of each cycle they could move to the non-progressed health state or to the progressed health state, or they could die. The model used daily cycles for the first 5 years during the trial period, and weekly cycles thereafter for a lifetime (30 year) horizon. The perspective adopted in the economic evaluation was that of the NHS and personal social services, and costs and benefits were discounted at 3.5% per year.
3.8 The proportion of people in each health state was calculated using progression-free survival and overall survival data from the MDX010-20 trial. Data on progression-free survival and overall survival for people receiving best supportive care were not available directly from the trial. However, results from the trial showed that treatment with gp100 alone led to a median overall survival of 6.4 months, which was consistent with survival estimates achieved with best supportive care. Therefore, data from the gp100 arm of the trial were assumed by the manufacturer to be a proxy for the course of disease in people receiving best supportive care. Adverse-reaction rates for ipilimumab and best supportive care were estimated from the MDX010-20 trial. The resource costs included in the model were drug acquisition and administration costs, and the cost of the disease, which included costs related to each health state and of treating adverse reactions.
3.9 In the manufacturer's original submission, 2 approaches to parametric curve fitting for the survival modelling were presented. The first strategy involved a single curve fit approach that showed that none of the curves fitted the Kaplan-Meier data from the MDX010-20 study. The second strategy involved using a 2-part curve fit in which the Kaplan-Meier estimates for overall survival and progression-free survival were used for the first 18 months and 'best-fit' parametric curves were used thereafter. The manufacturer concluded that the 'best-fit' curves were: exponential for progression-free survival in the ipilimumab arm, Gompertz for overall survival in the ipilimumab arm and exponential for overall survival in the best supportive care arm. Progression-free survival in the best supportive care arm was represented by the overall survival arm.
3.10 Health-related quality of life was measured in the MDX010-20 trial, which used the European Organisation for Research and Treatment of Cancer (EORTC) QLQ-C30 and the short form 36 (SF-36) questionnaires. The questionnaires were predominantly completed by trial participants at baseline and week 12, with only 26 questionnaires completed after week 12. For the economic analysis, utilities were obtained from responses to the EORTC QLQ-C30 from 971 trial observations using a recently developed preference-based version of the instrument. The utility values assumed for the progression-free disease and progressive disease health states in the model were 0.80 (95% CI 0.53 to 0.97) and 0.76 (95% CI 0.46 to 0.97) respectively. The manufacturer also conducted a systematic review to identify studies that included health-related quality-of-life data for people with metastatic melanoma. One study was identified that included 63 patients from the UK and 77 patients from Australia who valued 'vignettes' or descriptions of advanced melanoma health states developed by the researchers.
3.11 In the manufacturer's original base case, ipilimumab treatment led to an undiscounted incremental gain in overall survival of 33.8 months compared with best supportive care. The incremental cost-effectiveness ratio (ICER) for ipilimumab compared with best supportive care was £60,737 per quality-adjusted life year (QALY) gained (incremental cost of £83,351 and incremental benefit of 1.37 QALYs). This was based on the assumption that all patients received 3.3 doses of ipilimumab at 3 mg/kg body weight, corresponding with the average number of doses used in the clinical trial.
3.12 Probabilistic sensitivity analysis reported a 14% chance of ipilimumab being cost effective compared with best supportive care at £50,000 per QALY gained. Deterministic sensitivity analyses showed that the ICER was sensitive to the utility values assumed for the progressive disease health state. An increase in this utility value reduced the ICER and conversely a reduction in utility increased the ICER. For example, using a lower utility (0.60) for progressive disease increased the base-case ICER to £73,854 per QALY gained. Structural sensitivity analysis also showed that decreasing the discount rate to 0% reduced the ICER from £60,737 to £42,871 per QALY gained because the long-term benefits of ipilimumab in the base case were discounted to a large degree, whereas costs of treatment were only incurred in the first year of the model, and therefore were unaffected by discounting.
3.13 The manufacturer also conducted scenario analyses to explore the effect on the ICER of assumptions about the amount of each dose of ipilimumab needed per patient and the possibility of vial sharing. Results from these analyses showed that the dose of ipilimumab given per patient has a large impact on the manufacturer's ICER, with the minimum dose given in the trial and compassionate use programme (3×50 mg) resulting in an ICER of £38,387 per QALY gained and the maximum dose (2×200 mg) given resulting in an ICER of £88,788 per QALY gained. In addition, the results showed that vial sharing has the potential to reduce the manufacturer's original base-case ICER to £55,824 per QALY gained.
3.14 The ERG reviewed the clinical-effectiveness evidence for ipilimumab and noted that none of the studies included in the manufacturer's submission compared ipilimumab with any of the comparators listed in the decision problem (best supportive care, carboplatin-based chemotherapy and dacarbazine). The ERG commented that the MDX010-20 study was well designed and that it was satisfied that the participants were representative of patients in UK clinical practice. The ERG expressed concern that the manufacturer considered gp100 clinically comparable to best supportive care because patient outcomes in the gp100 alone arm of the MDX010-20 study appeared less favourable than might be expected in untreated people.
3.15 The ERG commented that the clinical data provided by the manufacturer suggested that treatment with ipilimumab was associated with a long-term overall survival benefit over gp100 for a small number of patients. However, it noted that to date no patient characteristics or biomarkers have been identified that can prospectively identify the people most likely to benefit from treatment with ipilimumab. The ERG noted that the European Medicines Agency considered a number of supplementary analyses carried out by the manufacturer in an attempt to identify possible subgroups of people who might (or might not) benefit from treatment with ipilimumab. However, the subgroups were small and the ERG determined that no conclusions could be drawn from this analysis.
3.16 The ERG considered that the manufacturer's model was well constructed, but it proposed a number of minor corrections and modifications, which resulted in a reduction in the base-case ICER from £60,737 to £54,462 per QALY gained. However, the ERG noted that the main weakness of the manufacturer's original model was the estimate of mean overall survival. The ERG acknowledged that the natural history and prognosis for metastatic melanoma is not well understood and the manufacturer claimed a substantial improvement in mean survival on the basis of results from a single trial. The ERG cited a study published in 1999 involving a re-analysis of 8 trials of interleukin-2 for people with metastatic melanoma. Of the patients, 80% died within 2 years but most of those surviving the 2-year follow-up period survived for a further 9 years. The ERG noted that this response pattern was replicated in the MDX010-20 study and suggested that this was likely to be because survival rates for people with advanced metastatic melanoma vary substantially. In light of this, it is possible that the data available for analysis are weakest when improved survival is likely to generate the most added life years from the treatment. The ERG therefore noted that, although the MDX010-20 trial used by the manufacturer showed a survival advantage for ipilimumab, it was unable to reliably quantify the long-term survival benefit.
3.17 The ERG had concerns about the manufacturer's interpretation of the MDX010-20 trial data. In particular, it noted that the fitted overall survival functions beyond 18-month follow-up generated mortality risks lower than those in the general (healthy) population at a comparable age and, as a consequence, the model predicted substantial numbers of people surviving to unreasonably advanced ages (beyond 100 years). To counter this anomaly, the manufacturer replaced the calculated model mortality risks with mortality risks experienced by the general population beyond 5-year follow-up. The ERG noted that this approach implied that anyone surviving beyond 5 years of second-line systemic treatment was effectively cured; however, no evidence was submitted by the manufacturer in the original submission to support this claim.
3.18 In an exploratory analysis, based on the manufacturer's original submission, the ERG adopted a pragmatic approach to model overall survival by calculating the area under the Kaplan-Meier curve to a common late time point beyond which both the ipilimumab and best supportive care arms could be seen to be following long-term trend lines. It then projected further life expectancy based on calibrating a parametric function. The results from this method suggested mean undiscounted life years of 11.2 months for gp100 alone and 27.4 months for the combined ipilimumab arms from the MDX010-20 trial, which equated to a mean gain in overall survival of 16.2 months. These results were noted to be less than half the value calculated in the base case of the manufacturer's original model (that is, a mean gain in overall survival of 33.8 months). Using the revised projections, the ERG noted that the manufacturer's original base-case ICER increased to £96,717 per QALY gained. The ERG stated that its exploratory analysis on overall survival cannot be considered definitive because the volume and duration of patient data available from the MDX010-20 trial were inadequate to achieve survival projections that can be used as a basis for decision-making. However, the ERG considered that the manufacturer's original model is likely to have overestimated the extent of survival benefit associated with treatment with ipilimumab, which would have a considerable effect on the ICER.
3.19 In response to consultation, the manufacturer offered an alternative approach to modelling long-term survival for people who are treated with ipilimumab. The manufacturer used a 3-part curve fit approach with Kaplan-Meier analysis results from MDX010-20 unmodified for the first 18 months, followed by a parametric model (Gompertz) fitted to the trial data from 18 months to 5 years and, thereafter, hazards derived from analysis of a malignant melanoma disease register modified by background mortality rates. The manufacturer used data from a published register of 1158 patients with stage IV melanoma in the USA. The manufacturer used the survival curve from this analysis as the basis for estimating the extended survival of patients (beyond 5 years) in the MDX010-20 trial. This was further modified to include age-related mortality because the register data included melanoma-related death only. In this revised model the survival estimates were further adjusted, using a Cox proportional hazards regression, to reflect the difference in overall survival for long-term survivors in the combined ipilimumab treatment groups (ipilimumab alone and ipilimumab plus gp100) and the gp100 alone group (HR 0.782). By including this revised approach in the model, the estimated survival gain for people treated with ipilimumab was 30.0 months (compared with their original estimate of 33.8 months).
3.20 The Department of Health agreed a patient access scheme in which a simple discount is applied to the list price of ipilimumab. For the analyses including the scheme, the manufacturer presented additional survival data from 3 smaller trials (CA 184-007, CA 184-008 and CA 184-022: all individual parent studies included in CA 184-025) comprising patients who initially received 0.3 mg/kg, 3 mg/kg or 10 mg/kg dosages of ipilimumab. The manufacturer identified 72 patients from a dose-ranging study (CA 184-022), who had received previous treatment and then ipilimumab at the licensed dose of 3 mg/kg. These were considered to be comparable to the patients treated with ipilimumab in the MDX010-20 trial, and the manufacturer presented a pooled analysis of the MDX010-20 trial patients supplemented by these patients from the CA 184-022 trial. The manufacturer also proposed a broader pooling of all data from patients treated with ipilimumab, regardless of dosing level. The manufacturer stated that these additional data, which provided follow-up information on patients treated with ipilimumab for between 50 and 70 months, confirmed the long-term effect of ipilimumab at all dosages.
3.21 The manufacturer's revised base-case analysis was calculated using the 3-part curve fit approach, the patient access scheme, and trial data for patients receiving the 3 mg/kg dose of ipilimumab in the MDX010-20 trial pooled with 72 patients from the CA 184-022 dose-ranging trial receiving the same dose. This resulted in a revised base-case deterministic ICER of £46,739 per QALY gained for ipilimumab compared with best supportive care. Using only the MDX010-20 trial data, the ICER was £42,211 per QALY gained. The manufacturer considered that the probabilistic sensitivity analyses showed a low level of parameter-related uncertainty around the baseline ICER and reported that there was an approximately 81% chance that the ICER for ipilimumab would be less than £50,000 per QALY gained when the patient access scheme was included.
3.22 The manufacturer conducted structural sensitivity analyses and scenario analyses exploring the impact of alternate curve fits, varied cut-off points in the 3-part curve fit approach, alternative sources of observational data and the use of the ERG approach on the overall survival benefit of ipilimumab. The manufacturer stated that the only scenario in which the ICER rose above £50,000 per QALY gained was when the manufacturer adopted the ERG approach for survival estimation, which resulted in an ICER of £55,807 per QALY gained. Further scenario analyses involved excluding patients who crossed over between different doses of ipilimumab and resulted in a nominal reduction in the ICER to £44,426 per QALY gained. The manufacturer stated that the areas identified as potentially problematic by the ERG, such as cut-off points and sources of observational data, did not have a large impact on the ICER, and that the increased patient numbers and number of trials with similar results reduced the uncertainty associated with the estimation of longer-term survival.
3.23 The manufacturer noted that, although the summary of product characteristics for ipilimumab does not recommend vial sharing, discussions with clinicians indicated that vial sharing may be possible in some clinical centres in the UK. The manufacturer provided scenario analyses to explore the impact of vial sharing and noted that, if 50% of ipilimumab wastage was avoided through vial sharing, then the revised base-case ICER would be reduced. The manufacturer also suggested that, if drug-specific utilities were used instead of pooled utilities (which were used in the original economic model), then the revised base-case ICER would decrease further.
3.24 The manufacturer conducted sensitivity analyses using the discount rate of 1.5% for costs and benefits, and also a discount rate of 3.5% for costs and 1.5% for benefits. These resulted in ICERs of £39,714 and £38,323 respectively per QALY gained. The manufacturer's probabilistic sensitivity analysis indicated that, in these scenarios, the ICER for ipilimumab is 94% and 100% likely to be less than £50,000 per QALY gained respectively.
3.25 The ERG considered the manufacturer's revised long-term survival projections submitted in response to consultation. It believed that the main weakness of the manufacturer's survival model was in not providing a rationale, other than replicating the observed data, to support the division of data into 3 time periods and the use of different methods in each period. Furthermore, there were substantial differences between the populations in the MDX010-20 trial and the disease register in terms of patients' initial diagnosis, treatment history and time from initial diagnosis. The ERG also considered that the manufacturer's approach to deriving the long-term hazard ratio was problematic because of the small number of people in the analysis for the gp100 group in the trial (n=19) and the fact that the long-term survival trends were established much earlier in the gp100 group (about 300 days) than in the combined ipilimumab treatment group (about 750 days), which could lead to bias in the hazard ratio. In light of these concerns, the ERG considered that the survival projections presented by the manufacturer in response to consultation were uncertain.
3.26 The ERG reviewed the manufacturer's assumptions about vial sharing. It considered that although vial sharing is theoretically possible, in reality it would be difficult to implement because only 250–300 people nationally will need treatment each year, which only equates to 1–2 people in each centre each month. Therefore, the ERG was not convinced that any specialist centre could achieve regular savings in ipilimumab costs from organised vial sharing.
ERG comments on manufacturer's additional analyses (including the patient access scheme)
3.27 The ERG reviewed the manufacturer's alternative data sets based on the pooling of selected data from several smaller clinical studies with data from the main MDX010-20 trial. The ERG thought that the patient population was too dissimilar to the MDX010-20 trial to allow direct comparison, and that not all studies involved the dosing regimen of ipilimumab (3 mg/kg) for which marketing authorisation has been granted. Therefore, the ERG considered that the broader pooling of data from all patients who received ipilimumab, regardless of dosing regimen or patient baseline characteristics, would lead to uninterpretable results.
3.28 Regarding the pooling of 3 mg/kg data from the MDX010-20 trial and the CA 184-022 study, the ERG thought that pooling isolated treatment arms across trials was inappropriate because it would break randomisation and therefore invalidate any comparison between the intervention treatment and the comparator. Furthermore, there were no equivalent comparator treatment arms available in the CA 184-022 study to balance the pooling of the ipilimumab trial arms. Finally, the ERG thought that the additional data from the CA 184-022 study did not extend the length of follow-up beyond that in the MDX010-20 trial, and therefore did not reduce the uncertainty surrounding long-term outcomes. Therefore, the ERG considered that it was only appropriate to include data from the MDX010-20 trial. Consequently, the ERG found that the patient access scheme would decrease the ICER for ipilimumab compared with best supportive care to £66,520 per QALY gained.
3.29 However, the ERG carried out further analyses aimed at clarifying the possible mechanisms underlying the pattern of overall survival observed in the clinical trials and disease registers for patients with advanced malignant melanoma. The ERG proposed a hypothesis assuming that patients with advanced malignant melanoma are drawn from 2 distinct subgroups of unknown aetiology and that each subgroup is characterised by a separate hazard rate that does not vary over time. Based on this, a mixed exponential distribution for overall survival, which consisted of the proportion of the population comprising one of the subgroups and separate hazard rates for each of the subgroups, was considered appropriate. The ERG tested this approach by applying it to published results from 2 large patient registries, which resulted in strong correspondence to the observed data in both data sets and for each disease stage. The results of this method of projecting overall survival in the submitted model, using only the MDX010-20 trial data, resulted in a mean gain in overall survival of 20.9 months. Using this method, combined with the patient access scheme, resulted in an ICER of £58,590 per QALY gained. The ERG's revised approach implies that patients fall into 2 distinct groups in relation to mortality risk, but currently no direct evidence is available to explain how such a differentiation may occur. The ERG also noted that the melanoma databases used in testing the revised approach featured patients from the time of diagnosis, but their use in modelling (both by the manufacturer and by the ERG) began at the time of randomisation, at which point patients may have survived several years of treatment. As a result, direct use of database trends could be misleading.
3.30 Full details of all the evidence are in the manufacturer's submission, the ERG report, the manufacturer's response to the appraisal consultation document, further evidence and analyses, and the ERG's critique of the manufacturer's responses, which are available from www.nice.org.uk/guidance/TA268.