Ipilimumab for previously untreated advanced (unresectable or metastatic) melanoma

Manufacturer's original submission

3.1 There were no trials directly comparing ipilimumab 3 mg/kg monotherapy with the comparators specified in the scope (dacarbazine or vemurafenib). The key clinical evidence came from 4 randomised controlled trials (CA184‑024, MDX010‑08, BREAK‑3 and BRIM‑3) that were used in an indirect comparison of the effectiveness of ipilimumab 3 mg/kg compared with dacarbazine, vemurafenib or dabrafenib. In addition, the manufacturer presented data from 2 ongoing US retrospective, observational trials (CA184‑332 and CA184‑338) because there was limited randomised controlled trial evidence directly investigating the clinical efficacy of ipilimumab 3 mg/kg monotherapy in people with previously untreated advanced (unresectable or metastatic) melanoma. The manufacturer also presented a pooled analysis of patients who had not had chemotherapy before (n=78), randomised to 3 mg/kg ipilimumab monotherapy in 4 trials: MDX010‑08, CA184‑004, CA184‑022 and MDX010‑020.

3.2 The CA184‑024 trial was a multinational, randomised, double-blinded trial observing adults with previously untreated advanced (unresectable or metastatic) melanoma. The intervention was ipilimumab 10 mg/kg in combination with dacarbazine 850 mg/m² (n=250), and the comparator was placebo plus dacarbazine alone 850 mg/m² (n=252). Treatment with ipilimumab or placebo was provided every 3 weeks for the first 10 weeks followed by 1 dose every 12 weeks from week 24. Treatment with dacarbazine was given once every 3 weeks for 22 weeks until disease progression, unacceptable toxicity or withdrawal of consent. The median patient age was 57 years, 60% of patients were male and 40% had an elevated serum lactate dehydrogenase level. The median time from first diagnosis to diagnosis of advanced melanoma was 1.7 years.

3.3 The primary outcome of the CA184‑024 trial was overall survival: median overall survival differed by 2.1 months, from 9.1 months with dacarbazine alone to 11.2 months with ipilimumab 10 mg/kg plus dacarbazine, and there was a 5.7 month survival gain over the 5 year trial. The hazard ratio for death was 0.72 (95% confidence interval [CI] 0.59 to 0.87; p<0.001). There was no statistically significant difference in the median progression-free survival between ipilimumab 10 mg/kg plus dacarbazine and dacarbazine alone (2.8 compared with 2.6 months), but ipilimumab 10 mg/kg statistically significantly increased progression-free survival compared with dacarbazine, with a hazard ratio for progression of 0.76 (95% CI 0.63 to 0.93; p=0.0064). The response rate was statistically significantly improved with ipilimumab 10 mg/kg plus dacarbazine compared with dacarbazine alone (15.2% compared with 10.3%; p=0.03) and the duration of response was statistically significantly longer (median 19.3 compared with 8.1 months; p=0.03) in the ipilimumab group. There were no statistically significant differences in disease control rate (which included response and stable disease rates), time to response, or European Organisation for Research and Treatment of Cancer questionnaire (EORTC‑QLQ‑C30) functioning scales or symptom scales between treatment arms.

3.4 Long-term safety data were available from the CA184‑024 trial, which indicated that the safety profile of ipilimumab was maintained throughout therapy. Severe, serious, drug-related and adverse events leading to drug discontinuation were all more frequent in the ipilimumab 10 mg/kg plus dacarbazine group (46%) than in the group treated with dacarbazine alone (18%). Discontinuations because of trial-drug toxicity led to 37% of patients not receiving all 4 doses of ipilimumab 10 mg/kg. Of the patients receiving ipilimumab, 77.7% experienced an immune‑related adverse event (41.7% were grade 3 or 4 events). The most commonly reported adverse events were hepatic-related with 17.4% to 20.7% of patients experiencing grade 3 or 4 elevations in liver function values, but these reactions were generally reversible. Other adverse events observed in the trial were dermatological events, gastrointestinal events, pyrexia, chills and weight loss. The safety profile of ipilimumab was similar in patients during the induction period and in patients having treatment for longer than 2 years.

3.5 The MDX010‑08 trial was a multicentre, randomised, open-label trial carried out in the USA, including adults with advanced (unresectable or metastatic) melanoma who had not received prior chemotherapy and had a life expectancy of over 3 months. Treatment was either with the intervention, ipilimumab 3 mg/kg plus dacarbazine 1000 mg/m² (n=36), or the comparator ipilimumab 3 mg/kg alone (n=40). The trial was randomised using a central randomisation scheme with stratification by random block size, and because it was carried out in the USA it did not contain any UK patients. The trial was also not designed to detect differences in survival between the 2 treatment arms. The manufacturer provided details of pre-specified subgroups of patients within the trials including median age (60 years and 66 years for the ipilimumab plus dacarbazine and ipilimumab groups respectively), sex (74.3% and 56.8% male for the ipilimumab plus dacarbazine and ipilimumab groups respectively), stage of metastasis, time since diagnosis and lactate dehydrogenase level.

3.6 The MDX010‑08 trial demonstrated that no statistically significant difference was observed for the primary outcome of objective response rate between ipilimumab alone and ipilimumab plus dacarbazine. There was also no statistically significant difference in overall survival between the ipilimumab 3 mg/kg plus dacarbazine group and the ipilimumab alone group. The results appeared to favour ipilimumab plus dacarbazine over ipilimumab alone (median overall survival of 14.3 months compared with 11.4 months with a hazard ratio for overall survival of 0.75 [95% CI 0.45 to 1.24] and a 1‑year overall survival rate of 62% compared with 45%), but the trial was underpowered to detect statistically significant differences in overall survival. All patients in both treatment groups experienced at least 1 adverse event, with 65.7% in the ipilimumab plus dacarbazine group compared with 53.8% in the ipilimumab alone group experiencing an immune-related adverse event. Serious adverse events, drug-related adverse events and adverse events leading to drug discontinuation were more frequent in the ipilimumab alone group than in the ipilimumab plus dacarbazine group. One person died from a suspected drug-related adverse event in the ipilimumab plus dacarbazine group and 2 people died in the ipilimumab alone group.

3.7 The BRIM‑3 trial was a multinational, randomised, crossover trial including adults with previously untreated advanced (unresectable or metastatic) melanoma that was also BRAF V600 mutation-positive, and who had a life expectancy of over 3 months. People received either vemurafenib 960 mg (n=337) twice daily or dacarbazine 1000 mg/m² (n=338) every 3 weeks. Randomisation was used to assign patients in a 1:1 ratio. The manufacturer included details of sub‑populations within the trial. These included the age (median 56 years in the vemurafenib group and 52 years in the dacarbazine group) and sex of patients (59.4% and 53.6% male respectively), their Eastern Cooperative Oncology Group (ECOG) performance status (most patients in both groups had a score of 0), their stage of metastasis (most patients in both groups had metastases at the M1c stage; distant metastases were found) and lactate dehydrogenase level (most patients in both groups had levels above the upper limit of normal).

3.8 The BRIM‑3 trial demonstrated that vemurafenib statistically significantly increased overall survival in patients who had BRAF V600 mutation-positive melanoma when compared with dacarbazine. Median overall survival was increased by 3.6 months in the vemurafenib group (13.2 months compared with 9.6 months; hazard ratio 0.62; 95% CI 0.49 to 0.77; p<0.001). Overall survival rates were higher with vemurafenib than dacarbazine at 6 months (84% compared with 64%). Progression-free survival was also statistically significantly increased for patients in the vemurafenib treatment group, with a median progression-free survival of 5.3 months compared with 1.6 months and a hazard ratio for progression of 0.26 (95% CI 0.20 to 0.33; p<0.001). The response rate was statistically significantly improved with vemurafenib compared with dacarbazine (48.4% compared with 5.5%; p<0.001). Time to response was 1.5 months for vemurafenib compared with 2.7 months for dacarbazine, although duration of response was not reported. The BRIM‑3 trial used the Functional Assessment of Cancer Therapy Melanoma (FACT-M) questionnaire to measure health-related quality of life but the data were not reported because of low completion rates.

3.9 The manufacturer also presented the BREAK‑3 trial that compared dabrafenib 150 mg with dacarbazine 1000 mg/m² in adults with advanced (unresectable or metastatic) melanoma who tested positive for the BRAF V600 mutation. The manufacturer included this trial as part of the mixed treatment comparison described in section 3.15 but did not include it in the cost-effectiveness analyses because it was not included in the NICE scope, had limited publicly available data, and dabrafenib did not have a UK price.

3.10 Two ongoing US retrospective, observational studies (CA184‑332 [n=61] and CA184‑338 [n=120]) for ipilimumab 3 mg/kg in people who had not previously received treatment were also included in the manufacturer's submission. The median overall survival for ipilimumab 3 mg/kg monotherapy was 11.5 months in the CA184‑332 trial and 14.3 months in the CA184‑338 trial. The manufacturer reported that in the CA184‑338 trial, 54.2% of people treated with ipilimumab 3 mg/kg experienced a drug-related adverse event. BRAF V600 mutation status data were available, and the manufacturer suggested that a post-hoc analysis of CA184‑338 supported the conclusion from a previous post-hoc analysis of CA184‑004 that tumour mutation status does not impact on the clinical activity of ipilimumab with no differences in survival observed between patients who have BRAF V600 mutation-positive melanoma and those who have BRAF V600 mutation-negative melanoma.

3.11 The manufacturer presented a pooled analysis of patients (n=78) randomised to 3 mg/kg ipilimumab monotherapy (MDX010‑08, CA184‑004, CA184‑022 and MDX010‑020). It was noted that 43 out of 78 patients had received prior immunotherapy. The manufacturer stated that CA184‑004 and CA184‑022 were not included as stand-alone trials because the patient numbers were too small but stated that they demonstrated the clinical equivalence between 3 mg/kg and 10 mg/kg doses of ipilimumab and supported the extrapolation of the CA184‑024 data. The MDX010‑020 trial was a double-blind trial including patients with advanced (unresectable or metastatic) melanoma who had previously been treated with regimens containing 1 or more of the following: interleukin‑2, dacarbazine, temozolomide or other chemotherapies. Patients were randomised into 3 groups (in a ratio of 3:1:1) who received either 3 mg/kg ipilimumab plus an investigational gp100 peptide vaccine (n=403), 3 mg/kg ipilimumab alone (n=137) or gp100 alone (n=136). Patients were enrolled regardless of BRAF V600 mutation status. Follow-up was up to 55 months. The hazard ratio for comparison of overall survival between 3 mg/kg ipilimumab alone and gp100 was 0.66 (95% CI 0.51 to 0.87; p=0.0026). The median overall survival for ipilimumab 3 mg/kg monotherapy in this pooled analysis was 13.5 months (95% CI 11.2 to 19.6).

3.12 The manufacturer made several assumptions to support the clinical and cost effectiveness of 3 mg/kg ipilimumab monotherapy. The first key assumption was that ipilimumab 3 mg/kg and 10 mg/kg were clinically equivalent. Data from 2 trials (CA184‑004, 36 chemotherapy-naive patients and CA184‑022, 18 chemotherapy-naive patients) comparing ipilimumab 3 mg/kg and 10 mg/kg were presented in support of this assumption. The manufacturer highlighted that the trials indicated that the survival associated with ipilimumab 3 mg/kg and 10 mg/kg was similar, with median overall survival of 14.3 and 11.2 months respectively. The manufacturer also provided pooled data comparing overall survival profiles of ipilimumab 3 mg/kg (MDX010‑020 and CA184‑022) and 10 mg/kg (CA184‑007, CA184‑008 and CA184‑022) for a mixed population. The manufacturer stated that no statistically significant difference in survival was observed between the 3 mg/kg and 10 mg/kg treatment arms across the whole population. The European Medicines Agency's (EMA) Committee for Medicinal Products for Human Use (CHMP) has requested that the manufacturer conduct a study on any relevant difference in efficacy between 3 mg/kg and 10 mg/kg.

3.13 The second key assumption made by the manufacturer was that ipilimumab plus dacarbazine was equivalent to ipilimumab alone. The manufacturer stated that this was demonstrated in the MDX010‑08 trial in which ipilimumab 3 mg/kg plus dacarbazine (n=32) provided comparable survival times to ipilimumab 3 mg/kg monotherapy (n=32) after a median follow-up of 20.9 and 16.4 months respectively. Median overall survival times were 14.3 months and 11.4 months, and 1‑year survival rates were 62% and 45% in the ipilimumab 3 mg/kg plus dacarbazine and ipilimumab 3 mg/kg alone groups respectively. This difference was not statistically significant. The median overall survival with ipilimumab 3 mg/kg alone was directly comparable with that observed with ipilimumab 10 mg/kg plus dacarbazine in CA184‑024. The CHMP concluded that ipilimumab pharmacokinetic data were not significantly affected by concomitant dacarbazine.

3.14 The manufacturer provided information to demonstrate that ipilimumab efficacy is similar in patients with previously untreated and previously treated melanoma. The manufacturer stated that the results of the MDX010‑020 (previously treated melanoma) and CA184‑024 trials (previously untreated melanoma) demonstrated similar 2‑year overall survival rates: 24% and 29% respectively. Although these trials used different regimens (3 mg/kg ipilimumab and 10 mg/kg ipilimumab plus dacarbazine respectively), the manufacturer stated that the CHMP accepted this evidence from the MDX010‑020 trial supported by high-level results from the CA184‑024 trial as part of the marketing authorisation granted in 2011 for ipilimumab for the treatment of adults with previously treated advanced (unresectable or metastatic) melanoma. The CHMP commented in the licensing assessment report that ipilimumab 3 mg/kg alone could be supported on the basis of the following considerations:

3.15 Data from 3 (CA184‑024, BREAK‑3 and BRIM‑3) of the 4 randomised controlled trials identified were analysed as a mixed treatment comparison to provide a comparison between ipilimumab 10 mg/kg and BRAF inhibitors. The manufacturer stated that given that ipilimumab 3 mg/kg and ipilimumab 10 mg/kg could be considered equivalent, the results of the mixed treatment comparison would also hold for a comparison of ipilimumab 3 mg/kg with BRAF inhibitors. The manufacturer stated that hazard ratios of death from the trials were used to populate the mixed treatment comparison analysis because there were different follow-up times and event numbers across trials. The manufacturer constructed a forest plot of overall survival which demonstrated that although ipilimumab plus dacarbazine was associated with a statistically significant improvement in survival compared with dacarbazine alone (hazard ratio [HR] 0.72; 95% CI 0.55 to 0.95), there were no statistically significant differences for ipilimumab plus dacarbazine compared with vemurafenib (HR 1.16) or dabrafenib (HR 1.19). Indirect comparisons using the Bucher equation showed that there was no statistically significant difference in efficacy for ipilimumab plus dacarbazine compared with vemurafenib (HR 1.16; 95% CI 0.86 to 1.56) and dabrafenib (HR 1.18; 95% CI 0.48 to 2.93). The manufacturer commented that the main difference in patients enrolled in the clinical trials was their BRAF V600 mutation status and previous treatment.

3.16 The manufacturer conducted a de novo analysis to estimate the cost effectiveness of ipilimumab compared with dacarbazine in patients who have BRAF V600 mutation-negative melanoma, and of ipilimumab compared with dacarbazine and vemurafenib in patients who have BRAF V600 mutation‑positive melanoma. The manufacturer developed a semi-Markov partitioned survival model with health states used to represent tiers of treatment, incorporating second-line active treatment and third-line best supportive care. The proportion of patients moving between health states was derived by initially calculating the number of patients who died and then adjusting the proportion of patients at each line of treatment by those who would be expected to receive palliative care (defined as 12 weeks before death). The model assumes the per-cycle risk of death to be equal for ipilimumab, dacarbazine and vemurafenib and for the patients entering palliative care to be a proportion of patients in each treatment group.

3.17 The manufacturer distributed patients across 6 health states, each associated with a utility value, and 6 time-to-death sub-health states, to capture quality of life as a function of time to death. In the base-case model, a utility decrement for people treated with ipilimumab or vemurafenib was included to account for treatment-related adverse events. Patients' health-related quality of life was estimated from time to death as an intermediate outcome because the manufacturer determined that disease progression was the most meaningful way of estimating health-related quality of life. The proportion of patients receiving each of the 4 doses needed during the induction phase of the trial was used within the model to predict how many patients would receive each dose in clinical practice. The number of patients receiving subsequent re‑inductions was estimated from the MDX010‑020 clinical trial.

3.18 To add to the manufacturer's previous assumptions about dose equivalence and that ipilimumab plus dacarbazine and ipilimumab alone were equally effective, the manufacturer also assumed that the efficacy of ipilimumab in patients with and without the BRAF V600 mutation was equivalent. The manufacturer justified this by stating that a post-hoc analysis of a subgroup containing 69 people from the CA184‑004 trial indicated there was no difference in objective response and stable disease based on the BRAF V600 mutation status. The manufacturer also presented an analysis using the pooled chemotherapy-naive dataset analysis (CA184‑004, MDX010‑08, CA184‑022 and MDX010‑020), stating that overall and progression-free survival outcomes were similar using the 2 datasets. For vemurafenib, the manufacturer stated that there was no difference between the dacarbazine arms of the CA184‑024 and BRIM‑3 trials and therefore data from the vemurafenib arm of the BRIM‑3 trial were incorporated directly into the model.

3.19 The transition to second- and third-line treatment was modelled based on progression‑free survival data, whereas overall survival data were used to model transition to death. For first-line treatment with ipilimumab or dacarbazine, a 3‑part curve fit for overall survival was used based on data from CA184‑024 and for vemurafenib a 5‑part curve fit was used based on data from the BRIM‑3 trial. For second‑line treatments, overall survival was based on first‑line survival curves but adjusted downwards to account for poorer outcomes on second‑line treatment using a constant proportional hazard derived from expected survival with second‑line ipilimumab. The duration of response to second‑line treatments was based on the number of pre‑progression life years for second‑line ipilimumab. Third‑line treatment was assumed to be best supportive care, which consisted of a proportion of patients on 'no treatment' and a proportion on commonly prescribed chemotherapy drugs, including dacarbazine. The overall survival for patients receiving third‑line best supportive care was assumed to be the same as those on first‑line treatment who had not progressed to next line of treatment. The manufacturer highlighted that for patients treated with ipilimumab, using progression-free survival overestimates the number of patients moving to second‑line treatment because ipilimumab's mode of action means it is possible for a patient's condition to initially progress and then become stable or respond to treatment. This may overestimate the cost in the ipilimumab arm because costs of first‑line treatment with ipilimumab are almost static, whereas the costs of second-line treatment depend on the duration of treatment.

3.20 The same ipilimumab patient access scheme will be in place as agreed in Ipilimumab for previously treated advanced (unresectable or metastatic) melanoma (NICE technology appraisal guidance 268; hereafter referred to as TA268). The cost of vemurafenib was presumed to be 4 packs of tablets every 4 cycles as in Vemurafenib for treating locally advanced or metastatic BRAF V600 mutation-positive malignant melanoma (NICE technology appraisal guidance 269) and included the agreed patient access scheme. The costs of dacarbazine and best supportive care were calculated based on an average height (170 cm) and weight (78.65 kg) for the patients, taken from the CA184‑024 trial. Dosing schedules for the 2 drugs were taken from the specific product characteristics. Second‑line costs for ipilimumab were also taken from the previous NICE appraisal, TA268. The administration and drug costs for second‑line vemurafenib treatment were assumed to be equal to first‑line costs. To account for the wastage incorporated in the costs for first‑line vemurafenib treatment, an additional 5.78% was added to the drug cost.

3.21 For patients who have BRAF V600 mutation-positive melanoma, base-case results indicated that vemurafenib was dominated by (that is, was more expensive and less effective than) ipilimumab because it was associated with higher costs and fewer quality-adjusted life years (QALYs). A comparison of ipilimumab with dacarbazine resulted in an incremental cost-effectiveness ratio (ICER) of £31,559 per QALY gained using the CA184‑024 results and £28,465 per QALY gained when using the pooled chemotherapy-naive data for ipilimumab. For patients who have BRAF V600 mutation-negative melanoma, the ICER for ipilimumab compared with dacarbazine was £16,958 per QALY gained using the CA184‑024 trial results and £17,866 per QALY gained using the pooled chemotherapy-naive data.

3.22 The manufacturer conducted a one-way deterministic sensitivity analysis using a tornado diagram to assess the impact of key uncertainties on the ICERs and probabilistic sensitivity analyses using 1000 simulations. Sensitivity analysis was only carried out for the BRAF V600 mutation-positive population because the ICER for ipilimumab compared with dacarbazine was higher for this group and was therefore considered the worst-case scenario. The deterministic sensitivity analysis indicated that the model was most sensitive to the parameters used to model overall survival for ipilimumab and dacarbazine, the time spent on second‑line treatment and the time spent on first-line treatment with ipilimumab compared with dacarbazine. Data from the CA184‑024 trial were used to calculate ICERs for ipilimumab compared with vemurafenib and dacarbazine. Ipilimumab dominated (that is, was less expensive and more effective than) vemurafenib and the ICER was £31,619 per QALY gained compared with dacarbazine. The manufacturer used 2 different maximum acceptable ICERs to calculate the incremental net benefit: £30,000 per QALY gained when comparing with vemurafenib and £50,000 per QALY gained when comparing with dacarbazine. The manufacturer stated, after carrying out probabilistic sensitivity analysis, that the probability of ipilimumab being cost effective when compared with dacarbazine was 96%, using £50,000 per QALY gained as the maximum acceptable ICER. There was a 40% probability of ipilimumab being cost effective if the maximum acceptable ICER was £30,000 per QALY gained. The manufacturer suggested an ICER for ipilimumab compared with dacarbazine of £49,579 per QALY gained as the most pessimistic outcome, assuming a single parameter curve fit using a log-normal distribution for overall survival, but it emphasised that single parametric curve fits were a poor fit to the data.

3.23 The ERG stated that the manufacturer had identified all relevant randomised controlled trials and that adequate trial details were included in the submission. The ERG was satisfied that the CA184‑024 trial was a large, good-quality trial but stated that it did not provide direct evidence for the effectiveness of ipilimumab 3 mg/kg monotherapy (without maintenance treatment) compared with dacarbazine, vemurafenib or dabrafenib for treating previously untreated advanced (unresectable or metastatic) melanoma. The ERG noted that the MDX010‑08 trial included treatment every 3 weeks rather than every 4 weeks as per the marketing authorisation and also stated that the trial was underpowered to detect a statistically significant difference in overall survival. The ERG stated that the pooled analysis of chemotherapy-naive patients could result in double counting because the MDX010‑08 trial was also included independently. The ERG highlighted that there were differences in performance status, disease stage, presence of brain metastases, duration of melanoma and prior immunotherapy across the 2 observational trials and the pooled analysis and, additionally, it was inappropriate to compare the results of these trials with the dacarbazine-alone arm of the CA184‑024 trial because of differences in trial design and patient characteristics.

3.24 The ERG stated that the manufacturer's assumption that 3 mg/kg and 10 mg/kg doses of ipilimumab have equivalent clinical effectiveness was not appropriate. The ERG highlighted that survival was better with ipilimumab 10 mg/kg than 3 mg/kg in the CA184‑022 trial (n=18; chemotherapy treatment‑naive patients), though this improvement was not statistically significant (HR 0.875; 95% CI 0.593 to 1.291) and the overall response rate was statistically significantly improved with ipilimumab 10 mg/kg, whereas grade 3 or 4 adverse events were more common at the higher dose. The ERG noted that although the CA184‑004 trial indicated no meaningful differences with different doses of ipilimumab, the numbers in this trial were very small (n=36; chemotherapy treatment-naive patients). In response to clarification, the manufacturer presented results of a pooled analysis that compared overall survival profiles for ipilimumab 3 mg/kg from the MDX010‑020 and CA184‑022 trials and 10 mg/kg from the CA184‑007, CA184‑008 and CA184‑022 trials for a mixed population of patients with previously treated or untreated melanoma. The ERG stated that the results of this pooled analysis suggested that a 10 mg/kg dose could be better than a 3 mg/kg dose in terms of overall survival but that methods for pooling had not been presented and it was unable to confirm the reliability of this analysis. The ERG also noted that this analysis included primarily previously treated patients. The ERG stated that the US Food and Drug Administration and the European Medicines Agency both commented on the lack of evidence for the most clinically effective dose of ipilimumab and there is currently an ongoing trial, CA184‑169, comparing 3 mg/kg and 10 mg/kg doses of ipilimumab.

3.25 The ERG also stated that the manufacturer's assumption that ipilimumab alone and ipilimumab plus dacarbazine have equivalent clinical effectiveness was not appropriate. The ERG did not agree with the manufacturer that the MDX010‑08 trial provided evidence of equivalence between ipilimumab monotherapy and ipilimumab plus dacarbazine, noting that the hazard ratio for overall survival with ipilimumab plus dacarbazine was 0.75 (95% CI 0.45 to 1.24) compared with ipilimumab alone, and highlighting that the trial included only 64 patients and was underpowered to detect a statistically significant difference in overall survival.

3.26 The ERG stated that it did not consider the indirect comparisons and mixed treatment comparisons conducted by the manufacturer to be appropriate because of different patient characteristics and BRAF status. The ERG also noted the difference in trial designs and that the difference in the mechanism of action between ipilimumab and the BRAF inhibitors resulted in a violation of the proportional hazards assumption. Therefore the ERG stated that there was no reliable clinical-effectiveness evidence for a comparison of ipilimumab with vemurafenib. Based on these concerns, the ERG stated that the survival benefit associated with ipilimumab 3 mg/kg was likely to be overestimated in the manufacturer's submission.

3.27 The ERG expressed some major concerns about the assumptions in the manufacturer's model, particularly about the relative efficacy data used in the model because of the clinical assumptions necessary (see sections 3.12 and 3.13). The ERG also highlighted that using data only from the vemurafenib arm of the BRIM‑3 trial for comparing ipilimumab with vemurafenib was inappropriate because it broke randomisation and raised concerns about the exchangeability of populations across the trials. In addition, the ERG stated that using the mixed treatment comparison to check consistency of results indicated that using the independent arm from the BRIM‑3 trial directly favoured ipilimumab.

3.28 The ERG raised concerns around the treatment sequencing approach used to structure the model, stating that the existing evidence for ipilimumab does not include a comparison of sequential use of treatments for previously untreated advanced melanoma, resulting in oversimplified assumptions. The ERG stated that the analysis and modelling conducted by the manufacturer favoured ipilimumab and that an alternative model structure based on first-line treatment only was more plausible. The ERG was unclear why the manufacturer did not attempt to use the overall survival and progression-free survival for second-line ipilimumab used in the previous ipilimumab appraisal, TA268. The ERG acknowledged the manufacturer's clarification that this would have been difficult to implement in the cohort model structure because patients progressed to second‑line treatment at different time points, but emphasised that if treatment sequencing was included, the additional complexity to model sequencing should be incorporated. When observing the manufacturer's scenario analysis, the ERG considered the 'no active second-line treatments' to be the most important because this represents the stage model for pre- and post-progression and death. The only second-line treatment is best supportive care. The resulting ICER for comparison between ipilimumab and dacarbazine increased from £31,559 to £42,449 per QALY gained and when comparing with vemurafenib, the ICER became £28,980 per QALY gained in the BRAF V600 mutation-positive population.

3.29 The ERG was satisfied with the individual treatment pathways but had concerns relating to the set of assumptions used to model survival for the different lines of treatment. Patients who received ipilimumab as first-line treatment were assumed to follow the overall survival curve from the CA184‑024 trial until progression. When the patients move to best supportive care, the assumption used in the model was that they continue to follow the ipilimumab overall survival curve from the CA184‑024 trial until they die, indicating a sustained overall survival benefit for first-line ipilimumab. Patients receiving dacarbazine as first-line treatment were assumed to follow the dacarbazine overall survival curve from the CA184‑024 trial until progression to ipilimumab. The overall survival for these patients was based on a downward adjustment of the first-line overall survival curve for ipilimumab (HR 1.21). Once patients move onto best supportive care, the assumption used in the model was that the patients followed the overall survival curve of first-line dacarbazine. The ERG stated that the manufacturer had not supplied any evidence for the assumption that patients receiving first-line ipilimumab maintained sustained benefit of overall survival in the long term whereas patients receiving ipilimumab second-line did not. The ERG commented that this approach favoured ipilimumab.

3.30 The ERG stated that the modelled treatment pathways for patients who have BRAF V600 mutation-positive melanoma demonstrated similar inconsistencies in the use of overall survival curves. When ipilimumab was provided to patients as first-line treatment, the overall survival curve from the CA184‑024 trial was used in the modelling. At the point of progression, modelled using the progression-free survival curve from the CA184‑024 trial, patients switch to vemurafenib. Patients then follow the overall survival curve for second-line vemurafenib, based on a downward adjustment of the first-line vemurafenib curve from the BRIM‑3 trial. When patients switch to third-line best supportive care, they are assumed to follow the overall survival of first-line ipilimumab without any adjustment of the curve. The ERG stated that this switch was difficult to justify and unlikely to be supported by clinical evidence.

3.31 The ERG had concerns that no direct EQ‑5D data were collected and that the Rowen algorithm may not be sufficiently generalisable to the current appraisal population. The ERG was also concerned about the progressively lower completion rates of EORTC QLQ‑C30 among surviving patients at subsequent points in time, which could reflect selection bias. The ERG requested clarification on the reasons for non-completion but this was unavailable. The ERG also noted that the utilities did not capture positive treatment effects.

3.32 The ERG explored the impact of different assumptions regarding overall survival on second- and third-line treatment on cost-effectiveness estimates. For the BRAF V600 mutation-negative population, if it was assumed that patients remain on the same overall survival curve of second-line treatment, the ICER for ipilimumab compared with dacarbazine increased from £16,958 to £18,833 per QALY gained. The ERG also carried out analyses using the overall survival curve of ipilimumab second line for best supportive care and the overall survival curve of dacarbazine first-line for best supportive care, resulting in an increase in the ICER to £40,005 per QALY gained and £56,486 per QALY gained respectively. In the BRAF V600 mutation-positive population, in the manufacturer's base case, ipilimumab dominated vemurafenib in the BRAF V600 mutation-positive population but moved to the south-west quadrant of the cost-effectiveness plane with an ICER between £27,180 and £84,980 per QALY gained. The ERG stated that this exploration highlighted the sensitivity of the manufacturer's analysis to the modelling of overall survival and emphasised that a model structure with only first‑line treatments was more appropriate.

3.33 The ERG noted that it was possible to 'turn off' the treatment sequencing to allow for direct comparison between the first-line ipilimumab and dacarbazine treatments in terms of overall survival and progression-free survival. The ERG therefore turned off the sequential use of treatments in the manufacturer's model so that it followed a conventional 3‑state cancer model with the only additional line of treatment being that of best supportive care. This followed the more conventional 3‑state cancer model. The ERG also presented an analysis assuming the same overall survival curves for patients who progress to best supportive care that resulted in an ICER of £123,676 per QALY gained for ipilimumab compared with dacarbazine. When the same comparison was carried out between ipilimumab and vemurafenib, vemurafenib dominated ipilimumab; whereas in the manufacturer's analysis, when assuming that patients did not receive any second-line treatment, the ICER was £28,980 per QALY gained.

3.34 The ERG further explored the manufacturer's assumption that ipilimumab 3 mg/kg was clinically equivalent to ipilimumab 10 mg/kg, using the pooled overall survival data provided by the manufacturer in response to clarification. The ERG estimated the implied hazard ratio for 3 mg/kg relative to 10 mg/kg by extracting data from the Kaplan‑Meier curves for both doses. The ERG stated that this adjustment increased the ICER for ipilimumab compared with dacarbazine from £16,958 per QALY gained in the manufacturer's analysis to £59,942 per QALY gained in the BRAF V600 mutation‑negative population and from £31,559 per QALY gained to £85,806 per QALY gained in the BRAF V600 mutation‑positive population. For the comparison with vemurafenib, ipilimumab was no longer less costly and more effective as in the manufacturer's base‑case analysis, but instead became less costly and less effective with a resulting ICER of £56,958 per QALY gained for vemurafenib compared with ipilimumab.

3.35 The ERG did not believe there was conclusive evidence to suggest that ipilimumab plus dacarbazine was equivalent to ipilimumab alone. The ERG found that the ICER for ipilimumab compared with dacarbazine increased from £16,958 to £73,615 per QALY gained in the BRAF V600 mutation-negative population when using a hazard ratio of 0.75 from the MDX010‑08 trial. For the comparison with vemurafenib, ipilimumab was no longer less costly and more effective as in the manufacturer's base-case analysis, with an ICER of £52,199 per QALY gained for vemurafenib compared with ipilimumab.

3.36 The ERG noticed a discrepancy in the cost per weekly cycle of ipilimumab depending on whether treatment was first-line (£1055) or second-line (£1499). When the ERG explored this price difference by incorporating second-line costs, the ICER for ipilimumab compared with dacarbazine increased from £16,958 to £25,720 per QALY gained in the BRAF V600 mutation-negative population. Ipilimumab still dominated vemurafenib in the BRAF V600 mutation-positive population. The ERG also compared the estimates of cost effectiveness with those based on utility values for pre- and post-progression as used in the previous ipilimumab appraisal, TA268. When the ERG used these utility values, the ICER increased from £16,958 to £19,320 per QALY gained for the comparison of ipilimumab with dacarbazine in the BRAF V600 mutation-negative population.

3.37 The ERG presented additional analyses exploring the cost effectiveness of ipilimumab 3 mg/kg monotherapy, based on a conventional 3‑state model that observed only first-line treatment, and also incorporating the adjusted overall survival data. This resulted in an ICER of £331,091 per QALY gained for ipilimumab compared with dacarbazine because the adjustment produced lower QALYs for ipilimumab (reduced from 2.35 to 1.56 mean QALYs). The ERG carried out a similar analysis adjusting overall survival for concomitant treatment with dacarbazine and this resulted in an ICER of £674,144 per QALY gained for ipilimumab compared with dacarbazine (reduction in QALYs from 2.35 to 1.50 for ipilimumab). When the ERG compared ipilimumab with vemurafenib, vemurafenib dominated ipilimumab in both scenarios.

Manufacturer's response to consultation

3.38 In response to consultation, the manufacturer presented updated analyses using data from the MDX010‑20 and BRIM‑3 trials. The manufacturer adjusted the overall survival curve for previously treated patients receiving 3 mg/kg ipilimumab in the MDX010‑20 trial to predict overall survival for previously untreated patients receiving 3 mg/kg ipilimumab. The manufacturer used ipilimumab overall survival and progression-free survival data from the MDX010‑20 trial and overall survival and progression-free survival data for dacarbazine from the CA184‑024 trial. The baseline characteristics of patients in the 2 trials were similar, although patients in the MDX010‑20 trial generally had greater tumour burden. Therefore the manufacturer used a Cox proportional hazards model fitted to data from the 3 mg/kg ipilimumab arm of the MDX010‑20 trial to predict overall survival for a group of patients with the baseline characteristics of those in the CA184‑024 trial.

3.39 Five prognostic factors – gender, ECOG performance status, visceral disease status, brain metastases and lactate dehydrogenase levels – were modelled to produce an average survival curve using weightings from the CA184‑024 trial. The resulting overall survival curve represented predicted overall survival for the 3 mg/kg ipilimumab dose in the first-line setting. The updated overall survival Kaplan–Meier data were applied directly into the model and the curve extrapolated based on evidence from the CA184‑024 trial. In the updated base‑case analysis, the manufacturer used the adjusted overall survival data for the first 4.5 years and then used the overall survival curve from the CA184‑024 trial. The manufacturer considered this appropriate because the proportion of patients alive at 4.5 years was equal to that for the adjusted overall survival data. The manufacturer incorporated the adjusted overall survival data directly into its updated model for the base-case analysis. The higher number of doses received by patients in the MDX010‑20 trial was also incorporated in the base-case analysis. No adjustment was carried out for the progression-free survival curve because it was considered unnecessary.

3.40 In the manufacturer's original submission, it assumed that there was no difference in the overall survival for patients between the dacarbazine control arms of the CA184‑024 and BRIM‑3 trials. The manufacturer reported in its response to consultation that the latest follow‑up data from the BRIM‑3 trial show differences in the overall survival curves for the 2 dacarbazine patient populations. A proportion of patients (more than 20%) in the BRIM‑3 trial received ipilimumab after vemurafenib or dacarbazine, so survival for vemurafenib was potentially overestimated. The manufacturer updated the vemurafenib overall survival curve from the BRIM‑3 trial to remove the effect of patients who received ipilimumab second-line. No adjustments had to be made for patients in the MDX010‑20 and CA184‑024 trials because patients were not allowed to cross over from the dacarbazine group to the ipilimumab group, and no patients received vemurafenib after ipilimumab or dacarbazine. The overall survival curve for vemurafenib was also further adjusted to account for different patient baseline characteristics in the BRIM‑3 and CA184‑024 trials.

3.41 The manufacturer presented updated ICERs using the 3‑state model and adjusted overall survival curves. In the base case, the manufacturer calculated an ICER of £47,899 per QALY gained for ipilimumab compared with dacarbazine and an ICER of £28,642 per QALY gained for ipilimumab compared with vemurafenib. The manufacturer considered that the ICER presented for ipilimumab compared with vemurafenib may be a conservative estimate because the benefits of vemurafenib over dacarbazine looked unrealistically high, in light of further data from the BRIM-3 trial. The manufacturer addressed this in its scenario analysis (by assuming that the proportion of patients alive in the vemurafenib group was equal to dacarbazine at and after 30 months from the start of the trial) and the ICER for ipilimumab compared with vemurafenib was reduced to £12,967 per QALY gained.

3.42 The manufacturer also carried out 2 modifications to the sequential model to enhance its validity. Firstly, the mortality hazard for third-line treatment was assumed to be the same as for second-line ipilimumab. The second modification involved the use of survival curves from the MDX010‑20 trial rather than the use of hazard ratios to estimate efficacy of second-line ipilimumab. Using the sequential model decreased the ICER for ipilimumab compared with dacarbazine from £47,899 per QALY gained to £41,016 per QALY gained. The manufacturer also indicated that ipilimumab may be used by clinicians to treat patients who have BRAF V600 mutation-positive melanoma if they had favourable prognostic characteristics but only observational studies were available to support this assumption. The lack of data meant that the manufacturer did not present a sequential comparison of ipilimumab and vemurafenib.

3.43 The manufacturer also calculated a 'worst-case scenario' ICER for the treatment of patients who have previously untreated melanoma using unadjusted data from the MDX010‑20 trial. For the ICER calculation, the overall survival and progression-free survival were taken from the model submitted in the previous appraisal, TA268. The efficacy outcomes for dacarbazine were from the CA184‑024 trial and for vemurafenib were from the BRIM‑3 trial because these are likely to overestimate the efficacy of the treatments compared with ipilimumab. The ICER for ipilimumab compared with dacarbazine was £58,593 per QALY gained and vemurafenib dominated ipilimumab. The manufacturer also applied an adjustment of the MDX010‑20 overall survival curve as a hazard ratio and this produced an ICER of £37,575 per QALY gained for ipilimumab compared with dacarbazine and £15,592 per QALY gained for ipilimumab compared with vemurafenib. The manufacturer also provided an analysis to demonstrate that the proportion of patients alive with vemurafenib and dacarbazine at 30 months was equal. This scenario analysis produced an ICER of £12,967 per QALY gained when comparing ipilimumab and vemurafenib.

Evidence Review Group's response to manufacturer's consultation comments

3.44 The ERG questioned the prognostic factors used by the manufacturer to adjust the overall survival curve from the MDX010‑20 trial to reflect previously untreated melanoma. The ERG noted that baseline disease stage was excluded from the model despite differences between trials and, although the ERG agreed with the use of a Cox proportional model, it could not verify the overall survival curve for ipilimumab or vemurafenib because the ERG did not have the individual patient‑level data to fit the model. The ERG also had reservations about the use of this corrected curve in the model, with the adjusted overall survival curve directly modelled for the first 4.5 years and then an extrapolation being used beyond this point, because this was not consistent with the previous appraisal, TA268. The ERG was also unsure why a cut‑off of 4.5 years was used and would have preferred the manufacturer to explore alternative curve fits and explain whether adjustment for censoring had been carried out. There was also concern about the uncertainty in the overall survival curve beyond 3 years because, in the MDX010‑20 trial, only 37 patients were still at risk at the 3‑year point. The ERG would have preferred to see the use of parametric tools in developing the adjustments and analysis that did not break randomisation when comparing the comparator treatment arms.

3.45 The ERG considered the manufacturer's use of a 3‑state model and survival curves from the MDX010‑20 trial directly in the model to be appropriate. No details were provided of how the manufacturer had implemented the curves within the model and therefore the approach could not be verified by the ERG. There was also concern about the switching of overall survival curves between lines of treatment. The ERG noted that the manufacturer had partly addressed this issue by assuming that the hazard mortality for best supportive care was the same as second-line ipilimumab rather than first-line, as in the manufacturer's original submission. The ERG commented that the manufacturer's approach only addressed the observed differences between trial populations and did not control for unobservable differences between patients.

3.46 Full details of all the evidence can be found in How this guidance was produced.