3 Evidence

The appraisal committee (section 4) considered evidence submitted by Celgene and a review of this submission by the evidence review group (ERG). See the committee papers for full details of the evidence.

Clinical effectiveness

3.1 The company presented evidence from 1 randomised controlled trial, AZA‑AML‑001. This was an international, multicentre, controlled, phase III study with an open-label, parallel-group design. It included 488 adults of 65 years and older who had newly diagnosed acute myeloid leukaemia with more than 30% bone marrow blasts and an eastern cooperative oncology group (ECOG) performance status of 0 to 2 with adequate organ function. Before randomisation patients were screened and assigned to one of 3 conventional care regimens: intensive chemotherapy with anthracycline and cytarabine plus best supportive care; low-dose chemotherapy with cytarabine plus best supportive care; and best supportive care only. Patients were then randomised to have either azacitidine (n=241) or the preselected conventional care regimen (n=247).

3.2 The primary outcome was overall survival. AZA‑AML‑001 was powered to detect a difference in overall survival between azacitidine and the combined conventional care regimen, which comprised intensive chemotherapy with anthracycline and cytarabine plus best supportive care, low-dose chemotherapy with cytarabine plus best supportive care and best supportive care alone. The secondary outcomes included 1‑year overall survival rate, overall remission rate, duration of remission, cytogenetic complete remission rate, partial remission, stable disease, safety and tolerability, patient-reported quality of life outcomes (assessed using the European Organization for Research and Treatment of Cancer [EORTC‑QLQ‑30] questionnaire), measures of healthcare resource use and transfusion status. Patients were not allowed to switch treatments during the study, but further treatments were allowed after the study drug was stopped. After stopping, 67 patients in the azacitidine arm and 75 patients in the conventional care arm had further treatments.

3.3 Azacitidine was associated with improvements in overall survival compared with the combined conventional care regimen. However, the intention-to-treat analysis showed that azacitidine was not statistically significantly superior to the combined conventional care regimen (see table 1).

Table 1 Clinical effectiveness outcomes in AZA‑AML‑001; summary of overall survival in the intention-to-treat population

Outcome	Azacitidine (n=241)	CCR (n=247)
Median overall survival (95% CI), months	10.4 (8.0 to 12.7)	6.5 (5.0 to 8.6)
Difference (95% CI), months	3.8 (1.0 to 6.5)
Hazard ratio [AZA:CCR] (95% CI)	0.85 (0.69 to 1.03)
Stratified log-rank test: p-value	0.1009
Hazard ratio [AZA:CCR] (95% CI)	0.84 (0.69 to 1.02)
Unstratified log-rank test: p-value	0.0829
Abbreviations: AZA, azacitidine; CCR, conventional care regimen; CI, confidence interval.

3.4 Secondary outcomes, including measures of haematological response, duration of remission and remission-free survival, were similar between azacitidine and the combined conventional care regimen, with no statistically significant differences between treatments. Azacitidine and the combined conventional care regimen were associated with general improvements in health-related quality of life in the 4 prespecified QLQ-C30 domains of fatigue, dyspnoea, global health status and physical functioning. Statistical analyses were not presented in the submission for health-related quality of life.

3.5 In response to the use of further treatments after stopping the study drug in the clinical trial, the company presented a series of sensitivity analyses that censored patients at the date of first subsequent therapy (see table 2). The company indicated that these results suggested that the subsequent therapies may be confounding the treatment effect of azacitidine.

Table 2 Summary of sensitivity analyses on overall survival (intention-to-treat population)

Outcome	Azacitidine (n=241)	CCR (n=247)
Median overall survival (95% CI), months	12.1 (9.2 to 14.2)	6.9 (5.1 to 9.6)
Hazard ratio [AZA:CCR] (95% CI)	0.76 (0.60 to 0.96)
Stratified log-rank test: p-value	0.0190
Hazard ratio [AZA:CCR] (95% CI)	0.75 (0.59 to 0.95)
Stratified log-rank test: p-value	0.0147
Abbreviations: AZA, azacitidine; CCR, conventional care regimen; CI, confidence interval.

3.6 The company stated that there was heterogeneity in the study population as well as possible confounding in the results because of subsequent therapies. It did post-hoc analyses using Cox proportional hazards, inverse probability of censoring weighted analysis and regression-based imputation to estimate the effect on overall survival when baseline covariates and subsequent treatment were adjusted for. One inverse probability of censoring weighted analysis adjusted for any subsequent treatments in both trial arms and another adjusted only for the use of azacitidine in the conventional care regimen arm. The latter analysis was presented as academic in confidence and cannot be included here. Using these methods, azacitidine was shown to statistically significantly improve overall survival compared with the conventional care regimen (see table 3).

Table 3 Post-hoc overall survival estimates adjusted for baseline characteristics and/or subsequent therapy

Estimation method	HR (AZA vs CCR)	95% CI for HR	p-value
Cox proportional hazards
Adjusted for subsequent therapy	0.75	0.59 to 0.94	0.0130
Adjusted for baseline characteristics	0.80	0.66 to 0.99	0.0355
Adjusted for subsequent therapy and baseline characteristics	0.69	0.54 to 0.88	0.0027
Inverse probability of censoring weighted Cox proportional hazards models – adjusted for subsequent therapy in both treatment arms
Unadjusted for baseline characteristics	0.77	0.61 to 0.98	0.0310
Adjusted for baseline characteristics	0.71	0.56 to 0.90	0.0047
Regression-based imputation analysis adjusting for subsequent therapy
Adjusted for subsequent therapy	0.76	0.62 to 0.93	0.007
Abbreviations: AZA, azacitidine; CCR, conventional care regimen; CI, confidence interval; HR, hazard ratio.

3.7 The company presented an exploratory analysis of azacitidine compared with the individual components of the conventional care regimen. AZA‑AML‑001 was not powered to detect differences between azacitidine and individual treatments.

Median overall survival was 5.8 months in the azacitidine group (95% confidence interval [CI] 3.6 to 9.7, n=44) compared with 3.7 months in the best supportive care group (95% CI 2.8 to 5.7, n=45). There was a 40% reduction in the risk of death for patients having azacitidine (hazard ratio [HR] 0.60; 95% CI 0.38 to 0.95, unstratified log rank test p=0.0288).
Median overall survival was 11.2 months in the azacitidine group (95% CI 8.8 to 13.4, n=154) compared with 6.4 months in the low-dose chemotherapy group (95% CI 4.8 to 9.1, n=158). There was a 10% reduction in the risk of death for patients having azacitidine (HR 0.90; 95% CI 0.70 to 1.16, unstratified log rank test p=0.4270).
Median overall survival was 13.3 months in the azacitidine group (95% CI 7.2 to 19.9, n=43) compared with 12.2 months in the intensive chemotherapy group (95% CI 7.5 to 15.1, n=44). There was a 15% reduction in the risk of death for patients having azacitidine (HR 0.85; 95% CI 0.52 to 1.38, unstratified log rank test p=0.5032).

3.8 The company presented results for event-free survival and relapse-free survival for azacitidine compared with the individual components of the conventional care regimen.

Event-free survival was 4.5 months in the azacitidine group (n=44) compared with 3.1 months in the best supportive care group (n=45). There was a 33% reduction in the risk of an event for patients in the azacitidine group (HR 0.67; 95% CI 0.43 to 1.04, p=0.0756). Relapse-free survival results were not presented for this comparison.
Event-free survival was 7.3 months in the azacitidine group (n=154) compared with 4.8 months in the low-dose chemotherapy group (n=158). There was an 11% reduction in the risk of an event for patients in the azacitidine group (HR 0.89; 95% CI 0.70 to 1.13, p=0.3563). Relapse-free survival was 8.6 months in the azacitidine group (n=154) compared with 9.9 months in the low-dose chemotherapy group (n=158). There was an 11% reduction in the risk of relapse for patients in the low-dose chemotherapy group (HR 1.11; 95% CI 0.68 to 1.81, p=0.6638).
Event-free survival was 8.1 months in the azacitidine group (n=43) compared with 9.7 months in the intensive chemotherapy group (n=44). There was a 2% reduction in the risk of an event for patients in the intensive chemotherapy group (HR 1.02; 95% CI 0.64 to 1.63, p=0.9196). Relapse-free survival was 10.8 months in the azacitidine group (n=43) compared with 12.1 months in the intensive chemotherapy group (n=44). There was a 21% decrease in the risk of relapse for patients in the intensive chemotherapy group (HR 1.21; 95% CI 0.58 to 2.51, p=0.6135).

3.9 Subgroup analyses for patients with a poor cytogenetic risk and patients with myelodysplastic syndrome-related changes were included in the submission. Median overall survival for patients with myelodysplastic syndrome-related changes was 12.7 months in the azacitidine group compared with 6.3 months in the conventional care regimen group (HR 0.69; 95% CI 0.48 to 0.98, p=0.357). The median overall survival for people with a baseline cytogenetic risk rated as poor was 6.4 months in the azacitidine group compared with 3.2 months in the conventional care regimen group (HR 0.68; 95% CI 0.68 to 0.94, p=0.0185).

3.10 The company reported that azacitidine was generally well tolerated in AZA‑AML‑001, with more than 50% of patients in the azacitidine treatment group having 6 or more treatment cycles and one-third having 12 or more cycles. The most common haematological treatment-related adverse events with azacitidine were febrile neutropenia, neutropenia and thrombocytopenia. All frequent haematological adverse events were generally lower with azacitidine than with other conventional care regimen treatments. The most common non-haematological treatment-related adverse events were constipation, nausea and diarrhoea. In general, non-haematological adverse events occurred more frequently in the azacitidine group compared with the conventional care regimen treatments. The most common serious adverse events reported in the azacitidine group included febrile neutropenia, pneumonia and pyrexia.

Cost effectiveness

3.11 The company presented a semi-Markov model based on 4 states: remission, non-remission, relapsed or progressive disease, and death. The model used a cycle length of 4 weeks with a lifetime time horizon of 10 years. In the base case, the company compared azacitidine with the combined conventional care regimen. In the combined conventional care regimen, 18% of patients had intensive chemotherapy, 64% had low-dose chemotherapy and 18% had best supportive care. A comparison with the individual conventional care regimen treatments was presented in a scenario analysis. The model perspective was the NHS and personal social services, and costs and benefits were discounted at a rate of 3.5% per year.

3.12 The company estimated the proportion of people in each health state for every 4-week cycle using relapse-free survival, progression-free survival and overall survival curves. The model included subgroup analysis for patients with cytogenetic risk factors and myelodysplasia-related changes. The company identified extrapolation models based on whether the proportional hazards assumption was met, goodness of fit, clinical plausibility, and internal and external validation. For the base case, overall survival, progression-free survival and relapse-free survival were extrapolated using the exponential, Gompertz and Weibull distributions respectively.

3.13 Health-related quality of life was incorporated into the model by applying utility scores to each health state. Utilities were mapped from trial-based disease-specific EORTC-QLQ-C30 data using published algorithms. Two mapping algorithms were incorporated in the model, one from Proskorovsky et al. (2014) which was used for the base case and the other from McKenzie and Van der Pol (2009), used for a scenario analysis. The model also included the effect on quality of life of adverse effects, by applying utility decreases (decrements) for each effect of severity grade 3 or above.

3.14 The model incorporated costs in each health state, including costs associated with acute myeloid leukaemia treatment, management of adverse events (events of severity grade 3 or above), transfusion costs, best supportive care monitoring costs, tests to monitor disease and care at the end of life. Treatment costs included drug acquisition, administration and dispensing for azacitidine and the conventional care regimens.

3.15 In the company's base case, the incremental cost-effectiveness ratio (ICER) for azacitidine compared with the combined conventional care regimen was £20,648 per quality-adjusted life year (QALY) gained. In the probabilistic sensitivity analysis the ICER was £17,423 per QALY gained. The incremental costs and QALYs were marked commercial in confidence and cannot be included here.

3.16 The company's deterministic sensitivity analysis showed that the model results were most sensitive to the administration costs associated with the conventional care regimen, the hazard ratio for overall survival and the conventional care regimen remission rates.

3.17 The company presented scenario analyses to explore the effect of assumptions about survival modelling, treatment sequences and the proportions of patients having each of the conventional treatments. These scenario analyses demonstrated that the ICERs were most sensitive to changes in the proportions of patients assumed to have each of the conventional treatments, the use of the individual treatment regimens rather than the combined conventional care regimen, and the use of the censor at switch data to calculate overall survival.

ERG's comments

3.18 The ERG stated that there were limitations to the company's systematic review searches and inclusion criteria. However, it concluded that the company did not appear to have missed any evidence. The ERG noted that AZA‑AML‑001 was well designed and well conducted. It also stated that although unavoidable, the open-label design of the trial increased the risk of bias. It noted some limitations in this trial – in particular, the primary end point was a comparison of overall survival for patients randomised to azacitidine and patients randomised to the combined conventional care regimen. The trial was underpowered to compare azacitidine with each of the individual conventional care regimens. The ERG also commented that the use of subsequent therapies after stopping study treatments resulted in confounded estimates for the primary efficacy end point and other end points. Additionally, statistical analyses of time-to-event outcomes relied on the proportional hazards assumption, which the ERG considered not to be justified.

3.19 The ERG commented on the company's analyses that adjusted overall survival as a result of subsequent therapy. The ERG noted that the submission lacked clarity about which treatments the analyses had adjusted for. The ERG noted that the company presented inverse probability of censoring weighted (IPCW) analysis, in which both trial arms were adjusted for treatment switching, and that this appeared to adjust for any treatment switching. A further IPCW analysis was also presented in which only subsequent azacitidine use in the conventional care regimen arm was adjusted for. The ERG stated that the analysis in which both arms were adjusted was more appropriate when the mix of subsequent treatments did not reflect that used in clinical practice. The ERG also stated that the company misinterpreted the NICE decision support unit technical support document 16, which outlines the appropriate methods of adjustment when treatments are switched. The ERG noted that the IPCW analysis relied on assumptions that it could not assess fully from the available clinical trial data. The ERG commented on the 3 Cox proportional hazards models of survival. It stated that the results of the models were all susceptible to bias. The treatment effect in the model that did not adjust for subsequent treatment was likely to be biased because of subsequent treatment use. However, the adjustments made in the models for subsequent treatments assumed that prognoses are the same for both people who switch and people who do not. The adjustments conversely suggest that prognoses for these groups of people are different but evenly distributed across arms, and that subsequent treatments have the same average effect across arms. The ERG stated that in general the more sophisticated post-hoc adjustment methods appeared to make little difference compared with the sensitivity analyses that used simpler censoring at switch methods.

3.20 The ERG reviewed the company's economic model, and commented that it was transparent and simple. It did however note that some states were too broadly defined to capture important differences in costs and quality of life between the treatments being compared. The ERG commented that the main limitation of the model structure was the assumption that no subsequent active treatment was given after azacitidine or the combined conventional care regimen. It noted that this is inconsistent with AZA‑AML‑001, in which 29% of patients had active second-line treatment. Advice from clinical experts suggests that active second-line treatment is considered for some patients in the NHS.

3.21 The ERG identified 4 key areas of concern in the company's economic modelling, extrapolation of key outcomes and health resource use, including incorrect costs and treatment cycles:

The model assumed proportional hazards for all time-to-event outcomes, even though this was not supported for overall survival and relapse-free survival by the results from AZA‑AML‑001.
Overall survival in the azacitidine arm was not adjusted for subsequent active treatment, resulting in an inconsistency between the modelled health outcomes and costs, because only the costs of best supportive care were modelled following azacitidine.
There were significant differences in the costs associated with the relapsed and progressive disease state between the azacitidine and conventional care regimen arms, even though patients in both arms were expected to be receiving best supportive care at this point. The ERG noted that the biggest difference was in the number of inpatient days in the relapsed and progressive disease state, which were 1.73 for azacitidine and 2.61 for the conventional care regimen. The effect of this was that cost differences accumulated at a rate of £628 per month despite all patients having best supportive care.
The mean number of treatment cycles in the model did not reflect the mean number of treatment cycles in AZA‑AML‑001. In the azacitidine arm the mean number of treatment cycles was 5.6 instead of 8.8. In the conventional care regimen arm, intensive chemotherapy was calculated as 2.61 instead of 2.00 cycles (initiation and consolidation), and low-dose chemotherapy was calculated as 4.4 when estimating drug acquisition costs and 5.3 when calculating the costs of drug administration, tests and transfusion, instead of 6.10.

The ERG also identified issues in relation to health-related quality of life estimates and costs of adverse events. However, the ERG considered that these issues had only a minor effect on the results and were secondary to the other issues identified.

ERG's exploratory analysis

3.22 The ERG identified 12 implementation errors in the company model of which 9 affected the base-case analysis. They mainly related to the formula used to calculate health care resource use, but also to the extrapolation of outcomes. The amendments that increased the ICER the most were:

In the conventional care regimen arm, patients receiving best supportive care incurred drug administration costs in the remission and non-remission states. However, for other active treatments the costs of administering best supportive care were not included after stopping treatment until relapse or progression.
In the azacitidine and conventional care regimen arms, costs of tests and transfusions were not modelled for patients in the relapsed or progressive disease state.
In the azacitidine and conventional care regimen arms, drug administration, monitoring tests and transfusion costs were double-counted during the first model cycle.

Conversely one of the errors overestimated the base-case ICER. In the azacitidine and conventional care regimen arms, the formula used to calculate the costs of drug administration, monitoring tests and transfusions for patients in the non-remission (stable disease) state was incorrect. Amending the 9 implementation errors increased the ICER in the base-case analysis from £20,648 to £62,518 per QALY gained.

3.23 The ERG then made a series of changes to the parameter values to reflect current UK practice and to make the model logical. The effect of each of the individual changes to the corrected base-case ICER of £62,518 per QALY gained is shown below.

Calibrating the number of treatment cycles

The mean number of treatment cycles was set to match the mean number of cycles in AZA‑AML‑001. This increased the ICER to £131,698 per QALY gained.

Costs of relapsed and progressive disease

The costs of best supportive care for relapsed and progressive disease were set to be the same in the azacitidine and conventional care regimen arms. This increased the ICER to £159,352 per QALY gained.

Adjusting overall survival in both arms for subsequent active treatment

The method of modelling overall survival was changed to censoring for treatment switching in both arms. Because of the model coding, modelling of relapse-free and progression-free survival also switched to being censored for treatment switching in both arms. The effect of this analysis was to reduce the ICER to £47,482 per QALY gained.

Fitting separate parametric survival curves to relapse-free survival and progression-free survival in each arm

The parametric proportional hazards progression-free survival curves were replaced by Kaplan–Meier curves. This increased the ICER to £75,471 per QALY gained.
The relapse-free survival curves were replaced by Kaplan–Meier curves. This had little impact on the ICER (£63,569 per QALY gained).

Adjusting overall survival for baseline covariates

Overall survival was adjusted for treatment switching (censoring at switch in both arms) and baseline covariates. This increased the ICER to £65,188 per QALY gained. The method of producing this analysis does not affect relapse-free and progression-free survival and so azacitidine patients spend longer in the progressive disease model state with high costs and low utility.

The cumulative effect of all these changes was to increase the ICER to £273,308 per QALY gained.

3.24 The ERG did some exploratory analyses using the individual conventional care regimen treatments. It stated that for progression-free survival and relapse-free survival outcomes, the sample sizes make subgroup-specific time-to-event data highly unreliable. In the exploratory analyses subgroup-specific differences in overall survival outcomes were allowed using censor-at-switch data, while keeping common progression-free survival and relapse-free survival curves across the 3 subgroups. These exploratory analyses with the ERG changes (but without controlling for baseline covariates) produced ICERs above £100,000 per QALY gained for each of the individual conventional care regimens.