The Appraisal Committee considered evidence submitted by the manufacturer of mifamurtide and a review of this submission by the Evidence Review Group (ERG).
In the submission, the manufacturer compared mifamurtide as an add-on treatment to postoperative multi-agent adjuvant chemotherapy (3- or 4-agent adjuvant chemotherapy using high-dose methotrexate, doxorubicin and cisplatin with or without ifosfamide) with postoperative multi-agent adjuvant chemotherapy (3- or 4-agent) alone in patients with high-grade, resectable, non-metastatic osteosarcoma.
The evidence for the efficacy of mifamurtide in the manufacturer's submission was obtained from 1 multicentre, open-label randomised controlled trial, the Intergroup study 0133 (INT-0133). Most of the patients who participated in INT-0133 (n=678) were recruited in the USA. They received 10 weeks of neoadjuvant induction therapy with either chemotherapy regimen A (methotrexate, doxorubicin and cisplatin) or chemotherapy regimen B (methotrexate, doxorubicin, cisplatin and ifosfamide) before surgical resection of their tumour. Surgical resection was performed during weeks 10 to 11, when patients were not receiving chemotherapy. Adjuvant therapy was scheduled to begin at week 12 when patients received 1 of 4 regimens: regimen A, regimen A+ (regimen A with the addition of mifamurtide), regimen B, or regimen B+ (regimen B with the addition of mifamurtide). Using a 2 by 2 factorial design, the study compared mifamurtide plus multi-agent chemotherapy (regimens A+ and B+) with multi-agent chemotherapy alone (regimens A and B). Similarly, the study assessed the efficacy of ifosfamide (regimens B and B+ compared with A and A+). The primary endpoint was overall survival. However, the study was powered for the first planned analysis of the intermediate endpoint, which was disease-free survival (that is, time to progression or death).
The patients in the study were under 30 years of age with a new diagnosis of malignant high-grade osteosarcoma. Exclusion criteria included metastatic disease or unresectable primary disease, low-grade osteosarcoma, parosteal or periosteal sarcoma, radiation-induced sarcoma or osteosarcoma arising in premalignant bony lesions, or previous chemotherapy or radiation therapy.
The manufacturer presented analyses based on 3 datasets. The clinical study report presented data collected up to June 2003 (2003 dataset), and August 2006 (2006 dataset); an addendum provided the updated findings based on data to March 2007 (2007 dataset). Intention-to-treat (ITT) analyses were carried out on all 3 datasets. Both the manufacturer and the ERG considered the 2007 dataset to be the most up-to-date and comprehensive. Therefore, only the 2007 dataset is referred to in this document. The overall survival data in INT-0133 showed that after a median follow-up of 7.9 years, adding mifamurtide to chemotherapy (regimens A+ and B+ combined) statistically significantly improved overall survival compared with chemotherapy alone (regimens A and B combined) with an overall survival of 71% in the control arm (chemotherapy alone) and 78% in the mifamurtide arm (chemotherapy plus mifamurtide). For the ITT population, the hazard ratio for death was 0.72 (95% confidence interval [CI] 0.53 to 0.97). However, adding mifamurtide to chemotherapy (regimens A+ and B+ combined) did not statistically significantly increase disease-free survival compared with chemotherapy alone (regimens A and B combined). For the ITT population, the hazard ratio for disease-free survival was 0.78 (95% CI 0.61 to 1.01).
The manufacturer presented a number of post hoc subgroup analyses for the dataset combining regimens A and B. These analyses showed consistent increases in overall survival with mifamurtide plus chemotherapy compared with chemotherapy alone across a broad range of demographic factors (age, gender, ethnicity, study site and geographic location) and prognostic factors (tumour size, lactate dehydrogenase level, alkaline phosphatase level, cooperative study group and background chemotherapy).
The ERG requested additional post hoc analyses for both overall and disease-free survival comparing individual mifamurtide-containing regimens (regimen A+ or B+) with individual regimens not containing mifamurtide (regimen A or B). The analyses that compared mifamurtide plus 3-agent chemotherapy (regimen A+) with the chemotherapy regimen most commonly used in UK clinical practice (regimen A) gave non-significant increases in overall survival (hazard ratio for death 0.75; 95% CI 0.49 to 1.16) and disease-free survival (hazard ratio for progression 0.96; 95% CI 0.67 to 1.38) for regimen A+. For regimen B+ compared with 4-agent chemotherapy regimen B (both including ifosfamide), there was no significant improvement in overall survival (hazard ratio 0.68; 95% CI 0.44 to 1.05) but a significant improvement in disease-free survival for regimen B+ (hazard ratio 0.63; 95% CI 0.44 to 0.91).
In INT-0133, only severe adverse events (grade 3 or 4) were recorded. With the exception of hearing loss, the number of adverse events was similar in patients receiving mifamurtide plus multi-agent chemotherapy (regimens A+ and B+ combined) compared with multi-agent chemotherapy alone (regimens A and B combined). Adding mifamurtide to multi-agent chemotherapy significantly increased the incidence of objective hearing loss (11.5% with mifamurtide versus 7.1% without; p=0.047) and subjective hearing loss (3.6% with mifamurtide versus 0.6% without; p=0.007). The post hoc subgroup analyses by treatment regimen suggested that the increased incidence of hearing problems occurred only in those treated with 3-agent chemotherapy plus mifamurtide (regimen A+).
Additional data from phase 1 and 2 studies of over 700 patients suggested that the most common adverse events in patients and healthy volunteers treated with mifamurtide alone were fever, chills, fatigue, headache, nausea or vomiting, myalgia and tachycardia, hypotension, hypertension and dyspnoea. Chills and fever were reported as mild to moderate.
In INT-0133, the rates of discontinuation were higher in both mifamurtide groups (22% for regimen A+ and 30% for regimen B+) than in the groups not receiving mifamurtide (13% for regimen A and 17% for regimen B). The manufacturer stated that most of the withdrawals were not caused by adverse events that required clinically significant intervention. The manufacturer also stated that the adverse events were not life threatening, and did not require mifamurtide to be stopped. The manufacturer assumed that many patients, or their parents, decided to withdraw from mifamurtide treatment because it was an investigational drug of unproven benefit and was uncomfortable or inconvenient (no further details were provided by the manufacturer) when added to existing multi-agent chemotherapy.
The manufacturer presented an economic model of the cost effectiveness of adding mifamurtide to 3- and 4-agent chemotherapy regimens combining cisplatin, doxorubicin and methotrexate with or without ifosfamide. The economic model had 6 health states. These were: disease free (start state), disease progression (optional start state), recurrence, disease free post recurrence, disease progression post recurrence, and death. The model had a cycle length of 6 months and a time horizon of 60 years. The manufacturer assumed that patients in the disease-free health state at 12.25 years had a mortality rate equivalent to the general population. Patients in the disease-free post recurrence state were assumed to have a mortality rate dependent on the time to recurrence, which was derived from a study by Ferrari et al. (2003). For patients who had recurrence within 2 years, the 6-monthly mortality rate was 14.87% and for those who had recurrence after 2 years, the 6-monthly mortality rate was 4.98%.
The transition probabilities used in the deterministic base case were derived from INT-0133 for 604 patients who entered the adjuvant phase, while the post-recurrence estimates were mostly derived from the literature, except when death was recorded as an event post recurrence.
The number of mifamurtide doses to be administered to each patient was assumed to be 38.4, which was the average number of mifamurtide doses administered in INT-0133. The utility values used in the economic model were taken from a study using the EQ-5D in 22 patients from INT-0133 (for the recurrence health state), and a review by the manufacturer of utility values used in other NICE technology appraisals (for all other health states), including treatments of colon, colorectal, renal cell, and prostate cancer, myeloid leukaemia and glioma. The utility values used in the model were: 0.39 for disease progression, 0.85 for patients who remained disease free, 0.61 for recurrence, 0.85 for patients who were disease free post recurrence, 0.39 for disease progression post recurrence, and 0.00 for death. The manufacturer's submission only included adverse events considered clinically relevant (such as those associated with infusion) in the base-case analyses. From INT-0133, hearing loss was identified as the main adverse event for mifamurtide. A decrease in utility value associated with this adverse event was not included in the model because it was considered to be an anomaly of the data; hearing loss is associated with cisplatin and the number of additional cases seen in 1 of the mifamurtide arms was within the reported range for cisplatin-related hearing loss. An 18% decrease in utility value for hearing loss was explored in sensitivity analyses, based on data derived from 1 study found in the manufacturer's Medline search on hearing loss in people with cancer.
The economic model included the following costs: adjuvant chemotherapy (cisplatin, doxorubicin, ifosfamide and methotrexate) with or without mifamurtide, treating adverse events during the maintenance phase, routine monitoring, diagnostic tests, surgery, and second-line chemotherapy for disease progression (ifosfamide and etoposide). Costs and resource utilisation information were taken from NHS reference costs 2007 to 2008. Information on healthcare resource use was not collected in the study and the costs of these resources were therefore estimated from information provided by clinical specialists.
The ERG questioned whether using all the INT-0133 data from the 3- and 4-agent chemotherapy regimens (that is regimens A+ and B+ combined and regimens A and B combined) was appropriate. The ERG noted that the absence of an interaction between ifosfamide and mifamurtide was crucial to the validity of the manufacturer's statistical approach. However, the ERG highlighted a potential interaction between ifosfamide and mifamurtide in the INT-0133 results. The ERG noted that there were potentially significant differences in clinical effectiveness among the 4 groups, as demonstrated by the analyses that compared individual mifamurtide regimens (A+ or B+) with regimens without mifamurtide (A or B). This led to a high degree of variability in the cost-effectiveness results for the groups with and without mifamurtide. The ERG stated that if it was accepted that there was no such interaction, then the results could indeed be considered to represent 2 separate trials, of mifamurtide and of ifosfamide for osteosarcoma, which would indicate a high degree of uncertainty in the true cost effectiveness of mifamurtide.
The ERG considered that the model lacked face validity because the modelled survival rates at 6 years (83% with mifamurtide and 77% without mifamurtide) were higher than those seen in the clinical trial (78% with mifamurtide and 70% without mifamurtide). It was unclear what was driving this difference in estimated survival. If, for example, it was simply the length of the time cycle in the Markov model, then a more appropriate time cycle should have been chosen in the model. The ERG stated that although this lack of face validity increases the uncertainty in the results of the economic analysis, it is unclear what effect this would have on the incremental cost-effectiveness ratio (ICER) if the mortality rates seen in INT-0133 were accurately replicated in the model.
The results of the economic analysis included in the manufacturer's original submission, which incorporated the original patient access scheme, have been replaced by updated analyses. These updated analyses incorporated a revised patient access scheme (designated as commercial-in-confidence by the manufacturer) and were submitted by the manufacturer in response to the draft final appraisal determination released in August 2010. Sections 3.17 to 3.23 give details of the original economic analyses and the related ERG critique. Sections 3.24 to 3.26 describe the updated analyses, including the revised patient access scheme, the related ERG critique and the impact of altering the yearly discount rate for outcomes while fixing the discount rate for cost at 3.5%.
The manufacturer presented the following total costs per treated patient and total quality-adjusted life years (QALYs) per patient for the base case (excluding any patient access scheme):
Regimen A and B combined: total costs £31,481; total QALYs 15.38.
Regimen A+ and B+ combined: total costs £123,852; total QALYs 16.72.
Regimen A: total costs £29,709; total QALYs 16.10.
Regimen A+: total costs £122,604; total QALYs 16.69.
Regimen B: total costs £33,244; total QALYs 14.66.
Regimen B+: total costs £125,121; total QALYs 16.71.
The manufacturer conducted a series of one-way sensitivity analyses. The results showed that the model was very sensitive to the discount rate for outcomes. The model has a time horizon of 60 years, over which the benefits associated with treatment are accumulated and discounted. The larger the discount rate used for outcomes, the smaller the difference in QALYs gained between treatment with mifamurtide and treatment without mifamurtide. It should be noted that all treatment acquisition costs for mifamurtide are incurred in the first year of the model, and are therefore not affected by discounting. The sensitivity analysis also showed that the model was sensitive to the health-related utility value used for the disease-free health state.
The manufacturer's economic submission also presented a scenario analysis evaluating the effect of the following model assumptions on its base case, including the original patient access scheme and using the combined dataset:
Including amputation and limb salvage costs increased the ICER from £56,683 to £59,231 per QALY gained.
Including adverse events related to hearing loss increased the ICER from £56,683 to £71,065 per QALY gained.
Setting the post-recurrence mortality rate for patients who remain disease free after 5 years to the general population mortality rate increased the ICER from £56,683 to £61,580 per QALY gained.
Applying age-adjusted utility rates increased the ICER from £56,683 to £62,112 per QALY gained.
The manufacturer also carried out a scenario analysis that assessed applying all the assumptions described in section 3.19 simultaneously. This increased the base-case ICER from £56,683 to £91,442 per QALY gained. The manufacturer also carried out probabilistic sensitivity analyses, with analyses assuming a payment threshold of £50,000. The results showed that approximately 30% of the iterations were below this level.
The ERG noted that the base-case assumptions used by the manufacturer were favourable to mifamurtide and had concerns about the selection of the parameters entered in the model (for example, whether the most appropriate comparator was used, whether the effects of hearing loss should be included, whether a general population mortality rate should be used if there is no recurrence in 5 years, whether amputation or limb salvage costs should be used and whether age-related utility values should be used). The ERG stated that, as a result, the ICER for regimen A+ and B+ combined compared with regimen A and B combined was likely to be substantially higher than the £56,683 per QALY gained reported in the manufacturer's base-case analysis.
The ERG was concerned that the statistically significant difference between hearing loss rates reported in INT-0133 was omitted from the base-case economic analysis. The rates were included only in the scenario analyses; 15% for objective or subjective hearing loss for the mifamurtide regimens compared with 8% for the regimens without mifamurtide.
The ERG carried out a number of exploratory sensitivity analyses that included:
comparing regimen A+ with regimen A rather than using all the INT-0133 data from the 3- and 4-agent chemotherapy regimens (that is, regimens A+ and B+ combined and regimens A and B combined)
applying age-adjusted utility values
setting post-recurrence mortality rates to those of the age-matched general population if patients were disease free for 5 years
including amputation and limb salvage costs.
All increased the cost per QALY gained compared with the manufacturer's base case. The ERG's base-case analysis produced a deterministic ICER of £109,296 (probabilistic ICER of £103,494) per QALY gained. These analyses have been replaced by those described below.
After submission of a revised patient access scheme (see section 2.5), the manufacturer provided further updated analyses based on the Committee's preferred assumptions in the economic model (that is, applying age-adjusted utility values, setting post-recurrence mortality rates to those of the age-matched general population if patients were disease free for 5 years, including amputation and limb salvage costs, but still excluding hearing loss as an adverse event) over a 60-year time horizon. The deterministic analysis of regimens A+ and B+ combined compared with regimens A and B combined gave an ICER of £60,205 per QALY gained and the probabilistic analysis gave an ICER of £56,677 per QALY gained. The manufacturer conducted a series of one-way sensitivity analyses on its deterministic base-case results. This showed that the model was sensitive to the discount rates used for outcomes. A discount rate of 0% for outcomes (while fixing the discount rate for costs at 3.5%) reduced the ICER to £25,135 per QALY gained. A discount rate of 6% for outcomes (while fixing the discount rate for costs at 3.5%) increased the ICER to £95,097 per QALY gained.
The ERG carried out a number of exploratory sensitivity analyses on the manufacturer's updated analyses that included:
comparing regimen A+ with regimen A rather than using all the INT-0133 data from the 3- and 4-agent chemotherapy regimens (that is, regimens A+ and B+ combined and regimens A and B combined)
assuming that people receiving mifamurtide experienced hearing loss, as seen in the trial
assuming that a small proportion of patients enter the model in the disease progression health state
assuming that 8% of patients would require 2 vials of mifamurtide per dose.
All these increased the cost per QALY gained compared with the manufacturer's base case. The ERG stated that the sensitivity analyses showed that even with the revised patient access scheme, it was unlikely that the cost per QALY gained was below £60,000. The ERG reported that if clinically meaningful hearing loss can be attributed to mifamurtide, the cost per QALY gained could plausibly be much higher.
The ERG also undertook an analysis to assess the impact of altering the yearly discount rate used for outcomes (while fixing the discount rate for costs at 3.5%) on the manufacturer's and ERG's probabilistic ICERs:
A discount rate of 0% for outcomes reduced the manufacturer's probabilistic ICER to £23,831 per QALY gained and the ERG's probabilistic ICER to £36,893 per QALY gained.
A discount rate of 1.5% for outcomes reduced the manufacturer's probabilistic ICER to £36,076 per QALY gained and the ERG's probabilistic ICER to £54,334 per QALY gained.
A discount rate of 6% increased the manufacturer's probabilistic ICER to £89,810 per QALY gained and the ERG's probabilistic ICER to £141,766 per QALY gained.
Full details of all the evidence are in the manufacturer's submission and the ERG report.