3 Cost effectiveness
Section 3 provides detailed guidance on the level of information that should be provided in the evidence submission template about the cost effectiveness of the appraised technology.
When completing the template, also refer to NICE's health technology evaluations manual.
3.1 Published cost-effectiveness studies
In appendix E, provide details of the identified studies.
In the main submission, summarise the published cost-effectiveness studies using a table similar to the following.
Abbreviations: ICER, incremental cost-effectiveness ratio; QALYs, quality-adjusted life years.
3.2 Economic analysis
Summarise how the cost-effectiveness studies identified in appendix E inform the economic analysis.
If a de novo model economic model is included in the submission, please justify why this is necessary.
Patient population
3.2.1
State which patient groups are included in the economic evaluation and how they reflect the population defined in the scope and decision problem for the NICE technology evaluation, marketing authorisation or CE marking, and the population from the trials. If there are differences, please provide the rationale. Explain the implications of this for the relevance of the evidence base to the decision problem. For example, indicate if the population in the economic model is different from that described in the (draft) summary of product characteristics (SmPC) or information for use (IFU) and included in the trials.
Model structure
3.2.2
Describe the model structure and provide a diagram of the model submitted, including the following:
-
Type of analysis (for example, decision tree, Markov model, discrete event simulation model).
-
Justification of the chosen structure in line with the clinical pathway of care described in section 1.3.
-
How the model structure and its health states capture the disease or condition for patients identified in section 1.3.
-
Where appropriate, state the cycle length and whether a half-cycle correction has been applied.
3.2.3
Complete the table after 3.2.4 presenting the features of the analysis. If there have been NICE technology evaluations in the same disease area, please summarise the main inputs to the economic models accepted by evaluation committees. If the model in this evaluation uses different inputs, give a rationale.
3.2.4
Compare and justify your chosen values with the methods specified by NICE in the reference case (see NICE's health technology evaluations manual, section 4.2, table 4.1).
Intervention technology and comparators
3.2.5
If the intervention and comparator(s) are not implemented in the model as per their marketing authorisations or CE marking, describe how and why there are differences. Make it clear whether the intervention and comparator(s) included in the model reflect the decision problem. If not, briefly describe how and why, cross referencing to the decision problem section in your submission.
3.2.6
If a treatment continuation rule has been assumed for the intervention and comparator(s), provide the rationale for the continuation rule and where it is referenced (for example, [draft] SmPC, UK public assessment report, comparator use, clinical practice, or clinical trial protocols). Please note that this refers to clinical continuation rules and not patient access schemes or commercial arrangements. If a treatment continuation rule is included in the model that is not stated in the (draft) SmPC or IFU, this should be presented as a separate scenario by considering it as an additional treatment strategy alongside the base-case interventions and comparators. Consideration should be given to the following:
-
the costs and health consequences of implementing the continuation rule (for example, any additional monitoring required)
-
the robustness and plausibility of the end point on which the rule is based
-
whether the 'response' criteria defined in the rule can be reasonably achieved
-
the appropriateness and robustness of the time at which response is measured
-
whether the rule can be incorporated into routine clinical practice
-
whether the rule is likely to predict those people for whom the technology is particularly cost effective
-
issues about withdrawal of treatment for people whose disease does not respond and other equity considerations.
3.3 Clinical parameters and variables
This section should be read with NICE's health technology evaluations manual, section 4.6.
When relevant, answers to the following questions should be derived from, and be consistent with, the clinical evidence section of the submission (section 2). Cross references to the clinical evidence section should be provided. If alternative sources of evidence have been used, the method of identification, selection and synthesis should be provided as well as justification for the approach. The answers should clearly specify the approach taken in the base-case analysis.
3.3.1
Describe how the clinical data were incorporated into the model, also commenting on the following factors:
-
Whether intermediate outcome measures were linked to final outcomes (for example, if a change in a surrogate outcome was linked to a final clinical outcome). If so, explain how the relationship was estimated, what sources of evidence were used, and what other evidence there is to support it.
-
Whether costs and clinical outcomes are extrapolated beyond the trial follow-up period(s). If so, explain and justify the assumptions that underpin this extrapolation, particularly the assumption that was used about the longer-term difference in effectiveness between the intervention and its comparator. For the extrapolation of clinical outcomes, present graphs of any curve fittings to patient-level data or Kaplan–Meier plots and the methods and results of any internal and external validation exercises. The NICE Decision Support Unit has published technical support document 14, which provides additional information on the implementation of methods and reporting standards for extrapolation with patient-level data, and technical support document 21, which provides information on flexible methods for survival analysis.
Although the Decision Support Unit is funded by NICE, technical support documents are not formal NICE guidance or policy.
3.3.2
Demonstrate how the transition probabilities were calculated from the clinical data. If appropriate, provide the transition matrix and describe the details of the transformation of clinical outcomes or any other relevant details here.
3.3.3
If there is evidence that transition probabilities may change over time for the treatment effect, condition or disease, confirm whether this has been included in the evaluation. If there is evidence that this is the case, but it has not been included, provide an explanation of why it has been excluded.
3.3.4
If clinical experts have assessed the applicability of the clinical parameters or approximated any of the clinical parameters, provide the following details:
-
the criteria for selecting the experts
-
the number of experts approached
-
the number of experts who participated
-
declaration of potential conflict(s) of interest from each expert whose opinion was sought
-
the background information provided and its consistency with all the evidence provided in the submission
-
the method used to collect the opinions
-
the medium used to collect opinions (for example, was information gathered by direct interview, telephone interview or self-administered questionnaire?)
-
the questions asked
-
whether iteration was used in the collation of opinions and if so, how it was used (for example, the Delphi technique).
3.3.5
Details to report in this section include:
-
subsequent treatments included in the model and a discussion of, and rationale for, any differences from the subsequent treatments' data in the trial(s)
-
drug exposure data used in the model and the source of this data
-
treatment discontinuation data used in the model and a rationale if different from the data in the trial(s).
3.4 Measurement and valuation of health effects
This section should be read with the NICE's health technology evaluations manual, section 4.3.
The NICE Decision Support Unit has published several technical support documents that provide additional information on measuring and valuing health benefits in economic evaluation:
-
An introduction to the measurement and valuation of health for NICE submissions (technical support document 8).
-
The identification, review and synthesis of health state utility values from the literature (technical support document 9).
-
The use of mapping methods to estimate health state utility values (technical support document 10). Also consider the NICE Decision Support Unit report on methods for mapping between the EQ-5D-5L and the 3L for technology appraisal, where relevant.
-
Alternatives to EQ-5D for generating health state utility values (technical support document 11).
-
The use of health state utility values in decision models (technical support document 12).
Although the Decision Support Unit is funded by NICE, technical support documents are not formal NICE guidance or policy.
Mapping
3.4.2
If applicable, describe the mapping methods used to estimate health state utility values from the quality-of-life data collected in clinical trials. Please include the following information:
-
which tool was mapped from and onto which other tool (for example, SF-36 to EQ-5D)
-
details of the methodology used
-
details of validation of the mapping technique
-
if the mapping technique is published or has been used in other NICE technology evaluations for similar diseases or health conditions.
Adverse reactions
3.4.4
Describe how adverse reactions affect health-related quality of life. The effect of adverse reactions on health-related quality of life should be explored regardless of whether they are included in a cost-effectiveness analysis in the base-case analysis. Any exclusion of the effect of adverse reactions on health-related quality of life in the cost-effectiveness analysis should be fully justified.
3.5 Cost and healthcare resource use identification, measurement and valuation
This section should be read with NICE's health technology evaluations manual, section 4.4.
3.5.1
All parameters used to estimate cost effectiveness should be presented clearly in a table with details of data sources. For continuous variables, mean values should be presented and used in the analyses. For all variables, measures of precision should be detailed.
Resource identification, measurement and valuation studies
-
In appendix G, describe how relevant cost and healthcare resource use data for England were identified.
-
In appendix I, provide the relevant price details for each treatment, including the intervention, comparator and subsequent treatments used in the model, including concomitant treatments.
3.5.2
When describing how relevant unit costs were identified, comment on whether NHS reference costs or payment-by-results (PbR) tariffs are appropriate for costing the intervention being evaluated. Describe how the clinical management of the condition is currently costed in the NHS in terms of reference costs and the PbR tariff. Provide the relevant Healthcare Resource Groups and PbR codes and justify their selection with reference to section 2.
3.5.3
If clinical experts assessed the applicability of the cost and healthcare resource use values available, or approximated any of the values used in the cost-effectiveness analysis, provide the details (see section 3.3.4).
Intervention and comparators' costs and resource use
3.5.4
In a table, summarise the cost and associated healthcare resource use of each treatment. A suggested format is provided after section 3.5.6. Provide a rationale for the choice of values used in the cost-effectiveness model discussed in section 3.1. Analyses should be based on prices that reflect as closely as possible those paid in the NHS, including any known price reductions available across the NHS. Please include the price from the electronic market information tool (eMIT) if available for the treatment. eMIT prices should reflect the current version of eMIT, within 2 months of the company submission.
3.5.5
State where the medicine is likely to be commissioned and prescribed (for example, in primary care, by integrated care boards or in secondary care).
3.5.6
Drug wastage should be taken into account in calculating drug costs. For oral treatments dispensed in tablet packs, NICE will evaluate oral treatments costed on a per pack basis, unless clinical experts or other evidence suggests that prescribing and dispensing is done such that there would be no wastage.
Health state unit costs and resource use
3.5.7
Summarise and tabulate the costs included in each health state. See the following suggested format for a table. Cross refer to other sections of the submission for the resource costs. Provide a rationale for the choice of values used in the cost-effectiveness model. The health states should refer to the states in section 3.2.
Adverse reaction unit costs and resource use
3.5.8
Summarise and tabulate the costs for each adverse reaction listed in section 2.10 and included in the cost-effectiveness analysis. See the following suggested format for a table. Cross refer to other sections of the submission for the resource costs.
Miscellaneous unit costs and resource use
3.5.9
Describe and tabulate any additional costs and healthcare resource use that have not been covered elsewhere (for example, costs relating to subsequent lines of therapy received after disease progression, personal and social services costs). If none, please state.
3.6 Severity
This section should be read with NICE's health technology evaluations manual section 6.2.12 to 6.2.22.
3.6.1
When relevant, outline whether this technology meets the criteria for a severity weight. Provide details about the calculation of quality-adjusted life year (QALY) shortfall, including source of population EQ-5D data and survival data. Present supporting evidence and validation of model outcomes. Complete the tables after 3.6.2 and, when relevant, cross reference to where this information is found in the company submission.
3.6.2
The data used to estimate both absolute and proportional QALY shortfall should focus on the specific population for which the technology will be used and be based on established clinical practice in the NHS. Calculation of absolute and proportional shortfall should include an estimate of the total QALYs for the general population with the same age and sex distribution as those with the condition.
3.7 Uncertainty
If relevant, include a statement on how the nature of this condition or technology impacts the ability to generate high-quality evidence.
3.8 Managed access proposal
This section should be read with NICE's health technology evaluations manual, sections 5.5.20 to 5.5.29.
A managed access proposal may be made for any technology that is eligible for the Cancer Drugs Fund or the Innovative Medicines Fund. The committee can consider a recommendation with managed access for eligible technologies when:
the technology has the plausible potential to be cost effective at the currently agreed price, but the evidence is currently too uncertain, and
new evidence that could sufficiently support the case for recommendation is expected from ongoing or planned clinical trials, or could be collected from patients having the medicine in clinical practice, and
the data could feasibly be collected within a reasonable timeframe (up to a maximum of 5 years) without undue burden.
A managed access proposal should include the following:
3.8.1
Specify whether you consider the technology to be eligible for one of the managed access funds. Cancer drugs are eligible for the Cancer Drugs Fund. Medicines that have the potential to address an unmet need and provide clinically significant benefits to patients who are eligible for the Innovative Medicines Fund. When detailing the unmet need and clinically significant benefits cross refer to other parts of the submission, including the severity section and the incremental QALYs gained within the base-case increment cost-effectiveness analysis results.
3.8.2
List the key uncertainties that you consider could prevent the committee from making a recommendation from routine use, and the outcome data and data source that could be collected to sufficiently support the case for recommendation after a period of managed access. Where there are multiple sources identified, mark in bold the data source you consider would be the primary source to address the evidential uncertainty. See the following suggested table format.
3.8.3
When a primary source of data is not currently included within the NICE economic model, for example, a yet to be published clinical study or data collected in clinical practice, describe how the data would be analysed and, if applicable, how it would be incorporated into the economic model at the end of managed access.
3.8.4
Provide an overview of all the clinical studies or registries listed within the suitability for managed access section. The following is a suggested format for clinical trial data sources and data collected through the Systemic Anti-Cancer Therapy (SACT) dataset.
Table X Overview of data source
Table X Overview of data source
3.8.5
For registries other than the SACT dataset please include whether you have approached the registry to explore collecting, analysing and sharing the data in your managed access proposal and whether there are any considerations around information governance and data sharing that may need to be addressed.
3.8.6
Specify the anticipated timeframe of data collection required to provide meaningful data. Please justify why you consider this timeframe is as short as necessary to address the identified uncertainties.
3.8.7
Describe any additional considerations that may impact the feasibility of data collection within managed access. These may include:
-
any additional burden that you have identified that a managed access may cause patients, clinicians, or the NHS
-
potential barriers to agreeing or implementing a managed access
-
any ethical, equality, or patient safety concerns with the proposed data collection and analysis
-
actions you have taken to improve the feasibility of a managed access.
3.8.8
You must submit a separate commercial access proposal as part of the managed access proposal. The process for submitting a patient access scheme or commercial access agreement is outlined in NICE's health technology evaluations manual, section 5.8.
3.10 Base-case results
This section should be read with NICE's health technology evaluations manual, sections 4.6.4 and 4.10.6 to 4.10.7.
3.10.1
Provide the results of the analysis. In particular, results should include, but are not limited to, the following:
-
the link between clinical- and cost-effectiveness results
-
costs, QALYs and incremental cost per QALY
-
when appropriate, expected net health benefits, using values placed on a QALY gain of £20,000 and £30,000
-
disaggregated results such as life years gained, costs associated with treatment, costs associated with adverse reactions, and costs associated with follow up or subsequent treatment.
Base-case incremental cost-effectiveness analysis results
3.10.2
When presenting the results of the base-case incremental cost-effectiveness analysis in the following table, list the interventions and comparator(s) from least to most expensive. Present incremental cost-effectiveness ratios (ICERs) compared with baseline (usually standard care) and then incremental analysis, ranking technologies in terms of dominance and extended dominance. If the company has formally agreed a patient access scheme or commercial arrangement with NHS England, present the results of the base-case incremental cost-effectiveness analysis with the patient access scheme or commercial arrangement.
Abbreviations: ICER, incremental cost-effectiveness ratio; LYG, life years gained; QALYs, quality-adjusted life years.
In appendix H, please provide the clinical outcomes and disaggregated results from the model.
3.11 Exploring uncertainty
This section should be read with NICE's health technology evaluations manual, sections 4.6 and 4.7.
3.11.1
Present an overall assessment of uncertainty, including the relative effect of different types of uncertainty on cost-effectiveness estimates, and an assessment of whether the uncertainties that can be included in the analyses have been adequately captured. Highlight the presence of uncertainties that are unlikely to be reduced by further evidence or expert input.
Probabilistic sensitivity analysis
3.11.2
All inputs used in the analysis will be estimated with a degree of imprecision. As specified in NICE's health technology evaluations manual, probabilistic sensitivity analysis is preferred for translating the imprecision in all input variables into a measure of decision uncertainty in the cost effectiveness of the options being compared. In non-linear decision models, probabilistic methods provide the best estimates of mean costs and outcomes. The mean value, distribution around the mean, and the source and rationale for the supporting evidence should be clearly described for each parameter included in the model. The distributions for probabilistic sensitivity analysis should not be arbitrarily chosen, but should represent the available evidence on the parameter of interest, and their use should be justified.
3.11.3
Consider evidence about the extent of correlation between individual parameters and reflect this in the probabilistic analysis. When considering relationships between ordered parameters, consider approaches that neither artificially restrict distributions nor impose an unsupported assumption of perfect correlation. Clearly present assumptions made about the correlations.
Provide the following information:
3.11.4
The distributions and their sources for each parameter should be clearly stated if different from those presented in section 3.5, including the derivation and value of 'priors'. If any parameters or variables were omitted from the probabilistic sensitivity analysis, please provide the rationale for the omission(s).
3.11.5
Present the incremental cost-effectiveness results of a probabilistic sensitivity analysis (including 95% confidence intervals). Appropriate ways of presentation include confidence ellipses and scatter plots on the cost-effectiveness plane and cost-effectiveness acceptability curves. Cost-effectiveness acceptability curves should include a representation and explanation of the cost-effectiveness acceptability frontier. Present results exploring uncertainty in a table, identifying parameters that have a substantial effect on the modelling results. As well as details of the expected mean results (costs, outcomes and ICERs), also present the probability that the treatment is cost effective if the ICER is £20,000 to £30,000 per QALY gained. Describe how the probabilistic ICER(s) were calculated and provide the rationale.
3.11.6
Describe and explain, if any, the variation between the incremental cost-effectiveness analysis results estimated from the base-case analysis (section 3.10) and the probabilistic sensitivity analysis.
Deterministic sensitivity analysis
3.11.7
If relevant, identify which variables were subject to deterministic sensitivity analysis, how they were varied, and the rationale behind this. Only report analyses when there is genuine uncertainty about a parameter, giving a rationale for why this is the case.
3.11.8
For example, there may be uncertainty about the extrapolation of outcomes or costs beyond the time horizon of a trial.
-
Do not deviate from the reference case. For example, there should not be sensitivity analysis around the discount rate for costs and outcomes.
-
Ensure that values are clinically plausible and not extreme. For example, do not present analyses assuming no treatment effect for comparators.
3.11.9
If relevant, present the results of deterministic sensitivity analysis, focusing on the key drivers of the model. Consider the use of tornado diagrams. Deterministic threshold analysis may be helpful if there are influential but highly uncertain parameters.
3.11.10
For technologies whose final price or acquisition cost has not been confirmed, sensitivity analysis should be done over a plausible range of prices. This may also include the price of a comparator that includes a confidential patient access scheme or commercial arrangement.
Scenario analysis
3.11.11
Sensitivity analysis should be used to explore uncertainty around the structural assumptions used in the analysis. Analysis of a representative range of plausible scenarios should be presented and each alternative analysis should present separate results.
3.11.12
Describe the methods and tabulate the incremental cost-effectiveness results of the scenario analyses done. Include details of structural sensitivity analysis.
3.11.13
Include the impact on the estimates of QALY shortfall when appropriate.
3.12 Subgroup analysis
This section should be read with NICE's health technology evaluations manual, section 4.9.
When subgroups have been considered in the cost-effectiveness analysis, provide the information specified in sections 3.12.1 to 3.12.6.
3.12.1
Types of subgroups that are not considered relevant are those based solely on the following factors:
-
Individual utilities for health states and patient preference.
-
Different treatment costs for individuals according to their social characteristics.
-
Subgroups specified according to the costs of providing treatment in different locations in England (for example, when the costs of facilities available for providing the technology vary according to location).
3.12.2
Please specify whether analysis of subgroups was carried out and how these subgroups were identified, referring to the scope and decision problem specified for the NICE technology evaluation. When specifying how subgroups were identified, confirm whether they were identified based on a prior expectation of different clinical or cost effectiveness because of known, biologically plausible mechanisms, social characteristics or other clearly justified factors. Cross refer to the clinical effectiveness section 2.6.
3.12.3
Clearly define the characteristics of patients in the subgroup.
3.12.4
Describe how the statistical analysis was carried out.
3.12.5
If subgroup analyses were done, please present the results in tables similar to those used in section 2.7.
3.12.6
Identify any obvious subgroups that were not considered and explain why. Please refer to the subgroups identified in the decision problem in section 1.
3.13 Benefits not captured in the QALY calculation
3.13.1
If you consider that there are potential health benefits of the technology that have been inadequately captured and may therefore misrepresent the health utility gained, identify and present the data and provide a rationale for your decision.
3.14 Validation
Validation of cost-effectiveness analysis
3.14.1
When describing the methods used to validate and quality assure the model, provide:
-
the rationale for using the chosen methods
-
references to the results produced and cross references to the evidence identified in the clinical evidence, measurement and valuation of health effects, and cost and healthcare resource sections.
3.15 Interpretation and conclusions of economic evidence
3.15.1
When interpreting and concluding your economic evidence, consider the following:
-
Are the results from this economic evaluation consistent with the published economic literature? If not, why do the results from this evaluation differ, and why should the results in the submission be given more credence than those in the published literature?
-
Is the economic evaluation relevant to all groups of patients who could potentially use the technology as identified in the decision problem?
-
How relevant (generalisable) is the analysis to clinical practice in England?
-
What are the main strengths and weaknesses of the evaluation? How might these affect the interpretation of the results?
-
What further analyses could be carried out to enhance the robustness or completeness of the results?