Early value assessment interim statement

3.1 This section of the early value assessment interim statement describes the processes and methods that are being used for the early value assessment pilots. Different approaches are being tested to identify where efficiencies can be made and where alternative approaches could be used, see table 3 in the appendix for different approaches being tested for each topic. The intention of this statement is to provide an overview of the methods and processes but not be prescriptive in detail, to allow flexibility to adapt and change approaches during the pilots. It should be read alongside NICE health technology evaluations: the manual (relevant sections of the manual have been referenced throughout this document).

3.2 The objectives of the early value assessment process and methods are to:

3.3 The overall timeframe of an early value assessment is approximately 6 months. This includes 8 weeks for scoping, stakeholder identification and specialist committee member recruitment; 9 weeks for external assessment; and 7 weeks for guidance production and public consultation.

3.4 The following section outlines the stages of an early value assessment and the key areas for pilots (for a summary of the types of approaches being trialled per pilot see tables 3 and 4 in the appendix).

3.5 Topics for the early value assessment pilots include technologies that address an unmet need or priority area, different types of technology, different purposes or use cases and differing availability of infrastructure for real world data collection. The aim of selecting a wide variety of topics encompassing these aspects is to test that early value assessment is a useful approach for all different types of health technology and use cases, and to show proof of principle of what could be achieved through early value assessment when strong partnerships are in place.

3.6 Technologies selected for early value assessment will be scoped in line with section 2 of NICE's health technology evaluations: the manual, except for the following process changes:

3.7 The standard approach to assessing the evidence for a NICE evaluation is outlined in section 3 of NICE health technology evaluations: the manual. The way that NICE assesses evidence has been amended for the purpose of the early value assessment pilots.

3.8 The evidence assessment processes being piloted include:

3.9 The remainder of this section outlines the methodology for reviewing the evidence for an early value assessment.

3.10 It is expected that there will not be a comprehensive evidence base available for technologies included in early value assessment. The evidence considered by the committee should be relevant to the evaluation in terms of patient groups, comparators, perspective, outcomes and resource use as defined in the scope wherever possible. The aim of the evidence review is to identify the most relevant evidence relating to the decision question defined in the scope. If no evidence directly relevant to the evaluation is available, inclusion criteria should be expanded to look at a broader evidence base.

3.11 In addition to reviewing the evidence on the technologies, additional reviews may be needed to look for studies that report on relevant information, but do not include the intervention technologies. For example, evidence on treatment effect for a condition in which people have a biomarker identified by the new technology or evidence on a relationship between surrogate and final clinical outcomes. Identifying whether there is such evidence will help the committee identify whether there are data gaps across the care pathway that would lead to uncertainties in a future model.

3.12 Broad evidence mapping searches may need to be done to identify evidence on the technologies because articles may be published in less well-known journals, studies may not be well indexed or may only be presented as conference abstracts. Unpublished data provided by companies and other stakeholders could be considered.

3.13 The evidence reviews may be done using pragmatic approaches in order for the assessment to be done in the time available. For example, using rapid review methods such as single screening and data extraction. For evidence reviews of outcomes where a large amount of data could be available, a restricted number of databases could be searched or a review of review done.

3.14 Searches for ongoing studies should be done. Any ongoing studies that were identified should be checked to see if they will help to fill key evidence gaps.

3.15 A full critical appraisal of studies using a validated tool is not needed, but there should be discussion on the potential biases in key studies and how the risk of bias could affect key outcomes. The report should explicitly detail the potential sources of bias such as the main confounding factors. Comments on the generalisability of the results to clinical practice in the NHS should also be made.

3.16 Evidence synthesis of the key findings should be provided in a simple narrative and descriptive format, a quantitative meta-analysis is not expected. Where the direction of the effect (and associated confidence intervals) is reported in the literature it should be included. A cautious interpretation of the findings should be provided. The report should include a discussion of the evidence gaps and uncertainties in the identified evidence and suggestions of outcomes to focus on in future evidence generation.

3.17 Economic analyses for early value assessment may differ from the standard approach for guidance products. The standard approach is outlined in section 4 of NICE health technology evaluation: the manual. For early value assessment, the economic evaluation work that is likely to be most beneficial for committee decision making is likely to vary by topic, needing flexibility from assessment groups doing the work. Proposed work should be discussed between the NICE team and the external assessment group from an early stage to inform development of the protocol for the assessment.

3.18 The objectives of the economic evaluation are to:

3.19 The cost of reversing a decision to adopt the technologies while further evidence is generated may also be considered. This is to assess the potential implications if, after an initial decision by NICE to recommend use of the technologies while further evidence is generated, a later assessment (after this evidence is collected) shows the technology not to be cost effective or cost saving. This could include any costs that could not be recouped related to implementation such as fixed or up-front costs related to the purchase of equipment, training costs or changes to organisation of care pathways.

3.20 Efforts should be made to identify relevant economic evaluations related to the technologies or disease area that could inform the work. For example, through targeted literature searches or work highlighted by experts or stakeholders as potentially relevant. Any models highlighted or submitted by companies could be considered.

3.21 Economic evaluation should as a minimum consider model structure(s) that would be needed for a future analysis to support NICE committee decision making. Relevant experts (for example appointed specialist committee members) should be consulted to validate this. Comparisons should also be made with any existing model structures identified where they are deemed relevant or robust (see section 3.19).

3.22 Development of a preliminary or early coded model (that is, a model implemented in a software platform) should be considered to help meet the objectives in section 3.17, particularly to explore uncertainty with the aim of identifying key drivers of the model results to inform decision making about further evidence generation. Existing models could be used here, if considered suitable and an agreement is in place to allow models developed by third parties to be made available to stakeholders.

3.23 The reference case will be the same as described in section 4.2 of NICE health technology evaluations: the manual.

3.24 Guidance for presenting data and results of models is described in section 4.10 of NICE health technology evaluations: the manual. Outputs of models should be presented to easily allow clinicians to validate model results and the extent of work done to validate model outcomes should be described.

3.25 If no coded model is produced (see section 3.21), targeted searches should still be done to attempt to identify data (and highlight where there may be a paucity of data) for model parameters considered likely to be key drivers of future cost-utility analysis (see section 3.9 and section 3.10).

3.26 New approaches to decision making are being piloted for early value assessment. NICE's usual approach to decision making and committee recommendations is outlined in section 6 of NICE health technology evaluations: the manual. The key points are outlined below.

3.27 Decision making will be the responsibility of one of the following:

3.28 When making decisions the committee should consider:

3.29 When multiple technologies are considered in a topic, each should be assessed independently. For example, considering any difference between technologies in terms of whether they can solve the specified unmet need and any differences in further evidence needs.

3.30 The committee or panel can make the following types of recommendations (specific wording subject to change):

3.31 For the conditional recommendation, the outcomes that the committee need further data on to support future decision making should be listed and include a focus on outcomes which are fundamental to decision making.