NICE process and methods
4 A new approach to evidence generation for medical and digital health technologies
4 A new approach to evidence generation for medical and digital health technologies
Intention
4.1 The early value assessment evidence generation approach is designed to help technology developers to work with patients and clinicians, along with NHS data custodians and analytical partners who can generate the new evidence needed either from new or ongoing research or from real-world data. The approaches outlined are iterative and may change to fit the needs of the project.
4.2 The aims of this approach are:
-
to collaborate on evidence generation with opportunities to use real-world evidence in line with NICE's 5-year strategy.
-
advice and support to identify proportionate and pragmatic approaches to evidence generation
-
collaborative engagement between a wide range of stakeholders to help adoption with evidence generation.
4.3 For technologies conditionally recommended through early value assessment, NICE will support development of a plan for new evidence generation by:
-
Engaging with companies about the evidence already available and any evidence gaps that could be addressed to identify opportunities for further evidence generation.
-
Assess the evidence gaps that would need to be addressed for a routine recommendation, identifying existing and ongoing sources of data and evidence that could address these
-
Producing clear statements about what new evidence generation could sufficiently address the evidential gaps
-
Broker relationships on behalf of the technology developer with:
-
NHS real-world data custodians, who hold data that can assist in addressing evidence gaps.
-
Analytical or research partners, who can advise on or do analyses to generate evidence.
-
Voluntary and community sector organisations, who can reach people who might like access to new technologies.
-
Clinical networks, who can provide information and guidance to the NHS about using these products and any data collection needs to support evidence generation.
-
Stakeholder roles
NICE's role
4.4 NICE's role around evidence generation for digital health and medical technologies will be to:
-
Identify what additional evidence is needed for NICE to support using the technology in the NHS.
-
Work with the technology developer to:
-
identify system partners who may be able to support the design and delivery of an evidence generation plan, and to broker relationships, as needed, with potential data controllers and research funders.
-
identify and engage with experts who can provide advice around evidence generation.
-
determine what should be included in an evidence generation plan, including guidance on suitable approaches to generating the evidence.
-
develop a proportionate and pragmatic approach to evidence generation with consideration for the burden on patients, healthcare professionals and the wider system.
-
help the development of evidence generation plans through multi-stakeholder engagements.
-
-
Share the published final evidence generation plan with stakeholders.
-
Link technology developers with potential research funding partners.
Technology developer's role
4.5 The technology developer's role will be to:
-
Support evidence reviews to identify the most relevant evidence and information relating to the decision question defined in the scope.
-
Engage with relevant stakeholders to help develop and implement an evidence generation plan, to:
-
Identify robust approaches to evidence generation, consider aspects such as study design, data quality and suitability.
-
Work with potential evidence sources to establish approaches to generating evidence.
-
Ensure new evidence is generated in accordance with all applicable data protection legislation.
-
Clinician, patient, carer, and data custodian role
4.6 Clinician, patient, carer, and data custodian input will inform the early value assessment by:
-
Providing the technology developer with relevant and constructive contributions, based on their experience, expertise, and knowledge to influence the evidence generation plan. These contributions could include feedback on the placement of the technology, its potential value, its acceptability or feasibility and its implementation. They could also incorporate reviewing any potential evidence generation plans around their feasibility and appropriateness.
-
Providing input to assess the burden of data collection to ensure it is deliverable.
-
Identifying potential implementation barriers which would need to be addressed as part of the evidence generation plan.
Guiding principles
4.7 These guiding principles are important because they help the early value assessment process to produce the high-quality information necessary to support NICE decision making and make sure that patients and the NHS can benefit from new technologies. These guiding principles should be used by all stakeholders when considering the early value assessment evidence generation process.
4.8 The principles that will guide our approach to evidence generation for medical technologies are outlined in table 5 of the appendix.
The stages of the evidence generation approach
4.9 There will be multiple opportunities throughout the early value assessment process to consider the evidence and gaps which could be resolved with further evidence generation. The objective of this evidence generation approach will be to produce a plan which the technology developer will be able to take forward in partnership with key stakeholders to generate the new evidence that will support a future NICE guidance.
Evidence review
4.10 During scoping and the evidence review, the NICE technical team, external assessment group and committee will identify areas of potential evidence gaps and inform the evidence generation team of these. The evidence generation team will then investigate and consider the data collection landscape around the technology in the NHS (through literature review and expert elicitation) to see if there are any suitable initiatives that could support evidence generation in an early value assessment. The evidence generation team will also seek to identify and work with independent academic advisors and representatives of the NICE committee to form the evidence generation panel.
4.11 The assessment by NICEs external academic partners will identify potential evidence gaps. Before the early value assessment recommendations are published, the evidence generation panel will consider the evidence gaps and create an outline of possible approaches to addressing them.
Formulation of preliminary evidence generation plan
4.12 If the committee or panel considering the topic makes a recommendation for conditional use with evidence generation, they will identify the evidence gaps that need to be addressed for the topic to be considered for a full recommendation. The evidence generation team will then liaise with the technology developers and stakeholders of ongoing studies involved in the topic to discuss the evidence generation process and how they can support it. The evidence generation panel will then meet and develop the initial evidence generation plan.
Multi-stakeholder engagement
4.13 Once a plan has been developed by the evidence generation panel with input from the technology developers it will be finalised through a multi-stakeholder engagement. Stakeholders involved in this could include:
-
evidence generation partners such as national registries and regional integrated data repositories
-
potential funders for early value assessment evidence generation
-
NHS England and Integrated Care Board commissioners
-
Academic Health Science Networks (AHSNs)
-
National Institute for Health and Care Research (NIHR)
-
clinicians and patient representatives
-
third-party data experts where relevant
-
patient groups and service users.
4.14 The evidence generation plan will be published alongside publication of the committee recommendations after it has been confirmed by multi-stakeholder engagement.
Pilot phase
4.15 The early value assessment pilot phase will inform the development of the evidence generation process, so evidence generation plans may be published later than committee recommendations.
The evidence generation plan
4.16 The evidence generation plan will refer and link to the sections of the NICE guidance document that summarise the existing evidence and describes the evidence gaps identified by committee. It will then propose how the evidence gaps can be addressed. It will be developed through a systematic approach, addressing practical considerations and challenges in generating new evidence.
4.17 The evidence generation plan is not a research protocol, a research or data collection contract, nor is it an undertaking from NICE to lead the subsequent data collection and analysis.
4.18 The evidence generation plan seeks to be an outline which the developer can use to produce a research protocol and engage with potential funders and evidence generation partners.