How are you taking part in this consultation?

You will not be able to change how you comment later.

You must be signed in to answer questions

    The content on this page is not current guidance and is only for the purposes of the consultation process.

    NICE health technology evaluations: the manual

    Closed for comments This consultation ended on at   Request commenting lead permission

    1 Involvement and participation

    1.1 Introduction

    1.1.1 Many groups and individuals take part in developing guidance within NICE and externally. The groups and their roles are summarised in participants of the technology evaluation process. NICE also invites evidence from a number of stakeholders, which can include:

    • companies

    • commissioning bodies

    • clinical experts, commissioning experts and patient experts

    • an external assessment group (EAG)

    • healthcare professional organisations

    • patient and carer organisations.

    The following information details which groups take part in an evaluation, including who is invited to provide written or oral evidence.

    1.2 Participants in the evaluation process

    Committee

    1.2.1 The committee considers and discusses the evidence for a technology. It is an independent standing committee that produces recommendations. NICE recruits committee members through open, competitive advertising and appoints members initially for 3 years. Committee members are from:

    • the NHS

    • lay backgrounds (with an understanding of patient and public perspectives on healthcare)

    • academia

    • life sciences companies

    • experts in regulation.

    1.2.2 Full details of how NICE recruits members are in the recruitment and selection procedure for advisory bodies. NICE is committed to the values of equality, diversity and inclusion and welcomes applications for membership of the committee from all sectors of the community.

    1.2.3 The diagnostics advisory committee recruits several specialist committee members alongside the standing committee members for each individual evaluation. They are committee members for that topic only. They typically include clinicians or researchers using the diagnostic technology or practising in the care pathway, as well as lay people with a perspective on the condition being diagnosed. Specialist committee members have the same decision-making role as standing members of the committee. Any reference to committee includes the specialist committee members.

    1.2.4 Members will normally remain in the same committee for the duration of their membership. Sometimes members from one committee may be needed to join another committee. This is to ensure that a meeting is quorate, and that business can be done in line with the committee standing orders and terms of reference.

    1.2.5 Although the committee seeks the views of organisations representing healthcare professionals, patients, carers, companies and government, its advice is independent. Names of committee members, standing orders and terms of reference of each committee are published on the NICE website.

    Lead team

    1.2.6 A lead team is selected from the committee members at the start of each evaluation. They work with the NICE team to guide the evaluation and present the topic to the committee. The lead team normally consists of 2 or 3 committee members who focus on clinical effectiveness, value for money or patient and carer evidence (called the lay lead).

    The technical team

    1.2.7 The technical team consists of the chair or vice chair of the committee along with the NICE team, which normally includes the associate director, the technical adviser and the technical lead.

    1.2.8 The technical team is responsible for considering the evidence submissions and the external assessment report. It identifies and explores issues, comes to preliminary scientific judgements, and advises the committee in its discussion of the evidence.

    1.2.9 The technical team will seek input from the lead team, the EAG, experts and committee members when appropriate.

    Company

    1.2.10 The company that holds the regulatory approval (marketing authorisation, UK Conformity Assessed or CE mark, or other equivalent regulatory approval or guidance issued by the regulator) for the technology being evaluated or its agents.

    Clinical experts and patient experts

    1.2.11 Clinical experts and patient experts are selected from those nominated by consultee organisations or by NICE, taking into account the NICE policy on declaring and managing interests for NICE advisory committees. Experts can be invited to provide written evidence, clarify issues about the evidence base and participate in committee meetings. They may be asked to provide advice before, during and after committee meetings. References to clinical and patient experts means the specialist committee members for diagnostics evaluations.

    External assessment group

    1.2.12 The EAG is an independent academic group that reviews the evidence including any stakeholder submissions and the clinical and cost effectiveness, or cost comparisons of the technology or technologies being evaluated. The EAG develops an external assessment report for the committee.

    NHS commissioning experts

    1.2.13 NICE invites NHS commissioning experts from NHS England and clinical commissioning groups (or other relevant commissioning organisations) to help clarify issues about the submitted evidence. They may be asked to provide advice before, during and after committee meetings about:

    • their views and experiences of the technology

    • the condition from an NHS perspective

    • considerations about how the technology could be delivered in the NHS

    • when treatment eligibility criteria may be used in the NHS for high-cost treatments, or for technologies recommended with managed access.

    NHS England national clinical lead

    1.2.14 The NHS England national clinical lead (or a nominated deputy) for the clinical area relevant to the technology being evaluated is invited to provide an evidence submission and attend committee meetings for technology appraisal and highly specialised technology evaluations.

    1.2.15 In some circumstances a national clinical lead is invited to attend the private session (part 2) of the committee meeting to discuss academic in confidence or commercial in confidence information.

    Stakeholders

    1.2.16 Stakeholders are organisations that have registered to participate in a technology evaluation.

    1.2.17 NICE invites the following stakeholders (when relevant) to take part in the evaluation:

    • the company that holds, or is expected to hold, the regulatory approval for the technology

    • relevant comparator technology companies

    • national organisations representing patients and carers

    • organisations representing healthcare professionals

    • the Department of Health and Social Care

    • relevant healthcare representatives from the devolved nations

    • NHS England as the commissioner for specialised services

    • relevant NHS commissioning organisations

    • a provider of NHS services

    • any relevant groups developing clinical and social care guidelines, or public health guidance

    • other related research groups (for example, the Medical Research Council and the National Cancer Research Institute)

    • other groups (such as the NHS Confederation, the NHS Commercial Medicines Directorate, the Medicines and Healthcare products Regulatory Agency and the Academic Health Science Networks).

    1.2.18 The following stakeholder groups, when relevant to the evaluation, are considered consultees and are invited to submit evidence and nominate clinical, patient and commissioning experts:

    • the company that holds, or is expected to hold, the regulatory approval for the technology being evaluated

    • national organisations representing patients and carers

    • organisations representing healthcare professionals

    • clinical, patient and NHS commissioning experts

    • the Department of Health and Social Care

    • the Welsh government

    • NHS England as the commissioner for specialised services (for relevant evaluations)

    • clinical commissioning groups (or other relevant commissioning organisations).

    NICE Decision Support Unit

    1.2.19 The Decision Support Unit is commissioned by NICE to provide a research and training resource to support NICE's technology guidance programmes and the methods of evaluation.

    Members of the public

    1.2.20 Members of the public may:

    • comment on draft guidance consultations

    • apply to observe committee meetings as public observers.

    NICE staff

    Centre director

    1.2.21 The centre director is responsible for delivering all outputs of the Centre for Health Technology Evaluation and ensures that evaluations are done in line with the published process and methods.

    Programme director

    1.2.22 The programme director is responsible for all aspects of managing and delivering the evaluation work programme. The programme director works with the NICE sponsor branch at the Department of Health and Social Care and other national bodies, and with healthcare industry bodies. The programme director is responsible for signing off guidance at specific stages of an individual evaluation. The programme director is also responsible for ensuring that evaluations are done in line with the published process and methods.

    Associate director

    1.2.23 The associate director is responsible for leading the development of individual evaluations within the programme and has delegated responsibility, from the programme director, for approving documents at specific stages of an individual evaluation.

    Technical adviser

    1.2.24 The technical adviser is responsible for the technical quality of the evaluation. This involves providing advice on technical issues, and if appropriate, reviewing and quality assuring the work of the technical lead. The technical adviser also ensures a consistent approach is taken across the programme.

    Technical lead

    1.2.25 The technical lead is the analyst responsible for the technical aspects of the evaluation, including liaising with the EAG and the company, scoping the topic, preparing drafts of guidance and advising the committee. There may be more than 1 technical lead for an evaluation.

    Project team

    1.2.26 The project team is responsible for planning individual evaluation timelines, ensuring the timelines and process are communicated and followed by all participants, and liaising with stakeholders and others contributing to the evaluation.

    Communications lead

    1.2.27 The communications lead is responsible for circulating and communicating the guidance to appropriate groups within the NHS in England, and to patients and the public, and companies.

    Guidance information services lead

    1.2.28 The guidance information services lead supports the technical lead during scoping. The information services lead gathers information on the technology and its evidence base to support the development of the scope. For some topics they will also track key information throughout the evaluation.

    Editorial lead

    1.2.29 The editorial lead is responsible for ensuring that guidance is clear and consistent. The editorial lead prepares the final versions of the guidance and information for the public.

    Public Involvement Programme (PIP) adviser

    1.2.30 PIP is the team at NICE that supports and develops public involvement across NICE's work programme. The public involvement adviser works alongside the evaluation team to support the involvement of patients, carers, people who use services, and the organisations who represent them, throughout the evaluation. A public involvement adviser is assigned to each evaluation.

    Commercial and Managed Access teams

    1.2.31 The Commercial and Managed Access teams work with stakeholders during NICE's evaluation processes. They inform their commercial and managed access activities and enable timely discussions between NHS England and the company. This helps ensure timely guidance and patients' access to cost-effective technologies.

    Resource impact lead

    1.2.32 The resource impact lead works with the technical team, committee, the company and experts to produce guidance-related resource impact assessment tools. The tools consist of a resource impact report (which may be a short summary report) and template to help organisations assess the resource impact of implementing NICE guidance. If the resource impact is not expected to be significant, a resource impact statement is produced. The tools are subject to a limited consultation and are published at the same time as the guidance. The resource impact lead may also be involved at the topic selection stage.

    Adoption lead

    1.2.33 The adoption and implementation team produces an adoption report at the scoping stage for medical technology and diagnostic guidance. The report is developed with NHS clinicians and focuses on the practicalities of adopting the technology. The report is shared with the committee when it drafts recommendations and is published as part of the committee papers.

    1.2.34 If needed, the team may also develop adoption support resources. They are developed with clinical experts, commissioners, patient and carer organisations and companies. They identify adoption barriers and solutions and describe the experiences of health and social care organisations.

    1.3 How participants are involved

    Companies

    1.3.1 Companies are invited to submit evidence on the technology or technologies being evaluated. They should identify all evidence relevant to the evaluation, including all studies known to them, including clinical trials, follow-up studies and evidence from registries. The submission may include confidential study evidence that is not in the public domain. Companies should provide a summary of information for patients written in plain English using the template provided by NICE.

    1.3.2 For evaluations to develop diagnostics guidance, companies are not formally invited to make an evidence submission but provide information requested on the evidence base and their technology to enable the EAG to develop the external assessment report.

    1.3.3 At the earliest opportunity, NICE will ask companies for details of the studies they intend to include in their submission. If information is unpublished, companies should include the study reports.

    1.3.4 In a single technology evaluation, the company must provide a systematic review of the clinical and cost evidence and an economic evaluation. Evidence requirements are explained in detail in the evidence section of this guide.

    1.3.5 If an evaluation is updating guidance on a technology that was recommended with managed access, the company must also provide the evidence described in the published data collection agreement.

    Participation of company representatives at the committee meeting

    1.3.6 Two representatives from the company (usually 1 with health economics expertise and 1 with medical expertise) for the technology being evaluated can attend the public session (part 1) of the committee meeting. The chair will ask them to respond to questions from the committee and comment on any matters of factual accuracy before concluding part 1. The chair may ask the representatives to remain for part of the private session (part 2) of the committee meeting, specifically to respond to questions from the committee about confidential information in the company's submission. Each representative must:

    • be an employee of the company or have been involved in the company's evidence submission, or participated in the evaluation on behalf of the company when no company submission is needed

    • have relevant detailed knowledge of the technology being evaluated

    • be able to comment on the clinical effectiveness or value for money of the technology

    • agree to be bound by the terms and conditions of NICE's confidentiality agreement

    • be willing and able to discuss the condition and the technology with members of a large committee at a meeting where members of the public and press may be observing

    • be familiar with the purpose and processes of NICE.

    1.3.7 Company representatives will not receive any confidential appendix that the EAG creates for an evaluation with a comparator that has a confidential commercial arrangement.

    Clinical experts, patient experts and commissioning experts

    1.3.8 Clinical experts, patient experts and commissioning experts provide their views and experience throughout the evaluation. They help clarify issues that the technical team has identified, give written evidence, participate in any technical engagement (when needed), and attend the committee meeting (if required).

    1.3.9 For diagnostics guidance, specialist committee members act as clinical and patient experts (but they do not give written evidence). Specialist committee members are recruited in accordance with the appointments to advisory bodies policy and procedure.

    Expert nomination

    1.3.10 Stakeholder organisations are invited to nominate clinical experts, patient experts and commissioning experts, which are then selected by NICE to contribute to the evaluation. NICE may also nominate clinical experts who have been involved in evaluations in related care pathways or who have relevant knowledge of using the technology.

    1.3.11 Experts involved during scoping may be invited to continue participating during the evaluation. They do not have to continue to participate and there is still the opportunity for stakeholders to nominate alternative experts. All expert nominations have the same review and selection process.

    1.3.12 The public involvement adviser can provide advice and support to patient and carer organisations when nominating experts. Patient and carer organisations may nominate both patient and clinical experts.

    1.3.13 Professional organisations may nominate patient, clinical and commissioning experts for the evaluation.

    1.3.14 For technology appraisals and highly specialised technologies guidance, NICE asks NHS England and 2 clinical commissioning groups selected at random to nominate NHS commissioning experts.

    1.3.15 For diagnostics and medical technologies guidance, relevant NHS commissioners of the technology are invited to nominate NHS commissioning experts only if commissioning expertise is specifically needed or if the population is covered by an NHS England specialised commissioning group.

    1.3.16 The nominating organisation and the experts (clinical, patient or NHS commissioning) complete a nomination form that includes information on their experience and knowledge of the condition, experience of the technology, any conflicts of interest and any previous involvement with NICE.

    Expert eligibility and selection

    1.3.17 NICE selects experts from the nominations received and those invited to continue participation from scoping. Clinical and patient experts are chosen based on their experience of the technology and the condition that the technology is designed for. Selection takes into account NICE's policy on declaring and managing interests for committees. Ideally, the clinical and patient experts will have complementary rather than similar backgrounds and experiences. Clinical experts, patient experts and NHS commissioning experts must be able to meet the following requirements:

    • They agree to be bound by the terms and conditions of NICE's confidentiality agreement.

    • They agree to their name and affiliation appearing in the guidance documents.

    • They have knowledge or experience of the condition, the technology being evaluated, or the way it is used in the NHS.

    • They are willing and able to discuss the condition and the technology at a committee meeting when members of the public and press are observing.

    • They are familiar with the purpose and processes of NICE (the public involvement adviser can give patient experts support so they can contribute to the evaluation and discussions at committee meetings).

    • They are prepared to declare any interests they have in writing and at committee meetings.

    1.3.18 Clinical experts must meet the following additional requirements:

    • They are in active clinical practice and have specialist expertise in the subject area of the evaluation.

    • Their principal place of work is in the NHS.

    • If they have acted as a clinical expert for the company, or the EAG, they agree to declare this in any submission and at committee meetings.

    • They hold no official office (that is, no paid employment, unpaid directorship or membership of a standing advisory committee) with the company or any relevant comparator technology companies. However, there is discretion to invite an expert who holds official office when the work of the committee would be seriously compromised without their testimony.

    • They are not under investigation by the General Medical Council, do not have interim restrictions placed on their practice, and have not been removed from the medical register.

    • They are not under investigation for professional misconduct and have not been found to be in breach of appropriate professional standards by the relevant professional body.

    1.3.19 Usually, a maximum of 2 clinical experts or 2 patient experts are selected for each evaluation. NHS commissioning experts are selected for technology appraisals and highly specialised technologies guidance and for other guidance when needed.

    1.3.20 For diagnostic evaluation s clinical and lay specialist committee members are recruited at the beginning of the evaluation process. Additional specialist committee members may be appointed after the final scope is published if gaps are identified in the knowledge and expertise needed by the committee. They may support the EAG on behalf of the committee during the evaluation. However, they cannot be appointed as advisers to the EAG so they can maintain sufficient independence from the evidence and contribute to the committee's discussions on the quality of the external assessment report and the development of guidance recommendations.

    Expert participation in the evaluation

    1.3.21 NICE asks experts to submit written evidence on the technology, the way it should be used in the NHS in England, and current management of the condition, for all evaluations except cost comparisons. If the clinical and patient experts choose to support their nominating organisation's written submission, they do not need to submit a separate statement. The committee uses these submissions in its discussion, and they are published as part of the committee papers.

    1.3.22 Experts are invited to participate in technical engagement (if held) before the committee meeting. They are also expected to comment during part 1 of the meeting on the evidence in the written submissions, and to interact fully in the discussions with the committee, including responding to questions.

    1.3.23 The experts' views shared at the committee meeting can:

    • Identify important variations in clinical practice in managing the condition and in the current use of the technology, including:

      • geographical variations

      • identification of subgroups

      • constraints on local implementation

      • specific issues for implementation that affect patients and carers directly.

    • Identify what support is needed to implement guidance on the technology, including:

      • extra staff or equipment

      • education and training for NHS staff and for patients on how to use the technology

      • special requirements in the community for patients and carers (for example, travel to hospital for treatment)

      • ways in which adherence to treatment can be improved.

    • Give personal perspectives on the use of the technology and any difficulties, what benefits are important to patients and carers, and the range and significance of adverse effects.

    • Inform how response to the technology should be assessed and in what circumstances its use might be stopped.

    • Identify subgroups of patients for whom the benefits and risks of the technology might differ.

    • Respond to queries from the NICE technical team, lead team and issues raised by the chair and other committee members, the EAG, other experts, and responses from the company to questions.

    1.3.24 The experts attend the committee meeting (if held) as individuals and not as formal representatives of their nominating organisation. NICE aims to select a cross-section of people from the nominations received, taking into account any declared conflicts of interest. For example, for patient experts, NICE would select a person with direct personal experience of the condition and, if possible, the technology, and a member of a patient, carer or professional organisation. The experts are asked to leave the meeting before the committee makes its decision and finalises the recommendations in the guidance in the private session (part 2) of the meeting, which is closed to the public.

    1.3.25 Committee meetings are not held for cost-comparison evaluations and other technology appraisals and highly specialised technologies evaluations that are considered appropriate for a streamlined approach where the committee decision can be taken outside of a formal meeting. Experts who have been selected to take part in the appraisal may be invited to contribute on a case-by-case basis if they are needed to address specific questions.

    1.3.26 NICE publishes the names and affiliations of the clinical, patient and NHS commissioning experts and the NHS England national clinical lead (or their deputy) in the minutes of committee meetings.

    1.3.27 It is important that there is enough expertise at all stages of the evaluation. NICE welcomes and values the input from all experts. Experts can opt out of attending a committee meeting if they feel their views are adequately reflected in the committee papers, key areas of uncertainty have been addressed, and their attendance would not add to the committee discussion.

    External assessment groups

    1.3.28 NICE commissions independent experts from one of several EAGs to review and critique the available evidence for each technology under evaluation. They produce and are responsible for an external assessment report.

    1.3.29 For a single technology evaluation, the EAG prepares a report that assesses the evidence and any evidence submissions. The EAG may recommend that NICE requests additional analyses from the company, may do additional exploratory analyses itself, or both.

    1.3.30 For a multiple technology evaluation (including all diagnostic evaluations), the EAG creates a report that independently synthesises the evidence from published information and any evidence submissions about the clinical effectiveness and value for money of the technologies. In addition to a systematic review of the clinical and cost evidence, the external assessment report normally includes an economic evaluation and an economic model informed by a review of the evidence. Evidence requirements are explained in section 3.

    NHS England (technology appraisals and highly specialised technologies only)

    1.3.31 NHS England are invited to provide a written submission. It may also develop treatment eligibility criteria based on the licensed indication for the technology, clinical trial evidence, biological plausibility and treatment pathways. Treatment eligibility criteria are developed for highly specialised technologies, cancer, and high-cost technologies. NHS commissioning experts will include information about the proposed treatment eligibility criteria as part of their submission to NICE. When treatment eligibility criteria are not defined at the point of the evidence submission, NHS commissioning experts will explain in their submission what factors are being considered in the development of treatment criteria (for example, the presence of specific biomarkers).

    Patient and carer organisations

    1.3.32 NICE invites written submissions from all patient and carer organisations involved in the evaluation to provide perspectives on:

    • the experience of having the condition (before or after diagnosis) or caring for someone with the condition

    • the experience of receiving care for the condition in the healthcare system

    • the experience of having specific treatments or tests for the condition

    • treatment outcomes that are important to patients or carers (which may differ from the outcomes measured in the relevant clinical studies and the aspects of health included in generic measures of health-related quality of life)

    • the acceptability of different treatments and modes of treatment

    • their preferences for different treatments and modes of treatment

    • their expectations about the risks and benefits of the technology.

      If the technology was previously recommended with managed access, additional patient and carer perspectives could include:

    • whether and how the experience of living with the condition has changed with access to treatment during the managed access period

    • how the treatment of the condition in the NHS has changed since the original evaluation

    • the experience of having the technology during the period of managed access.

    1.3.33 The information is best received directly from people with the condition (or their family or carers) as written accounts of their experiences and points of view. NICE's PIP team has a template for collecting patient and carer perspectives.

    1.3.34 The committee is interested in a range of patient and carer perspectives, especially if there are differences of opinion.

    1.3.35 In the context of the evaluation, the committee is interested in any limitations in the published research literature identified by patient organisations. In particular, the extent to which patient-reported outcome measures, or other end points reported in clinical studies, capture outcomes that are important to patients. Patients may assess research-based evidence from a different perspective to researchers and clinicians and they may judge the evidence according to different criteria. Also, it is helpful to have the perspective of patients or carers about how relevant the clinical outcomes and the standardised generic instruments for measuring health-related quality of life (as specified in the reference case) are to the disease or condition.

    Healthcare professionals and commissioners of health services

    1.3.36 NICE invites submissions from all professional organisations and relevant NHS organisations involved in the evaluation, including:

    • the Royal Colleges of the appropriate clinical disciplines

    • the specialist societies of the appropriate clinical disciplines

    • other appropriate professional bodies and NHS organisations including commissioners of NHS services.

    1.3.37 Healthcare professionals and commissioners of health services provide a view of the technology in relation to current clinical practice. This puts into context the evidence from studies and will help to identify any differences in outcomes from the clinical trials to that achieved in routine clinical practice.

    1.3.38 The written submissions provide a professional view of the place of the technology in current clinical practice and in the care pathway. This includes evidence that relates to some or all of the following:

    • Variations between groups of patients, in particular different baseline risks of the condition and the potential for different subgroups of patients to benefit.

    • Identifying appropriate outcome and surrogate outcome measures.

    • Significance of side effects or adverse reactions.

    • The clinical benefits.

    • Circumstances in which the technology or treatment is used, including:

      • the need for concomitant treatments

      • the settings in which technology is used (for example, primary or secondary care, or in specialist clinics)

      • the need for additional professional input (for example, community care, specialist nursing or other healthcare professionals).

    • Relevant potential comparators.

    • Information on unpublished evidence. Such information should be accompanied by sufficient details to enable a judgement as to whether it meets the same standards as published evidence and to determine potential sources of bias.

    • Evidence from registries and clinical audit.

    • Published clinical guidelines produced by specialist societies.

    • The effect of potential guidance on how care is delivered.

    • Education and training requirements of NHS staff.

    • Patients who would use the technology.

      If the technology was previously recommended with managed access, then the written submissions could include:

    • experience of the technology during the managed access period

    • how many patients had the technology and whether anyone declined treatment, and the reasons for this

    • the variation between groups of patients who had the technology and the potential for different subgroups of patients to benefit

    • expected use of the technology in clinical practice

    • any points of learning arising from the managed access period.