NICE process and methods

5 How medical technologies guidance is developed

5 How medical technologies guidance is developed

For information on the technical assessment of medical technologies, please refer to the MTEP methods guide.

5.1 Agreement of evaluation schedule

Once a topic is selected for evaluation, NICE schedules the evaluation. If the sponsor does not consider the timing to be appropriate, NICE is not able to guarantee when the evaluation will start.

5.2 Scope

The scope is the first document to be produced after the topic oversight group has selected a technology for consideration. It provides the framework for assessing the technology, taking into account how it works, its comparator(s), the relevant patient population(s), and its effect on clinical and system outcomes. The scope is based on the sponsor's case for adoption. For further information, see the MTEP methods guide.

Once the start date for the evaluation has been agreed, the programme team prepares a draft scope. The scope is intended to define the most important questions about clinical and resource impacts. It sets the boundaries for assessing the evidence and for the committee's decision-making. The scope includes:

  • a description of the technology and its claimed benefits

  • information about the disease, condition or clinical problem relevant to the technology

  • the regulatory status of the technology

  • the TOG's rationale for developing medical technologies guidance, which can include any relevant equality considerations

  • the decision problem to be addressed by the evaluation of the technology

  • a list of the professional and patient organisations involved in providing comments on the technology

  • a list of the societies or organisations to be invited to comment on the scope.

The scope may also include technical questions raised by the TOG or the programme team at selection stage, which may relate to the technology's ease of use or ability to generate the claimed patient or healthcare system benefits. The technical questions do not extend to a full technical evaluation of the device.

MTEP then makes the draft scope available for comment, and invites contributions from within 5 working days from the sponsor, the expert adviser(s), relevant patient and carer organisations, professional societies and other registered stakeholders. An interest can be registered at any time after the selection decision is published (section 4.4). The committee chair reviews the comments and agrees changes to the scope as appropriate. The chair and the programme director then agree the final scope before it is published on the NICE website. Once the scope if published, the medical technology formally becomes part of the committee's work programme and the website records that guidance development for this technology is in progress.

5.3 Equality considerations

MTEP was developed in accordance with the NICE equality scheme. At specific stages of guidance development the committee considers how medical technologies guidance may affect equality, including scoping and during its draft and final recommendations. Any potential equality issues raised and considered for a topic are recorded in an equality impact assessment. The programme or centre director approves the equality impact assessment and it is published with the scope and the final guidance. Any relevant equality issues that relate directly to the guidance topic and recommendations are also accounted for in the final guidance itself.

5.4 Sponsor's submission

The sponsor makes a submission to NICE using the template and guidance notes. The contents are based on the scope, which guides the selection of relevant clinical and economic evidence and analysis.

The submission is made in 2 parts:

  • Clinical evidence submission. This is submitted within 2 weeks of the scope being published, and includes all relevant clinical evidence and the decision problem.

  • Economic evidence submission with cost model. This is submitted within 6 weeks of the scope being published, and includes all relevant economic evidence with a model of relevant costs.

If the sponsor has developed an economic model, it must submit a fully executable electronic copy of the model to NICE with full access to the programming code. The submitted versions of the model and the written content of the evidence submission must match. NICE accepts executable economic models using standard software, specifically Excel, TreeAge Pro, R or WinBUGs. If the sponsor plans to submit a model in a non-standard package, it must inform NICE in advance. NICE and the external assessment centre will investigate whether the requested software is acceptable, and establish if either NICE or the external assessment centre need temporary licences for the non-standard software for the length of the assessment. NICE reserves the right to reject economic models in non-standard software.

Sponsors must sign a statement declaring that all material and knowledge relevant to the evaluation of their product has been disclosed to NICE. This includes unpublished data such as register data compiled for regulators or post-marketing surveillance. If the company is not the data owner (for example, register data), it should provide NICE with enough information for it to identify all relevant data owners.

To ensure that the process is as transparent as possible, NICE considers it essential that evidence on which the committee's decisions are based is publicly available. Unpublished evidence is accepted under agreement of confidentiality and is not made available to the public. Such evidence includes commercial-in-confidence information (confidential because its public disclosure may affect the commercial interests of a particular company) and academic-in-confidence data (confidential because the full data are yet to be published).

If the owner of any unpublished data included in the submission believes that the data should be treated as commercial- or academic-in-confidence, they should clearly state the rationale, taking into account the following principles:

  • Information and data that have been made publicly available anywhere in the world are not considered confidential.

  • When trial results are to be published in a journal at a date later than the first public release by NICE of documentation quoting data from these trials, a structured abstract relating to the future journal publication should, as a minimum, be made available for disclosure.

NICE asks data owners to reconsider restrictions on release of data either when the reason for the restrictions is not clearly explained, or when such restrictions would make it difficult or impossible for NICE to show the evidential basis for its guidance.

5.5 Assessment report

The external assessment centre reviews the sponsor's submission and prepares an assessment report.

The assessment report reviews and critically evaluates the sponsor's clinical and economic evidence and cost model. In some rare cases, if the external assessment centre considers that the sponsor's submission does not adequately address the issues in the scope, the centre may suggest to the MTEP team that further analyses should be done; these may include a new cost model. In these circumstances the additional analysis is usually done by the external assessment centre, as directed by the programme team, and forms part of the assessment report. If changes are made to the submitted cost model, the external assessment centre includes technical details of these amendments, and their impact, in the assessment report.

If necessary, the external assessment centre will approach experts in the technology when preparing the assessment report. These experts are listed in the report. The external assessment centre may also ask the sponsor questions when preparing the assessment report. The sponsor has the opportunity to review the report for factual accuracy.

External assessment centres are asked to declare conflicts of interest in line with the NICE policy on conflicts of interest.

5.6 Contributions from expert advisers

Depending on the scope and the characteristics of the technology, 1 or more expert advisers (see section 3.7) advise the committee, in person or by telephone, when the committee meets to develop its draft and final recommendations. The MTEP team produces a summary of their advice which is published alongside the draft and final guidance.

5.7 Contributions from patient and carer organisations

The public involvement programme always approaches patient and carer organisations to obtain their views on the technology. The committee may identify a need for detailed information from patient organisations or individual patients and carers (for example, an insight into living with the condition to which the technology relates or the use of the technology and/or comparator technologies). If the committee does not identify any specific questions or issues, a standard list of questions is used. The programmes presents all the information it has from patient and carer organisations to the committee when it meets to develop its draft recommendations on a technology.

5.8 Developing draft recommendations

The committee meets to develop draft recommendations on the technology under evaluation. It considers:

  • The assessment report and the sponsor's submission.

  • An overviewof the assessment report, prepared by the MTEP team. This may include the main features of the evidence base and the cost model, any additional analyses done, important uncertainties and the main issues the committee may wish to discuss (as well as the need for further research, if appropriate; see the MTEP methods guide for more details).

  • The contributions of the expert advisers

  • Important outcomes reported by patient and carer organisations, including outcomes not identified in the literature or by the expert advisers.

The committee meets in public, in line with NICE's commitment to openness and transparency. This allows stakeholders and the public to understand how evidence is assessed and interpreted.

In the public part of the meeting (part 1), the committee considers the evidence and commentary on the technology and invites expert advisers, the external assessment centre and the sponsor's representatives to respond to questions from the committee and provide clarification.

In the private part of the meeting (part 2), the committee considers any commercial-in-confidence or academic-in-confidence information and agrees its recommendations for use of the technology. The chair may ask the specific representatives to remain for some of part 2, specifically to respond to questions about confidential information in the submission. Otherwise part 2 of the meeting is closed to the public, including the expert advisers and the sponsor's representatives.

On occasion a meeting may be entirely public or entirely private (public if there is no confidential information and the committee is not making any decisions, and private if all the content of the meeting is confidential). This decision is made by the committee chair and the programme director and is published on the NICE website.

5.9 Draft guidance

When the committee has made draft recommendations, NICE issues a medical technology consultation document. This includes:

  • the draft recommendations

  • a brief description of the technology, the indications under review and its intended benefits

  • a summary of the evidence considered by the committee, including a summary of the advice from expert advisers and patient and carer organisations

  • the issues the committee took into account when it developed its recommendations

  • information about the implementation support tools that may be available for the guidance

  • research recommendations

  • related NICE guidance that has been published or is in development.

5.10 Consultation

Any person or organisation may comment on the medical technology consultation document. NICE informs the following groups when consultation starts and where to find the consultation document on the website:

  • national patient organisations

  • the Association of British Healthcare Industries and the British In Vitro Diagnostics Association, which in turn inform their members

  • relevant expert advisers

  • professional bodies of the relevant expert advisers, and professional bodies whose members might use the technology

  • the sponsor of the technology being evaluated.

In addition, people and organisations who have registered an interest on the website receive an automatic email alert when consultation starts.

NICE publishes the following documents on its website for the 4-week consultation period:

  • the medical technology consultation document

  • the scope

  • the sponsor's submission (with confidential information redacted)

  • the assessment report

  • the overview

  • the names and professional organisations of the expert advisers

  • a summary of comments from expert advisers and patient and carer organisations.

NICE makes an executable version of the cost model available to those who register an interest in the topic, on request and with the following conditions:

  • NICE releases the model as long as it does not contain information that was designated confidential by the model owner, or the confidential material can be redacted by the model owner without producing severe limitations on the functionality of the model.

  • The recipient must sign a confidentiality agreement and is advised that the model is protected by intellectual property rights, and can be used only for the purposes of commenting on the model's reliability and informing comments on the medical technology consultation document. The recipient agrees to these terms in writing before receiving the model.

Anyone may submit comments through the website, by email, fax or post. Comments longer than 20 pages are not normally accepted, other than at NICE's discretion in exceptional circumstances.

NICE is committed to having due regard to the need to eliminate unlawful discrimination and to promote equality, and fostering good relations between people with a characteristic protected by the equalities legislation and others. NICE encourages comments from all sectors of the community and specifically asks if there are any equality-related issues that need special consideration which are not covered in the document.

The committee particularly welcomes the following:

  • comments on the draft recommendations

  • notification of factual inaccuracies

  • additional relevant evidence, with bibliographic references if possible

  • views of patients, their carers and patient organisations on how well the technology works, including benefits or risks to the patient that were overlooked.

All comments are important and potentially influential in developing the guidance, including those that entirely support the draft recommendations.

Only people who comment during consultation can be involved in the resolution process.

5.11 Final guidance

After the consultation period ends, NICE collates the comments and presents them to the committee. Comments received after the consultation period are only shown to the committee if agreed in advance by the programme director, who consults with the chair and associate director.

The committee meets to discuss whether to amend its draft recommendations in view of the consultation comments. This meeting is held in public on the same basis as the first meeting.

If the committee's recommendations change significantly after consultation (for example, if important new evidence emerges during the consultation period), it is normally appropriate to reissue the consultation document for a further public consultation. The programme director makes this decision in consultation with the committee chair.

The committee agrees the final recommendations and submits them to NICE's guidance executive for approval. After approval, the guidance proceeds to resolution as outlined in section 6.

5.11.1 Late receipt of evidence

In exceptional circumstances (for example, if relevant information is published while the final guidance is being developed or because of comments received during consultation), NICE may choose to do further analyses. The external assessment centre (or another organisation commissioned by NICE) normally carries out these analyses before NICE circulates the final guidance for comment. The centre director makes this decision after discussion with the committee chair and the MTEP team. The decision is not taken lightly and is made to ensure that NICE is able to provide robust guidance to the health and social care system.

NICE reserves the right, while the final guidance is being developed, to refuse to accept evidence presented by the sponsor that could reasonably have been included in the sponsor's original submission.

5.12 Suspending or cancelling an evaluation

Appendix E lists the criteria for suspending or cancelling an evaluation. In summary, the criteria are:

  • the sponsor does not bring the product to market or withdraws it

  • reports of adverse events emerge

  • a technology is not appropriate for medical technologies guidance

  • the sponsor does not provide data for the evaluation according to the agreed schedule.

Information that has been made public before the suspension or cancellation decision will remain publicly available on NICE's website.