NICE process and methods

5 Production

5 Production

5.1 Equality and diversity considerations

ESUOMs are developed in accordance with the NICE equality scheme (available from http://www.nice.org.uk/aboutnice/howwework/niceequalityscheme.jsp).

5.2 Process and timescales

ESUOMs are not formal NICE guidance and therefore are not subject to the same intensity of process as other NICE products.

The table below shows the key steps in the production of ESUOMs.

Table 1: Key steps of production of an ESUOM with timelines

Key step

Responsible party

Completed by

Topic selected and referred to supplier

NICE

Week 0

Scope topic

Supplier

Week 1

Contact manufacturer and/or regulator for data

NICE

Week 1

Literature search

Supplier

Week 1

Searching for evidence

Sifting and selecting the evidence

Appraising and categorising the evidence

Supplier

Week 1

Week 2

Week 2

Authoring the ESUOM:

Produce initial draft of ESUOM

Supplier

Week 4

Preliminary technical check of initial draft by supplier

Supplier

Week 5

Review of draft ESUOM:

Initial draft sent to manufacturers, regulators and specialist commentators for review

Initial draft of summary for patients sent to Patient and Public Involvement Programme

Supplier

Week 5

Review comments received and revised draft produced

Supplier

Week 7

Quality assurance of the ESUOM:

Technical and editorial check of content by supplier

Supplier

Week 7

Revised draft sent to NICE

Supplier

Week 8

Technical check of content by NICE senior adviser

NICE

Week 9

Editorial check of content by NICE Editorial team

NICE

Week 10

Final check of content by Medicines and Prescribing Programme Director

NICE

Week 10

Sign off and publication of the ESUOM:

Guidance Executive sign off

NICE

Week 11

Inform manufacturer of date that ESUOM will be published

NICE

Week 11

Publication on NICE website

NICE

Week 12

5.3 Scoping of individual topics

The Medicines and Prescribing Centre and the contracted external supplier meet to scope each ESUOM topic. The scoping confirms the following.

  • Key contacts at the pharmaceutical manufacturer

  • Key contacts at the Medicines and Healthcare products Regulatory Agency (MHRA) or European Medicines Agency (EMA), to obtain evidence held on file which is not confidential

  • Specialist commentators (via the New Medicines Community of Practice and other existing NICE networks)

  • Commentators for the summary for patients, in liaison with the NICE Patient and Public Involvement Programme

  • Terms for a literature search to identify published clinical trial data that reflect the indication for the unlicensed or off-label medicine

  • Arrangements for identifying:

    • regulatory status

    • relevant published trials, or other data on the unlicensed or off-label medicine

    • evidence of clinical effectiveness for the UOM in the condition under consideration

    • safety issues, encompassing key adverse drug reactions, precautions and contra-indications providing an indication of frequency of the adverse effects where possible

    • incidence and prevalence of the condition, what treatment alternatives exist and an estimate of current drug usage

    • cost of the medicine and the cost of alternative treatment options.

5.4 Contacting the manufacturer and medicines regulator

NICE informs the manufacturer[4] of the medicine that an ESUOM is going to be produced and asks the manufacturer to provide data to support the production of the ESUOM.

The NICE also contacts the MHRA (or EMA, as appropriate) to ask them for any evidence they hold on file that is not confidential.

Both the manufacturer and the MHRA (or EMA) are sent the timelines for the production of the ESUOM, including the deadlines for receipt of data and the expected dates for comment on a draft of the summary.

5.5 Literature search

5.5.1 Searching for evidence

A literature search is produced by the contracted external supplier according to the agreed scope and strategy. The search strategy is documented and included in an appendix to the published ESUOM. Quality assurance for the search process is also documented.

5.5.2 Sifting and selecting the evidence

The contracted external supplier sifts the final set of search results using the title and abstract of each article, applying first exclusion and then inclusion criteria.

These criteria include the basic criteria as set out below:

First sift

This process removes evidence based on the following exclusion criteria:

  • articles of poor relevance against search terms

  • publication types that are out of scope:

  • non-English language abstracts, or non-English language studies with English abstract

  • conference abstracts

  • Cochrane, or other review protocols

  • articles without either abstract or full text available.

Second sift

This sift of evidence includes relevant primary research that addresses the use of the medicine within the defined indication under review. Where robust randomised controlled trials or systematic reviews are available, then they form the basis of the review. However, given the nature of the work, the best available evidence on which to produce the evidence summary may include evidence other than randomised controlled trials.

The contracted external supplier records the reasons for inclusion and non-inclusion based on the second sift, as well as a 'long list' of those studies that are excluded from the first sift.

5.5.3 Appraising and categorising the prioritised evidence

The contracted external supplier prioritises the evidence for critical appraisal and records the reasons for non-inclusion of evidence. The full text of the prioritised evidence is appraised using an assessment form suitable for the type of evidence, which is provided by NICE.

5.6 Authoring of the ESUOM

The contracted external supplier drafts the ESUOM using a standard template, which includes the following sections:

  • Title and contents page

  • Date and version control information

  • Summary for healthcare professionals, including:

    • Regulatory status

    • Clear summary statements of the evidence

    • Summary of the relative risks, benefits, strength of evidence, regulatory status and typical treatment costs

  • Details of the intervention, including:

    • General information about disease or condition, incidence/prevalence and overall survival rate

    • Alternative treatment options with links to relevant guidance/evidence

  • Review of available evidence, with relative strengths and weaknesses of evidence for:

    • Efficacy

    • Safety

    • Regulatory status

    • Cost and comparative costs of alternative options

    • Current drug usage

  • References

  • Appendices:

    • Search strategy

    • Evidence selection process and criteria used to inform this

  • Summary for patients

5.7 Review of the draft ESUOM

The contracted external supplier sends the draft ESUOM to the identified external specialist reviewers, the manufacturers and the drug regulators for review. The draft is also sent to the expert patients or commentators, from patient groups identified by the PPIP, to provide comments on the summary for patients.

Any comments received are recorded by the contracted external supplier and considered within the production of the revised draft. Actions are also recorded.

5.8 Quality assurance of the ESUOM

Initial quality assurance of the ESUOM is carried out by the contracted external supplier. This will involve a detailed check of all content, to ensure all sections of the document contain statements and conclusions that are fair and balanced. They must, accurately reflect the evidence reviewed and be substantiated by an explicit and appropriate source of evidence. This will be carried out to a checklist provided by NICE. A further check for clarity, grammar, spelling and style is also undertaken by the contracted external supplier. All drafts and any changes to drafts are recorded for audit purposes.

The revised draft is then sent to the Medicines and Prescribing Centre, who, in conjunction with the NICE Editorial team, review it and produce a final draft. Once sign-off is received from the Programme Director of the NICE Medicines and Prescribing Centre, NICE Guidance Executive reviews the ESUOM, and if appropriate, approves the ESUOM for publication, ensuring that due process has been followed in its development.

5.9 Publication of the ESUOM

The final ESUOM is uploaded and made available online through the Medicines and Prescribing Centre page of the NICE website.

The NICE Communications team develops a communications plan for the ESUOM, together with the Associate Director within the Medicines and Prescribing Centre, and is responsible for disseminating the ESUOM once it has been published.


[4] For some unlicensed or off-label medicines, more than one manufacturer may be involved in the development of the drug. On these occasions, all relevant manufacturers based in the UK will be contacted for information.