Tools and resources

How NICE clinical guidelines are developed: an overview for stakeholders, the public and the NHS

How stakeholders can get involved

How stakeholders can get involved

Stakeholder organisations can contribute to and comment on the clinical guideline at various stages during its development. A summary of the clinical guideline development process showing the key points of stakeholder involvement can be found in the How are NICE clinical guidelines developed? section.

Stakeholder involvement is managed by the Centre for Clinical Practice working with the PPIP at NICE.

Internal clinical guidelines

All tasks in this section described as being the responsibility of an NCC will usually be carried out by the Internal Clinical Guidelines Programme at NICE.

The scope

What is the scope?

The Department of Health or the NHS Commissioning Board gives NICE a short 'remit' for each clinical guideline. The next stage is to define exactly what the guideline will and will not cover. This process is called 'scoping', and the document containing this information is the scope.

The scope is drafted by the staff at the NCC, with input from the GDG Chair, the Clinical Adviser (if there is one), and the guidelines team and the PPIP team at NICE.

The scope gives an overview of what the clinical guideline will and will not include, and defines the aspects of care that it will cover. It may describe the following:

  • Groups of patients whose care is to be included or excluded – for example, particular age groups, or people with certain types of disease. Equality groups that may merit specific consideration (for example, specific ethnic groups or people with learning disabilities) are identified.

  • Where treatment will be carried out – for example, by GPs (primary care), in hospital (secondary care) or in specialist units (tertiary care).

  • Treatments to be included and excluded – for example, diagnostic tests, surgical, medical and psychological treatments, rehabilitation.

The scope may also include draft 'review questions', which specify in some detail the particular interventions to be compared and the health outcomes of interest.

The scope should also identify topics from other NICE guidance programmes that are relevant to the clinical guideline. For more information, see chapter 8 of The guidelines manual.

The stakeholder scoping workshop

We arrange a workshop for all registered stakeholder organisations before public consultation on the scope. Key staff from the Centre for Clinical Practice at NICE, the PPIP and the NCC attend, as well as the GDG Chair and (if applicable) the Clinical Adviser. People attending the meeting are sent a first draft of the scope, which is intended as a starting point for discussion. At the workshop we:

  • provide an overview of the NICE clinical guideline development process

  • describe how stakeholders can contribute to the guideline by:

    • informing their members and associates about GDG vacancies

    • commenting during the consultations on the draft scope and draft guideline

  • discuss the first draft of the scope and hear stakeholders' views on the key clinical issues that the guideline will cover.

What to do before the workshop

Each registered stakeholder can send one person to the workshop – please tell us who will be attending from your organisation. The person who attends should have a good understanding of the guideline topic. People attending from patient and carer organisations should have a good understanding of issues relating to the scope from a patient and/or carer perspective.

Note that each person is attending the workshop from their own perspective, and not to represent the views of their stakeholder organisation.

Key point

The stakeholder scoping workshop takes place before the public consultation on the scope. Note that expressing views at the workshop does not replace the formal scope consultation process. You should still send comments on the scope to NICE during the consultation.

Commenting during consultation on the draft scope

The NCC, GDG Chair, Clinical Adviser (if there is one) and NICE consider the issues raised at the scoping workshop and refine the draft scope for consultation. The draft scope is then posted on our website for a 4-week consultation period. We send a link to the document to registered stakeholders. Consultation dates are given on the website and in our monthly e-newsletter. Stakeholders should check the website regularly for any changes to timings.

We ask stakeholders to submit comments on the draft scope using the form provided. When commenting, it is important to take account of what NICE clinical guidelines can realistically be expected to cover (see box 3).

Some notes on how to comment during consultation are given in box 4 (these also apply to commenting on the draft guideline – see below).

Box 3 Considerations when commenting on the draft scope

  • NICE clinical guidelines apply to the NHS only, so they will not address the independent sector specifically. However, whenever an independent hospital, clinic or care home, social services or the voluntary sector is commissioned to provide NHS-funded care, it will be expected to adhere to NICE guidelines.

  • Guidelines are generally published within 2 years of the development process starting (1 year for short clinical guidelines), so that information is up to date at publication. If the scope is very wide it will not be possible to complete the work in this time, so the scope must be restricted to what can realistically be covered.

  • Guidelines will, if appropriate, address what drugs to use. However, it is assumed that prescribers will use the summaries of product characteristics[a] of medicines they are considering prescribing for individual patients. Therefore guidelines do not usually contain detailed information on contraindications and side effects.

  • The scope may specify or exclude certain groups of patients. It is helpful if stakeholders can comment on whether such inclusions or exclusions may discriminate on the grounds of age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion or belief, sex or sexual orientation.

[a] The summary of product characteristics for a drug includes information on uses for which the drug is licensed, dosages and contraindications. Summaries of product characteristics can be found on the eMC website.

Box 4 A guide to commenting on drafts of the scope and the guideline

When the draft scope or guideline arrives, you should:

  • circulate the draft within your organisation if appropriate, making it clear that it is for consultation and asking recipients to respond to you as the organisation's stakeholder contact (rather than directly to NICE)

  • prepare your response and return it to NICE, remembering to:

    • collate the comments into one response from your organisation using the form provided (do not make changes to the draft document)

    • include the name of your organisation in the response

    • return the response by the closing date

  • send comments electronically to the dedicated email address provided, adding your organisation's name in the subject box.

Please keep in mind the following:

  • We will accept only one response from each registered stakeholder organisation. If several responses are received, it may be unclear which represents the view of the organisation. We do not have the resources to acknowledge or respond to comments from several people within a registered stakeholder organisation.

  • All comments from registered stakeholders will be made public on our website, so do not include confidential information (such as information about individual patients).

  • Make sure that comments are constructive and clearly worded.

  • We will not consider comments that are not prepared according to these instructions, or that arrive after the deadline.

  • The Guidelines Coordinator (whose name is on the guideline page on our website) can answer questions on submitting comments.

Please see the document Protocol for managing guidance consultation comments for further details about how we deal with stakeholder comments received during consultation.

Key point

Comments on the draft scope must be submitted by the end of the 4-week consultation period, using the form provided by NICE. We notify registered stakeholders of the deadline for submitting comments.

The final scope

We collect together the stakeholder comments on the scope into a 'scope consultation table'. The NCC then finalises the scope, taking into account the comments received. We 'sign off' the final version of the scope, which is then posted on our website, along with the scope consultation table containing the NCC's responses to stakeholder comments.

The clinical guideline

Evidence from stakeholders

The GDG agrees the final review questions for the guideline from the key clinical issues defined in the final scope (draft review questions may also be included in the scope). Each review question takes account of issues that are important to patients, such as acceptability of treatment and patients' preferences for treatment options. There is more information about review questions, including examples, in chapter 4 of The guidelines manual. A search of the scientific literature is then carried out to answer the review questions.

For some of the review questions, the GDG and NCC may believe that their literature search has not found all the relevant information. For example:

  • the NCC may be aware that further research is being carried out

  • a drug or medical device may be relatively new

  • studies may have been published only as abstracts

  • the NCC may be looking for data on side effects, economic models or studies of the experiences of patients, carers or healthcare professionals.

In these situations, the NCC may call for evidence from stakeholders. They will specify the review question and the type of evidence they are looking for. These calls for evidence will be sent to all registered stakeholders, and may be made at any point during guideline development. Stakeholders are usually given 4 weeks to respond.

As well as published studies, stakeholders may submit relevant unpublished data or studies. Any confidential information should be clearly marked (for example, by using a highlighter pen, or the highlighter function in an electronic version). The NCC also asks stakeholders to complete a checklist that lists and identifies the location of all of the confidential information contained in their submission.

Box 5 summarises what may, and may not, be considered confidential by NICE.

Box 5 A guide to submitting confidential information

  • Data that may influence share price values ('commercial in confidence') or are 'intellectual property' ('academic in confidence'; that is, awaiting publication) may be considered as confidential.

  • Information marked as confidential should be kept to an absolute minimum – for example, just the relevant part of a sentence or a particular result from a table.

  • NICE will not agree to a whole study being designated as confidential. As a minimum, a structured abstract of the study or economic model will have to be made available for public disclosure during consultation on the clinical guideline.

  • Results derived from calculations using confidential data will not be considered confidential unless releasing those results would enable back-calculation to the original confidential data.

It is important that the amount of confidential information in a submission is kept to a minimum. At the least, a summary should be publicly available by the time the draft guideline is consulted on. We need to be able to justify the recommendations in our clinical guidelines on the basis of the evidence considered by the GDG, so the guidelines team and the NCC will work with the data owners to find an agreed solution to the balance between confidentiality and transparency[2].

The types of information listed in box 6 will not be considered by the GDG.

Box 6 Stakeholder material not eligible for consideration

  • Studies with weak designs if better-designed studies are available.

  • Promotional literature.

  • Papers, commentaries and editorials that interpret the results of a published paper.

  • Representations and experiences of individuals (unless assessed as part of a well-designed study or a survey).

Commenting during consultation on the draft clinical guideline

The GDG takes 12–18 months to develop a draft of the clinical guideline once the scope has been finalised. There is then a 6-week consultation period when registered stakeholders can comment on the draft guideline.

Short clinical guidelines

Development of the draft guideline takes 4–6 months, and the consultation period for the draft guideline is 4 weeks.

We notify registered stakeholders by email when the consultation draft of the guideline is posted on our website. Comments should be submitted using the form provided via the dedicated email address for the guideline. When commenting on the guideline, stakeholders should consult the final scope (on our website) to check what the guideline will and will not cover.

Stakeholders can comment on the full guideline (which includes the draft recommendations, as well as explanations of how the GDG has interpreted the evidence to make the recommendations) and/or the 'NICE guideline' (which contains just the draft recommendations and only brief supporting information).

Issues that stakeholders may wish to comment on during consultation include:

  • a general view (either positive or negative) of the quality and content of the draft guideline

  • points or areas that appear to fall within the scope but are not covered in the draft guideline

  • any gaps in the evidence that the recommendations are based on

  • potential inconsistencies in the interpretation of the evidence

  • disagreements with the interpretation of the evidence

  • the practical value of the guideline

  • wording (for example, could the recommendations be clearer, or the language more patient-centred; could the wording be perceived as excluding patients or groups of patients?)

  • whether the recommendations discriminate against some groups on the grounds of age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion or belief, sex or sexual orientation

  • how easy the recommendations will be to implement

  • the potential cost of implementing the recommendations.

Some notes on how to comment on the draft guideline are given in box 4.

Key point

There is a single consultation period when registered stakeholders can comment on the draft clinical guideline (6 weeks for standard guidelines and 4 weeks for short guidelines). Comments that are submitted late will not be considered.

Finalising the clinical guideline

We collect together all the comments from registered stakeholders in a 'guideline consultation table', and pass them to the NCC to consider. The NCC adds its responses to the consultation table.

In very rare cases, we may decide to hold a second consultation on all or part of the guideline (see section 11.3 of The guidelines manual for more details).

The NCC and GDG make changes to the guideline in the light of comments made during the consultation by:

  • registered stakeholders

  • external expert reviewers if applicable (see section 11.2.2 of The guidelines manual)

  • NICE staff – these include the PPIP lead, the implementation lead and the lead editor for the guideline, as well as technical advisers (including a health economist), the Guidelines Commissioning Manager and the Centre for Clinical Practice lead for the guideline.

Comments from NICE staff are entered into the guideline consultation table and are responded to in the same way as comments from registered stakeholders, but they are not posted on our website.

Publication

Once NICE's Guidance Executive has given final approval of ('signed off') the clinical guideline, the different versions (see box 2) are prepared for publication.

An advance copy of the final full guideline and a copy of the responses to stakeholder consultation comments are made available to registered stakeholders 2 weeks before the official publication date. This information is confidential ('embargoed') until the guideline is published. This allows stakeholders to prepare for publication, but it is not an opportunity to comment further on the guideline.

Registered stakeholders are also notified when the guideline is published.

Any stakeholder comments on the published guideline (other than those about errors that require correction) are addressed when the guideline is updated (see below).

After publication

Implementation support

Stakeholders are encouraged to use their networks and influence to encourage implementation of the clinical guideline at both national and local level.

We develop tools to help the NHS implement our clinical guidelines, and these are available on our website. These routinely include the following:

  • a baseline assessment tool

  • clinical audit tools

  • a costing report and costing template, or a costing statement.

We may also produce targeted implementation tools aimed at addressing specific learning or education needs of staff and organisations. These can include:

  • learning and development slide sets

  • clinical case scenarios

  • podcasts

  • training plans

  • online educational tools

  • examples of how NICE guidance has been put into practice

  • shared decision aids.

Tools may be produced jointly with other organisations, such as professional or patient groups.

See chapter 13 of The guidelines manual for more information about implementation support.

Comments and correcting errors

Comments on published clinical guidelines should be sent to us at nice@nice.org.uk.

Sometimes a comment after publication may highlight a potential error in a clinical guideline. Corrections or changes to a published clinical guideline will be made if an error:

  • puts patients at risk, or impacts on their care or

  • damages NICE's reputation or

  • significantly affects the meaning of the recommendation.

If it is necessary to correct an error in a published guideline, we will follow our internal policy for dealing with errors. The individual or organisation who reported the error will be contacted in writing, and we will explain our rationale for the decisions and actions taken.

If a correction is to be made, a notification is put on the guideline's 'home' page on the NICE website. Depending on the nature and significance of the error and the time since publication of the guideline, stakeholders may also be notified in writing (usually by email). The relevant web-based documentation is corrected, and this is also highlighted on the guideline's home page on the NICE website.

Reviewing and updating clinical guidelines

There is a formal process for reviewing and updating clinical guidelines, which is managed by NICE and the NCC. Chapter 14 of The guidelines manual gives details of this process.

Suspension of routine 3-year reviews

NICE's Senior Management Team, with the approval of the NICE Board, has suspended the routine review of the need to update clinical guidelines 3 years after their publication. This suspension is for 2013 and 2014. A new process for reviewing and updating guidelines is being developed.

Usually a guideline is considered for updating 3 years after publication. An exceptional update may be carried out before the usual 3 years if significant new evidence emerges.

In order to be brought up to date, a guideline may require:

  • an update of the whole guideline

  • an update of part of the guideline

  • no update.

Other possible options are:

  • Transferring the guideline to a 'static list' This will happen if the recommendations are unlikely to change in the foreseeable future, and so no further update is planned.

  • Withdrawing the guideline. This will be the case if the recommendations no longer apply, but the guideline is not a sufficient priority for updating. This decision will be consulted on with stakeholders.

For an update of a whole guideline, the usual process for producing and consulting on the scope is followed.

If only part of a guideline is being updated, there are 2 possible scenarios:

  • some recommendations need to be updated and/or

  • new clinical areas have been identified that require new recommendations.

In each case, a new scope is prepared and consulted on using the usual process. The scope will make clear exactly which sections of the guideline are and are not being updated.

The time needed to undertake an update is agreed between NICE and the NCC.


[2] For further details see the Agreement between the Association of the British Pharmaceutical Industry (ABPI) and the National Institute for Clinical Excellence (NICE) on guidelines for the release of company data into the public domain during a health technology appraisal.


This page was last updated: