NICE process and methods

13 Ensuring that published guidelines are current and accurate

13 Ensuring that published guidelines are current and accurate

This chapter describes the process and methods for checking that published guidelines are current and deciding whether updates are needed (surveillance).

13.1 Aims of surveillance

The aim of surveillance is to check that guidelines are up to date. This is done by exploring if there is any new evidence to contradict, reinforce or clarify guideline recommendations. Surveillance also identifies new interventions that may need to be considered within the guideline. Finally, it explores changes in context that may mean modifications are needed, for example, changes in policy, infrastructure, legislation or costs.

NICE has more than 350 published guidelines, so the number of checks needed is considerable. A proactive approach (with an assessment of priority) is used to respond to events that may impact guideline recommendations at any time after guideline publication (for example a safety alert, or publication of a key study)

13.2 Proactive surveillance

Some topic areas change frequently, and this increases the risk of guidelines having out-of-date recommendations. NICE takes a proactive approach to surveillance, and monitors key events (such as ongoing studies) that are judged to be relevant to the guideline.

Events are identified through constant intelligence gathering. This starts during initial guideline development, as the guideline committee and stakeholders can flag up future events that need to be monitored for impact. Ongoing studies are typically identified through discussions with the National Institute for Health Research. This approach means that NICE can quickly identify changes in the evidence base, and assess the impact on recommendations and the need for any changes.

An event that could affect the guideline could include:

  • publication of a study that is directly relevant to NICE guidance and has the potential to affect recommendations

  • substantial changes in policy or legislation (an example includes changes to the UK physical activity guidelines by the Chief Medical Office)

  • development of a related piece of NICE guidance that contradicts recommendations in another NICE guideline

  • withdrawal of a drug from the market, or a clinically significant drug safety update from the Medicines and Healthcare products Regulatory Authority (MHRA) or the Commission on Human Medicines.

This list is not exhaustive and individual events will be considered on a case-by-case basis. To make the most efficient use of resource, events are triaged, to determine whether surveillance assessment is needed

  • If an assessment is needed these are prioritised based on:

    • safety (always prioritised first)

    • health and social care system priorities

    • burden on services

    • population impact

    • potential impact on addressing health inequalities

    • evidence base: is it changing frequently and what is the degree of uncertainty?

    • what value NICE could add by incorporating the new information into a guideline.

13.3 Surveillance assessment process

The NICE surveillance programme considers how an event could affect a guideline. This involves checking how the event could affect the guideline recommendations, and taking feedback from topic experts in the area. The check may include intelligence gathering and literature searches, if needed.

Stakeholders are not normally consulted on the decision to update (or not update) a guideline in response to a surveillance check. This is because these checks usually focus only on one important event and potentially a small section of a guideline. The decisions are published on the NICE website.

If the guideline needs updating, registered stakeholders are informed of the planned approach.

Topic expert engagement

Topic experts (including members of NICE's Expert Advisers Panel) are invited to participate in surveillance. They provide their views about how an event affects the recommendations, and their knowledge of recent developments in the topic area. If the response from topic experts is limited, or further specialist input is needed, we may seek input from other experts. This could include other government organisations, or representatives from a Quality Standards Advisory Committee.

Intelligence gathering

If needed, additional intelligence may be sought. This might include:

  • feedback from internal teams within NICE who have expertise in the topic area (for example the NICE medicines or social care teams, if these are major areas in the guideline)

  • asking stakeholders for their views, including organisations representing the interests of patients, people using services, carers, and the public

  • external queries and comments received since publication of the guideline (these are collated in an issues log for consideration during surveillance)

  • related NICE guidance and quality standards (including placeholder statements in NICE quality standards) developed since the guideline was published

  • information about guideline implementation, including evidence derived from analysis of primary data on the uptake of recommendations

  • information about important ongoing studies in the area covered by the guideline (identified through searches of trial databases)

  • changes in licensing status of medicines

  • updated or new national policy.

Literature searching

If needed, published evidence is identified by searching a range of bibliographic databases relevant to the topic. Surveillance searches generally use the same databases that were used during the development of the original guideline. The sources searched may vary depending on the topic. In general, MEDLINE, MEDLINE in Process, Embase, Cochrane Database of Systematic Reviews and Cochrane Central Register of Controlled Trials (CENTRAL) would be considered.

Search approaches

The search approach will vary between topics. The following search approaches can be used:

  • population or population/intervention search as needed for the guideline scope with:

  • focused searches for a specific question or a new question, meaning that the study type searched for (RCTs or observational studies) should reflect the type expected to address the question

  • citation search forward or back (this option would be supplemented with either a restrictive full scope search or focused searches).

Other considerations

It may be appropriate to consider setting limits for the searches, which could include, but are not limited to:

  • study design using appropriate search filters

  • date

  • location

  • populations and subpopulations

  • intervention

  • service delivery aspect

  • prognostic factors.

Search period

The search period will start at the:

  • end of the search for the last update of the guideline

  • end of the search for the last standard check.

The search date ends on the date the search is conducted.

Decision making

Proposals on the need to update a guideline are based on:

  • an assessment of the event and any other relevant evidence published since guideline publication (abstracts of primary or secondary evidence)

  • information from topic expert engagement

  • if relevant, intelligence gathering and feedback from stakeholder consultation.

The decision also includes an element of judgement.

The possible decisions are:

  • update (reviewing the evidence and producing either a full update that replaces the original guideline, or a partial update of defined sections of the guidelines)

  • no update (this also includes a decision to defer the update until later)

  • amend (making changes to the guideline without reviewing the evidence)

  • withdraw (either some recommendations or the whole guideline).

Unless the guideline is fully updated or completely withdrawn, there may also be a refresh (for more information on refreshing, see the section on refreshing the guideline recommendations in the chapter on updating guideline recommendations).

When a guideline is being updated, the original scope may be used (unchanged), the original scope may be modified (for example, where new areas have been identified that require an extension to the scope) or a new scope may be developed. See the chapters on the scope and on updating guideline recommendations for further details.

Stakeholder consultation

Consultations on proposed decisions will be held if there is value in doing so (for example, when it is not clear to NICE if an update is needed or not). The consultation period will usually be 2 weeks.

Signing off the final decision

All surveillance proposals go through a validation and approval process at NICE, which includes signoff by the associate director and centre director.

13.4 References and further reading

Alderson LJ, Alderson P, Tan T (2014) Median life span of a cohort of National Institute for Health and Care Excellence clinical guidelines was about 60 months. Journal of Clinical Epidemiology 67: 52–5

Shekelle P, Eccles MP, Grimshaw JM et al. (2001) When should clinical guidelines be updated? British Medical Journal 323: 155–7

Shekelle PG, Ortiz E, Rhodes S et al. (2001) Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated? JAMA 286: 1461–7