Tools and resources

Developing NICE guidelines: the manual – appendices A to P

Appendix N: Surveillance decision framework and multi-criteria decision framework for deciding whether to develop or update recommendations and which methods to use

Appendix N: Surveillance decision framework and multi-criteria decision framework for deciding whether to develop or update recommendations and which methods to use

The following frameworks can be used to decide whether to develop or update recommendations and which methods to use. They should be used in conjunction with the appendix on interim principles for methods and processes for supporting digital living guideline recommendations. This appendix is a living document that is reviewed on a regular basis.

Guideline details

Guideline number

For example, NGXXX
Topic area

-
Date signal from living monitoring received

Day Month Year
Currently recommended

Summary of recommendations in this area in the guideline or suite
Signals from monitoring

Brief summary of the signal sources, such as evidence and other intelligence
Surveillance decision framework

Strength, significance and volume of new evidence

Triggers from monitoring activity: significance of new evidence (including health economics evidence) or issues identified (such as bias and sample size).

Are there ongoing trials or studies that we should wait for? Are they big impact trials? Are they likely to report within the next 6 months?

Does the new evidence bridge a gap identified in the guideline (do they address the recommendations for research or an important new topic area)?

Does the new evidence address previous lack of or limited evidence supporting existing recommendations?

Certainty of impact on existing recommendations

Original recommendations: was the area evidence-free or was there a high volume of evidence?

Does the new evidence have information on important outcomes, such as patient outcomes?

Does the new evidence apply to subgroups that were not addressed by current recommendations or where evidence was previously lacking?

From system feedback: are there implementation challenges with the existing recommendations?

Safety issues

Does the new evidence suggest that a recommendation may be unsafe or harm patients?

Resource impact and health economics considerations

Is there a resource or health economics impact that may change the direction of recommendations?

Is a health economics evaluation required? What was the evaluation for the current recommendations?

Health inequality issues

Are there health inequality issues that could be addressed by evidence review?
Multi-criteria decision framework: proportionate approach to planning

Complexity and system priority

  • Is this a complex topic area?

  • Is this a system priority? (For example, does it reduce waiting lists or current variation in practice?)
Methods of updating

  • Is this topic area part of 'living' systematic review? Are there good quality external evidence reviews? (For example, Cochrane)

  • Does this need full systematic review and evidence synthesis? Does current evidence (old and new) give certainty?

  • Are there good-quality external guideline recommendations? Could we collaborate with an external organisation?

  • Update with or without full systematic review?
Committee input

  • Do we need committee's input?

  • Committee's input: virtual or in-person meeting, email only or other efficient approach?
External validation

Full consultation or targeted engagement? Or no consultation? Give rationale.
Currency and overlap

  • When was the topic area or review question being considered last reviewed or updated?

  • Does the topic area overlap with other suites, independent guidelines, or foundational guidelines? Or other NICE guidance?
Other contextual intelligence

  • Should other factors or intelligence (not covered by the above) be considered? This could include ad hoc conversations with key people, or intelligence from enquiries, or Implementation and Insight Programme feedback.
Final decision

Update with

  • Full systematic review and evidence synthesis

  • No systematic review

  • External systematic review

  • Targeted review

  • Living systematic review

  • Collaboration with external developer

  • Curated content from external developer

  • Stand down recommendations with no further action

Note: there could be combinations of the above
Health economic model

–
Committee or topic expert input

  • Input through virtual or in-person meetings, or by email

  • No need for committee or topic expert input
Validation

  • Public consultation

  • Targeted engagement

  • No consultation

Note: a mix of validation options may be used, but will need to be justified in each case because this may be resource intensive

This page was last updated: