Developing NICE guidelines: the manual

Identification of key priority areas for monitoring

We have organised our guidelines portfolio into topic suites and within each suite key priority areas (KPAs) will be identified that consist of clusters of recommendations for active monitoring and associated updating. For details about topic suites and how we prioritise topics, see the webpage on maintaining and updating our guideline portfolio.

This appendix describes the interim principles for surveillance of KPAs using active evidence monitoring and active system monitoring.

We are in the process of defining KPAs and non-KPAs for topic suites. When KPA proposals for each topic suite have been validated, each KPA will be assessed to decide the most appropriate monitoring approaches at that time.

We will react to all intelligence received for other guidelines not in topic suites and will continue tracking ongoing studies in the surveillance internal system and central hub.

Monitoring approaches

There are 3 approaches for monitoring KPAs that will be used individually or in combination (including switching from 1 approach to another):

1. Active evidence monitoring of ongoing studies: systematically searching for ongoing studies that are assessed when results are published.

2. Active evidence monitoring of newly published studies: continuous searching of newly published evidence.

3. Active system monitoring: systematically collating and interpreting intelligence from various health and care system sources (for example safety alerts from the Medicines and Healthcare products Regulatory Authority [MHRA], HSIB and others; other intelligence from faculties, enquiries from the public, and working with our system partners to identify areas of change).

For non-key priority areas (non-KPAs), intelligence and evidence submitted by stakeholders and the public will be recorded and used to establish a baseline for when to consider re-prioritisation of topics. It may also be valuable to reactively set up monitoring approaches when intelligence and evidence suggest a potential change or impact to current non-KPAs.

Choice of monitoring approaches

The choice of monitoring approaches includes an element of judgement. Decisions will be made by considering all the information collated through validation of the KPA, including:

Review questions and search protocols for ongoing studies and published evidence

A review question for each KPA will be produced for both active monitoring of ongoing studies and for active monitoring of newly published evidence. Typically, the original guideline review question(s) underpinning recommendations in the KPA will be used. Where this is not possible, a review question will be adapted to cover the full KPA. This may be necessary if new evidence is emerging in an area outside the scope of the original evidence review (for example, an identified gap in the topic area). Any adapted review questions will always include outcomes that were considered critical or important by the original guideline committee in the original review protocol.

Using the review question, a search protocol will be drafted using the PICO, or SPICE approach (the guideline manual has further information on developing review questions). This search protocol provides the basis for any systematic searching; detailing approaches, sources and limits (the guideline manual has further information about developing search protocols).

Active evidence monitoring: systematic searching for ongoing studies

If a KPA is suitable for systematic searching for ongoing studies, the information services team will carry out searches for ongoing studies using the key trials registries. These may include but not be limited to 1 or more of:

A full list of sources of ongoing studies that could be used for monitoring can be found in the appendix on sources for evidence reviews.

Titles and details of the identified studies will be screened against the inclusion criteria defined in the KPA search protocol.

Studies meeting the inclusion criteria will be added to our internal system for monitoring ongoing studies and results will be assessed when the study publishes.

This search process will be repeated systematically to ensure that we identify all newly registered ongoing studies that are relevant to a KPA. The frequency of the search interval will be decided for each KPA individually dependent on the volume of relevant results identified; a higher volume of relevant ongoing studies will result in more frequent searches. Before each search, the search protocol will be assessed to ensure it remains current. The need for searches to be continued will be informed by our re-prioritisation assessment for each KPA.

Active evidence monitoring: continuous searching of newly published evidence

If a KPA is suitable for continuous searching of newly published evidence, multiple bibliographic databases from those listed in the guideline manual section on sources will be searched. This will typically include Medline, Embase and the Cochrane Library.

A title and abstract sift will be carried out against the inclusion criteria defined in the KPA search protocol. The search process will be repeated systematically to ensure that we identify all newly published evidence relevant to a KPA.

The frequency of the search interval will be assessed and decided individually for each KPA. The volume of relevant studies, and the number of those impacting on recommendations will be used to decide on the interval between searches after the initial search. If a high number of relevant studies are identified, a fixed interval between searches from 1 to 3 months will be used. If the yield is low, a test-adjust-phase-out approach will be used resulting in the interval gradually increasing from 3 to 12 months. Decisions to change search frequency are also dependent on the impact of new evidence and other intelligence on recommendations in the KPA (see the section on active system monitoring below).

After 12 months of searches with a low yield of relevant and impacting studies, searching will be stopped. This decision will inform our re-prioritisation assessment.

Searching will also be stopped if an accumulation of studies is identified that has sufficient impact to trigger an update to 1 or more recommendations. This will be assessed alongside other intelligence identified through alternative monitoring methods (see the impact assessment section below).

Active system monitoring: systematically collating intelligence from various sources

If a KPA is suitable for systematic collation of evidence from the health and care system, then all intelligence received or actively acquired about a related KPA will be logged in a central hub. This may include:

We will consider these individual pieces of information in 2 ways, firstly when the information arrives it will be assessed to determine the impact on the related KPAs, and if any changes need to be made because of the new intelligence. Secondly, we will assess the cumulative body of intelligence (including evidence monitoring if the KPA has more than 1 monitoring approach), to see if the cumulative intelligence and evidence suggest a need to change recommendations relating to a KPA.

The need to continue active system monitoring will be assessed during re-prioritisation assessments for each KPA. Active system monitoring will also be stopped if an accumulation of intelligence and evidence is identified as sufficient to trigger an update to 1 or more recommendations. This will be assessed alongside other intelligence identified through alternative monitoring methods (see the impact assessment section below).

Impact assessment

The publication and accumulation of relevant ongoing studies, or the accumulation of relevant newly published evidence over time will be assessed for their impact on recommendations in the KPA at each search timepoint, alongside other intelligence collated from active system monitoring. An impact may be shown if, for example, a large well conducted study is identified, or a significant volume of studies is identified consistently reporting superiority, ineffectiveness or harms for an intervention, or a change of significant national policy. Impact assessment is topic dependent and includes an element of judgement. See the appendix on surveillance decision framework and multi-criteria decision framework for deciding whether to develop or update recommendations and which methods to use.

Reactive monitoring of non-KPAs

For non-KPAs, we will conduct reactive monitoring of the intelligence and evidence and record in a central hub. This could include published studies, or information from any of the sources listed in the system monitoring section above. Information that is considered to be a safety issue related to a non-KPA will be assessed, and a decision will be made about the need to update the related recommendations. Information that is not considered to be a safety issue will be collated and assessed during the re-prioritisation of non-KPA.

Information on any changes to KPAs and non-KPAs during re-prioritisation will be available on the topic suite hub webpage.