NICE process and methods

Appendix A: Searching for and selecting evidence for use in NICE decision aids

Appendix A: Searching for and selecting evidence for use in NICE decision aids

In exceptional circumstances a targeted literature search is undertaken by NICE's information services. This step will be considered only if: 

  • evidence to address an essential component of the scope has not been identified within the evidence reviews or submission used in the development process for the guidance, or 

  • the relevant part of the guidance was last updated more than 3 years previously or more recent evidence has been highlighted to NICE's guideline surveillance process, or 

  • the topic for an externally commissioned decision aid has not been the subject of NICE guidance published in the previous 3 years. 

Reviewing surveillance reports

In any of the circumstances described, surveillance reports for the relevant guidance are first reviewed, if they have been conducted. A targeted literature search is not undertaken if either of the following circumstances apply and there is also no other reason to think that more recent evidence is available:

  • evidence which addresses the relevant components of the scope is contained within the evidence reviews or submission used in the development process for the guidance and the surveillance review(s) indicate that no more recent relevant evidence has been published within the past 3 years, or

  • the surveillance review(s) identify evidence which addresses components of the scope published within the past 3 years.

Searching for evidence

If, after available surveillance reports have been reviewed, an evidence search is needed, one or more search questions are developed by the decision aid development team that describe the evidence needed to address the relevant components of the scope. These specify the population, intervention, comparators and outcomes (PICO). NICE's information services do a literature search according to the agreed scope and PICO. The search strategy and quality assurance of the search process is recorded.

Selecting the evidence

Evidence identified from the literature search is reviewed to either:

  • confirm that no more recent or better evidence than that which is in the evidence reviews or submission is available, or

  • identify evidence to address the relevant components of the scope.

First sift

The first sift reviews the title and abstract of the study against the scope and PICO and removes evidence of low relevance. This may include non-English language studies, conference abstracts or studies that have not been published in full (because these cannot be critically appraised).

Second sift

The second sift of full papers further excludes articles that do not meet the criteria in the scope and PICO.

Prioritisation

When all relevant studies have been identified, the best available evidence is selected, taking into account:

  • Applicability: the ability to generate absolute rates of benefits or harms from the study results.

  • Risk of bias: this includes an assessment of the quality of the study size and design. For comparisons between options, systematic reviews of randomised controlled trials (RCTs) are prioritised first, followed by single RCTs and then large observational studies.

  • Precision: this includes a review of the 95% confidence interval around the best estimate of the absolute effect.

  • Consistency: this includes considering similarities and differences in relative effects if several studies are identified which potentially provide evidence.

  • Directness: how closely the population in the study matches the population in the aim for the decision aid and the study interventions match UK practice.

If evidence is identified which might impact on the guidance recommendations, the surveillance team is informed and the guideline surveillance process is followed.

The rationale for including or excluding evidence is recorded, but a narrative evidence review is not written. All evidence used in the decision aid is agreed with the project group.

ISBN: 978-1-4731-5355-4