Interim methods guide for developing good practice guidance

9.1.1 Clinical studies

Before acquiring papers for assessment, the NICE project team sifts the evidence identified in the search to discard irrelevant material. First, the titles of the retrieved citations are scanned. Those falling outside the GPG topic are excluded. A review of the remaining paper identifies those that are clearly not relevant to the review questions and are therefore excluded.

Once sifting is complete, full versions of the selected studies are requested for assessment (see section 8.6). Full versions of the studies failing to meet the inclusion criteria are excluded; those meeting the criteria are assessed. Because there is a potential for error and bias in selecting the evidence, double sifting (sifting by 2 people) of a random selection of abstracts is performed periodically (Edwards et al. 2002). Any doubts about inclusion are resolved by discussion with the Guidance Development Group (GDG) before the results of the study are considered. The quality of individual research studies is assessed using an appropriate NICE methodological checklist and process for the type of study under consideration. See links provided below:

The checklists have been linked to the appendices of the guidelines manual. The term 'guideline' can be replaced with 'guidance' for purpose of this methods guide.

9.1.2 Legislation and policy

Relevant legislation or NHS policies may be identified in the literature search and used as evidence to develop the GPG. This does not need assessment, given the nature of the source.

Assessing the robustness of NHS policies for inclusion as evidence is discussed at GDG meeting 1. The NICE project team use a standard template for this assessment.

9.1.3 Published guidance

Relevant published guidance from other organisations may be identified in the search for evidence.

Any guidance produced by a NICE-accredited guidance producer is not subject to further quality appraisal having already undergone formal process appraisal by NICE.

Guidance not produced by a NICE-accredited guidance producer is assessed for quality using the AGREE II (appraisal of guidelines research and evaluation II) instrument (Brouwers et al. 2010) to ensure there is sufficient documentation to be considered. There is no cut-off point for accepting or rejecting guidance and each GDG needs to set its own parameters. These are documented in the methods section of the GPG, along with a summary of the assessment. The results are presented as an appendix to the guidance.

Reviews of evidence from other guidance that considers questions formulated by the GDG and the NICE project team may be considered as evidence if:

The GDG creates its own evidence summaries or statements. Evidence tables from other guidance are referenced with a direct link to the source website or a full reference of the published document. The GDG formulates its own recommendations, taking into consideration the whole body of evidence.

9.1.4 Oral testimony and written evidence

During development of the GPG, if there is limited primary research available as evidence on which to produce recommendations on, oral or written evidence may be needed. The written submission of evidence identifies the evidence most relevant to current NHS practice and the GPG topic, therefore informing GDG decision-making. Inclusion criteria for the evidence specifies relevant population and interventions for the review question, to support filtering and selection of oral and written submissions to be included for GDG review.

9.1.5 Other evidence

Given the nature of the GPG and the topic being covered, other sources of relevant evidence such as editorials, reports, audits, qualitative studies (which may include patient experience), surveys and standard operational procedures to guide recommendations may be included. Such evidence is assessed for the reasonableness and rigour of the process used to develop the source of evidence in question and their relevance to the topic under consideration.