Interim methods guide for developing good practice guidance

8.1.1 Devising a search strategy

The evidence underpinning the GPG is derived from review questions arising from the scope and scoping workshop with stakeholders. The NICE project team develops the review questions into search terms for request using a PICO (population, intervention, comparator and outcome) framework (see section 4.4.1).

The search strategy is constructed by NICE guidance information services^[3] for terms relating to the population; this can be combined with terms relating to the interventions and comparators (if any) to be evaluated. Not all components of a search term or review question are always mentioned in the abstracts or subject headings of database records – outcomes are often not mentioned. Therefore, these components may not be included when developing a strategy. For GPGs being updated, previous strategies can be used to inform the search strategy design.

8.1.2 Identifying search terms

Search strategies usually consist of a combination of subject headings and 'free-text' terms from the titles and abstracts of relevant studies. Search terms used are derived from the review questions identified during the scoping process. Subject headings are used to identify the main theme of an article. When identifying subject headings, variations in thesaurus and indexing terms for each database are included; for example, MeSH in MEDLINE and the Cochrane Library, and Emtree in Embase. Free-text terms may include synonyms, acronyms and abbreviations, differences in terminology across national boundaries, different spellings, old and new terminology, brand and generic drug names, and lay and medical terminology. Misspellings or typographical errors may also affect a search, particularly with records being indexed, for which there may be only a title and no abstract or subject headings.

8.1.3 Sensitivity and precision

The key attributes of a search strategy are:

Both attributes are influenced by the time period covered and by the search terms used. Although it is important that searches for systematic reviews attempt to identify all the relevant literature, there needs to be a trade-off between conducting an exhaustive search requiring additional resources compared with undertaking a more modest search that may miss some studies. Identifying key studies for a review question can assist in checking search sensitivity; such studies can also act as a guide to search terms.

8.1.4 Grouping review questions

Review questions that overlap and can be grouped together are identified for searching purposes (for example, questions about factors contributing to medication errors in care homes).

Questions that have the 'population' and 'intervention' in common but a different comparator can be grouped together by identifying and combining search terms for the population and intervention only.

8.1.5 Limiting searches

Using certain parameters to limit searches can improve precision without affecting sensitivity:

8.2.1 Core and subject-specific databases

The core databases listed in table 8.1 are searched for every review question. Additional subject-specific databases and other resources may also need to be searched, depending on the subject area of the review question and the type of evidence sought. Links are provided in table 8.1 and table 8.2 for sources that can be easily accessed.

Table 8.1 Databases that can be searched

An awareness of the strengths and weaknesses of each database is important when undertaking a systematic literature search.

The various databases use different:

For example, Embase is considered stronger than MEDLINE in covering pharmacology, toxicology, drug research and psychiatric literature, but contains selected dental and nursing literature. MEDLINE contains a collection of scope notes for its subject heading (MeSH) terms, which can assist in developing the search strategy. Records retrieved from the different databases for a particular review question may overlap; the extent of this overlap for MEDLINE and Embase is reported as being between 10% and 87% depending on the topic (Lefebvre et al. 2008). Cross-database searching, although time-consuming, is needed to comprehensively identify evidence for GPG development.

8.2.2 Other sources of information

The sources listed in table 8.2, including databases and websites, can provide useful information about ongoing research, patient experience, audits and statistics to help guide the Guidance Development Group's (GDG's) decision-making. Because the GPG is aimed at providing advice and guide good practice for people involved in handling, prescribing, commissioning and decision-making about medicines, for most of the topics it is necessary to use regulatory authorities and professional bodies as a source of information. This list is not exhaustive; the 'Searching for studies' chapter in the Cochrane handbook provides an overview and further examples of sources to search (Lefebvre et al. 2011). Other sources used depend on the GPG topic.

Table 8.2 Other sources of information

8.5.1 First sift

A first sift reviews each article by title and abstract against search criteria, removing any irrelevant results. This may be done by either the NICE project team or NICE guidance information services^[3] (see appendix A).

8.5.2 Second sift

Two members of the NICE project team independently sift each article by title and abstract in the reference management database. An opinion is recorded on whether each article meets the criteria for inclusion or exclusion taken from the PICO framework (see section 4.4.1), drawn up to answer the review questions for the GPG.

Once complete, a reconciliation meeting assesses and resolves any difference of opinion on articles over which there is disagreement. If there is no resolution, a third opinion that is binding is sought from a third NICE project team member. A final list of articles for inclusion is signed off by the NICE project lead after agreement by the GDG at GDG meeting 1. Any articles excluded at this stage have a rationale for exclusion recorded in the reference management database.

8.6.1 Free online journal articles

Many journals provide free access to some or all of their content. Several journals apply this to all material more than 1 or 2 years old; others provide access to particular types of articles only (for example, the British Medical Journal provides free access to all research articles). Individual articles can be purchased from the websites of most journals that do not allow free access, but this can be expensive.

Some websites provide links to medical journal web pages with freely available articles. Two that are used are Free Medical Journals and Genamics JournalSeek.

NICE Evidence Services and its Welsh equivalent, NHS Wales e\u2011\Library for health, provide free access to some journals for all NHS staff and staff in organisations such as NICE using an Athens log-in. This log-in can be obtained by applying to the NICE guidance information services^[3].

8.6.2 Free online reports

Many institutions make their reports and guidelines freely available online, so the relevant websites can be checked.

8.6.3 Libraries

Many libraries that stock a wide range of journals, books and reports have an inter-library loan or document delivery service. All supply articles within copyright law and some loan documents. There is usually a charge for this service, and for loans the cost of postage is usually extra. Some libraries provide articles at a reduced cost if an annual subscription is taken out. Three major libraries offering this level of service are the British Library, the British Medical Association (BMA) Library and the Royal Society of Medicine Library. A British Library account also allows users to pay for articles from other libraries that accept payment in this way.

When these references are needed, the NICE project team uses NICE guidance information services^[3] to source the reference.

8.7.1 Internal documentation

The following information is recorded for each search carried out during the GPG development process:

Enough detail is provided to allow searches to be repeated if needed.

8.7.2 Documentation for the good practice guidance

A full description of the searching process is included in the methods section of the GPG, which should include:

8.9.1 Call for written evidence

For some review questions, the GDG and NICE project team may agree that information exists that has not been found using standard searches. Therefore, the GDG and the NICE project team may 'call for evidence'. This call goes to all stakeholders and/or relevant organisations or individuals with a significant role or interest in the GPG topic being developed. It specifies the question(s) being addressed and detail the type of evidence being sought, for example, in terms of PICO framework (see section 4.4.1) and study design for questions of effectiveness. The need for a call for evidence is agreed at GDG meeting 1. The opportunity to provide written evidence is usually open for 2 weeks. All written evidence submissions are considered at GDG meeting 2.

8.9.2 Call for oral evidence

After review of the written evidence submissions, the GDG select up to 10 submissions to be called to give oral evidence at GDG meeting 3. The NICE project team records a written summary of the oral evidence presented.

8.9.3 Experts

If the GDG and the NICE project team identify gaps in the evidence that can be addressed by experts in that topic, the identified experts can be contacted directly to provide evidence. This can either be as written evidence or can be presented directly at a GDG meeting (either in person or virtually). An example could be to aid interpretation of legislation or regulations.

8.9.4 Using formal consensus methods outside the Guidance Development Group

Exceptionally, if the literature search has found no evidence that addresses the review question, the GDG may identify best practice by using formal consensus methods (for example, the Delphi technique or the nominal-group technique). Any formal consensus method used is documented in the GPG.