NICE process and methods
9 Interpreting the evidence and writing the guideline
9 Interpreting the evidence and writing the guideline
This chapter gives guidance on how the committee should interpret the evidence and decide what recommendations to make. It also gives some advice on how to word guideline recommendations, although content designers will help committees with this.
9.1 Interpreting the evidence to make recommendations
Assessment and interpretation of the evidence to inform guideline recommendations is at the heart of the work of the committee.
Recommendations are developed using a range of evidence from the literature searches and other evidence – such as real world data and expert testimony (see the appendix on call for evidence and expert witnesses), views of stakeholders, people using services and practitioners, health inequalities briefings (if available) and the committee's discussions and debate (see the chapter on decision-making committees).
The committee should use its judgement to decide what the evidence means in the context of the scope of the guideline or area(s) for update. The quality of the evidence will have been assessed for both internal and external validity (see the chapter on reviewing evidence), but also needs interpretation. If a conceptual framework or logic model is being used to develop the guidance, the committee should consider this when interpreting the evidence.
The committee should decide what action to recommend and keep in mind which sectors (including which practitioners or commissioners within those sectors) should act on the recommendations. This will identify the likely impact of the recommendations on practice or services and the committee can decide whether to stipulate who the recommendation is aimed at.
The committee should discuss how they moved from the evidence to each recommendation, including the relative value placed on the agreed outcomes, the benefits and harms of any interventions, resource use, and the overall quality of the evidence, as well as other factors they took into consideration.
For each recommendation or group of recommendations, the committee should discuss and agree their rationale for making the recommendations and the likely impact of the recommendations on practice or services. They should also discuss how the recommendations address any equality issues or health inequalities identified during the guideline development process.
Quality of the evidence
Evidence review documents summarise the evidence obtained from the results of evidence searches. Depending on the topic and type of evidence, they may include GRADE tables, GRADE-CERQual tables or (if GRADE or GRADE-CERQual are not used) evidence statements.
The committee should ensure that the reviews are a fair summary of the evidence and should discuss any uncertainty in the review findings (including limitations of individual studies and inconsistency across studies).
For details, see the chapter on reviewing evidence.
Trade‑off between benefits and harms of an intervention
A key stage in moving from evidence to recommendations is weighing up the size and importance of the benefits and harms of an intervention, compared with the alternatives specified in the review protocol, and the potential for unintended consequences. This may be done qualitatively or quantitatively.
The committee should discuss the extent to which the effects seen in the evidence are representative of what would happen in the real world.
The committee should also assess the extent to which the recommendations may impact on health inequalities. This needs to be made clear, regardless of whether the recommendation is aimed at the whole population, specific subgroups or a combination of both. If there is potential to increase health inequalities, the committee should consider whether they can do anything to prevent this from happening or reduce the impact.
Trade-off between economic considerations and resource use
The committee should discuss cost effectiveness at the same time as effectiveness when formulating recommendations. Interventions that are not considered cost effective should not usually be offered.
The evidence review document should explain how costs, resource use and economic considerations were taken into account in determining the cost effectiveness of an intervention. This may be informal, or may be more formal and include economic modelling (see the chapter on incorporating economic evaluation).
If several possible interventions are being considered, the committee should consider sequencing them in terms of their cost effectiveness. This usually means preferring the most cost-effective intervention first, although other factors such as availability and acceptability need to be considered.
Considerations about equity may also affect the decision whether to recommend the intervention (see the section on equity considerations in the chapter on incorporating economic evaluations).
Use of indirect evidence
Sometimes, when there is no evidence directly relevant to a specific population, indirect evidence from other populations may be considered. For example, evidence on treating absence seizures in children and young people was extrapolated to adults because the disease has a similar pathophysiology in all 3 populations.
This needs careful consideration by the committee, with discussion of the features of the condition or interventions that allow extrapolation to a different population.
This also applies when extrapolating findings from evidence in different care settings (for example, between primary and secondary care). The committee should consider the similarities and differences in case mix, staffing, facilities and processes between the settings before extrapolating evidence in this way.
Consider the feasibility of putting recommendations into practice
The committee should judge to what extent it will be feasible to put the recommendations into practice.
The committee should consider the extent of change in practice that will be needed to implement a recommendation, staff training needs, policy levers and funding streams, and the possible need for carefully controlled implementation with, for example, training programmes. This should be documented in the guideline and in any resources to support implementation (see the chapter on support for putting the guideline recommendations into practice).
Wider basis for making recommendations
The committee should take into account a range of issues, including any ethical issues, equity considerations, health inequalities and national priorities for health and care, as well as equality legislation, to ensure that the guideline recommendations are ethical, practical and specific.
There are no hard-and-fast rules or mechanisms for doing this: the committee should make conscious and explicit use of its members' skills and expertise. All evidence needs interpretation: evidence alone cannot determine the content of a recommendation.
Developing evidence-based recommendations involves:
-
using what is known (inductive reasoning) while accepting that there is uncertainty about what is likely to happen because of implementing a recommendation
-
drawing on theory or methodological principles (deductive reasoning).
Alongside this manual, committees should use our principles and our charter to inform their decisions. The committee may also draw on the principles outlined in the report on ethical issues in public health by the Nuffield Council on Bioethics.
Promoting equality and reducing health inequalities
The equality and health inequalities assessment (EHIA) form should document how the committee's responsibilities under equality legislation and our equality scheme have been discharged in reaching the recommendations (see the section on key principles for developing NICE guideline recommendations in the introduction chapter), and how the recommendations address equality issues and health inequalities.
The committee needs to consider whether:
-
the evidence review has found evidence to support recommendations to address any equality issues and health inequalities identified during guideline development (if not, consider other sources of information for example expert testimony or health inequality briefings, if available)
-
criteria for access to an intervention might be discriminatory (for example, through membership of a particular group, or by using an assessment tool that might discriminate unlawfully)
-
any groups of people might find it impossible or difficult to receive or access an intervention.
Ideally, recommendations should be formulated to promote equality and reduce health inequalities (for example, by making access more likely for certain groups, or by tailoring the intervention to specific groups). If this is not possible, the committee should consider whether it is appropriate to make a research recommendation (for further details see the section on formulating recommendations for research).
Strength of recommendations
The concept of the 'strength' of a recommendation (Guyatt et al. 2008) is key to translating evidence into recommendations. This takes into account the quality of the evidence but is conceptually different.
If the committee believes that the vast majority of practitioners or commissioners and people using services would, based on the evidence seen by the committee, choose a particular intervention, they should make a strong recommendation for the intervention. This is generally the case if the benefits clearly outweigh the harms for most people and the intervention is likely to be cost effective. If the opposite is true, they should make a strong recommendation against the intervention (see below).
If the committee concludes, based on the evidence, that there is a closer balance between benefits and harms, and some people would not choose an intervention whereas others would, they should make a weak recommendation for the intervention.
If a specific subgroup of people is likely to benefit from an intervention and others are not, it may be possible to make a strong recommendation for that subgroup of people.
The committee should be aware that we reflect the strength of the recommendation in the wording (see the section on wording the recommendations).
Strong recommendations against an intervention
Reasons for the committee to make a strong recommendation against an intervention include:
-
potential harms outweigh the potential benefits
-
the intervention has no reasonable prospect of providing cost-effective benefits
-
stopping the intervention is not likely to cause harm
-
good-quality evidence shows a lack of efficacy or effectiveness
-
there is a lack of evidence of efficacy or effectiveness for an intervention, or the quality of the evidence is too low or too uncertain
-
the intervention has a major national safety warning, for example from the Medicines and Healthcare products Regulatory Agency (MHRA) or NHS England.
If most people are likely to experience no benefit or to experience harm but there may be a benefit for some, the committee can make a strong recommendation against the intervention but with a caveat. In this case, they should be as specific as possible about the circumstances under which, or population for whom, the intervention is appropriate.
'Only in research' recommendations
The committee can make an 'only in research' recommendation if the necessary research can realistically be set up or is already planned, or people using services are already being recruited for a study. The following criteria may also apply:
-
the intervention has a reasonable prospect of providing cost-effective benefits
-
there is a real prospect that the research will inform future NICE guidelines.
Little evidence of difference between interventions
There might be little evidence of differences in effectiveness or cost effectiveness between interventions. In this case, all effective or cost-effective interventions may be recommended.
Insufficient evidence
If published evidence of efficacy or effectiveness for an intervention is lacking, too low quality, or too uncertain for firm conclusions to be reached, the committee may use its experience and knowledge to do 1 of the following:
-
make recommendations by consensus
-
make research recommendations, or
-
make no recommendation.
The last option should be used sparingly on the basis that scoping will have shown that guidance was needed.
The principles in the section on wording the recommendations should be used.
Recording the committee's discussion and rationale for the recommendations
The committee's justifications for making the recommendation, and its strength, should be summarised in the rationale for the recommendation and fully explained in the committee discussion section of the relevant evidence review document.
The committee discussion follows a structured format, to ensure transparency about the issues considered. In most cases the committee reaches decisions through a process of informal consensus. If formal voting procedures are used, this is also recorded.
Principles of person-centred care
All NICE guidelines advocate the principles of person-centred care: people using services and the wider public should be informed of their options and be involved in decisions about their care, as described in our webpage on making decisions about your care.
The following foundational guidelines include general recommendations on the principles of person-centred care, such as communication, providing information and shared decision making, which should not be restated in topic-specific guidelines (see table 1):
Recommendations from these guidelines can be cross-referred to when needed.
Recommendations on person-centred care can be included in topic-specific guidelines if there is evidence of specific need for the topic.
Topic areas we do not usually make recommendations on
Topic area | What to do instead | Exceptions |
---|---|---|
General principles of care covered in foundational guidelines |
Link to the relevant foundational guideline: patient experience in adult NHS services service user experience in adult mental health people's experience in adult social care services babies, children and young people's experience of healthcare |
If there is evidence of issues specific to the topic of the guideline recommendations |
Repeating recommendations from another NICE guideline |
Link to the other guideline (see the chapter on linking to other guidance) |
If linking between specific recommendations would be cumbersome for users |
Recommendations on general lifestyle advice |
Link to relevant public health guidelines |
If there is evidence and a strong rationale to include a recommendation specific to the topic of the guideline recommendations |
Recommendations on good practice or general principles of care that are not linked to review questions or evidence |
Do not include |
If there is evidence and a strong rationale to include a recommendation specific to the topic of the guideline recommendations |
Recommendations on general medicines optimisation issues |
Link to the relevant guideline, for example: |
If there is evidence and a strong rationale to include a recommendation specific to the topic of the guideline recommendations |
Prescribing information covered by the BNF (for example, dosing, monitoring, adverse effects, contraindications) |
Nothing – this is covered by the BNF |
If the recommendation would not make sense without reference to the prescribing information |
National patient safety advice on medicines and devices |
The NICE medicines adviser will work with the MHRA if needed |
If there is a significant safety risk and clear evidence that safety advice is not routinely implemented in practice, if the recommendation will not make sense without the information |
Training or competency in areas that are the responsibility of professional bodies |
The implementation team can work with professional bodies to identify training needs |
Training or competency in areas that are not the responsibility of professional bodies, and are identified as being important to cover in guideline recommendations |
Service configuration or service delivery |
Do not include |
Recommendations on service delivery or service configuration that are evidence based or address system priorities |
Following laws or statutory guidance |
Do not include |
If there is evidence that guidance is needed on how to follow the law or statutory guidance |
9.2 Recommendations on medicines
When making decisions about treatment options, users of our guidelines are expected to take note of prescribing information, such as dosage, duration of treatment, contraindications, adverse effects, warnings, safety advice and any monitoring requirements for a medicine. This is available in the British National Formulary (BNF) or BNF for Children (BNFC), as well as the medicine's summary of product characteristics (available on the electronic medicines compendium). We do not usually include prescribing information in our recommendations though there are some exceptions to this (see table 1 and the section on off-label use of licensed medicines). For more information on prescribing, see our webpage on making decisions using NICE guidelines.
Overprescribing
Overprescribing is when people are given medicines they do not need or want, or where the harm outweighs the benefits. In line with the Department of Health and Social Care's national overprescribing review, we should include recommendations for reviewing and stopping medicines if overprescribing is a concern. Also see NICE's guidelines on medicines optimisation and medicines associated with dependence or withdrawal symptoms.
National medicines safety advice
National medicines safety advice includes national patient safety alerts and the MHRA's drug safety updates. We do not usually include patient safety information in our recommendations though there are some exceptions to this. See table 1 for details.
Antimicrobials and antimicrobial stewardship
Recommendations on antimicrobials should:
-
take account of antimicrobial resistance and the principles of good antimicrobial stewardship
-
name the specific antibiotic or class of antibiotics being recommended
-
include information on reviewing and stepping down treatment when recommending intravenous or prophylactic antibiotics.
Guidelines that cover antimicrobial prescribing may include prescribing tables that detail choice of antimicrobials, dosages, duration of treatment and routes of administration (for an example of an antimicrobial prescribing table, see the section on choice of antibiotic in NICE's guideline on Clostridioides difficile infection).
Off-label use of licensed medicines
Recommendations are usually about using medicines within their licensed indications. However, there are clinical situations in which recommending an off-label use of a licensed medicine may be in the best interests of the person, in line with the MHRA guidance (see appendix 2 of the MHRA guidance on the supply of unlicensed medicinal products). For example, this may happen if the clinical need cannot be met by using a licensed product within the terms of its marketing authorisation and there is enough evidence or experience of using the medicine to support its safety and efficacy.
Dosage information for off-label use of a licensed medicine is not usually included in the summary of product characteristics (SPC). If off-label use is being recommended, we will check whether there is any relevant dosage information in the BNF or BNF for Children. If there is none, we will work with the BNF to add the necessary information if needed. If we are recommending off-label use of a medicine, this will be clearly stated in the guideline.
Unlicensed medicines
The MHRA states that: If a UK licensed medicine can meet the person's clinical need (even if it is used off-label), it should be recommended instead of an unlicensed product. An unlicensed medicine should not be recommended if a product available and licensed within the UK could be used to meet the person's clinical need.
Committees should take account of the MHRA guidance (see appendix 2 of the MHRA guidance on the supply of unlicensed medicinal products) when making recommendations but consider each situation on its own merit.
Medical devices, including off-label use
Recommendations are usually about using devices within the terms of the instructions for their use. However, there are clinical situations in which the off-label use of a device may be in the best interests of the person. For example, when using a device outside the time period specified in the instructions for use.
Committees should take account of the MHRA guidance on the off-label use of medical devices.
9.3 Wording the recommendations
This section gives the key principles for writing recommendations. Following these principles helps ensure that recommendations meet user needs. The content designer works with the rest of the development team and committee throughout guideline development to ensure that recommendation wording reflects the committee's intent and is clear and easy to follow.
The recommendations should be in line with our style and principles, and accessibility regulations.
For information on NICE style, and using clear English and person-centred language, see our style guide and guide on writing for NICE.
For information on accessibility, see our webpages on accessibility and accessibility changes: notes for developers.
Focus on the action and what readers need to know
Recommendations should be clear about what needs to be done, without the reader having to read the rationale or committee's discussion in the evidence review document. When writing recommendations, keep in mind the following:
-
a reader asking, 'What does this mean for me?'
-
how a health and care professional will be able to implement them with an individual person, in a way that supports shared decision making.
Include only 1 action per recommendation or bullet point unless it is clearer to include a closely linked action in the same recommendation.
Be specific about actions and use direct instructions in recommendations wherever possible because these are easier to follow. Recommendations should start with a verb such as 'offer' (or 'do not'), 'consider', 'measure', 'advise', 'discuss', 'ask about'.
Exceptions to this principle include:
-
Recommendations that specify who should take action, or cover service organisation. For example: A multidisciplinary team should provide care.
-
Recommendations that use 'must' or 'must not' (because of a legal duty or serious consequences of not following the recommendation).
Think carefully about how much detail to include. Recommendations should be clear and concise. Including a lot of detail can reduce the impact and make them harder to understand.
Reflect the strength of the recommendation
In 'strong' recommendations (see the section on strength or recommendations) for actions that should (or should not) be offered, use directive language such as 'offer' (or 'do not offer'), 'advise', or 'ask about'. In keeping with the principles of shared decision making, people may choose whether or not to accept what they are offered or advised.
If there is a closer balance between benefits and harms (activities or interventions that could be used), use 'consider' to reflect that the recommendation is 'weak'.
Use 'person-centred', precise, concise, clear English
Key principles include using language that is person-centred, using clear and consistent wording, and using bullet lists and tables if they make recommendations easier to follow.
Language that is person-centred acknowledges the experience of people who are directly affected by the recommendations (and family members, carers or advocates), and their role in decision making. For more information, see the section on talking about people in the NICE style guide.
9.5 Formulating recommendations for research
The committee is likely to identify areas for which there are uncertainties or for which robust evidence is lacking. They can suggest up to 5 key recommendations for research that are likely to inform future decision-making (based on a systematic assessment of gaps in the current evidence). The committee should justify and document why they have made each recommendation for research. For further information, see the NICE research recommendations process and methods guide.
9.6 References and further reading
Alonso-Coello P, Oxman AD, Moberg J et al. for the GRADE working group (2016) GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ 353: i2089
Claxton K, Sculpher MJ (2006) Using value of information analysis to prioritise health research: some lessons from recent UK experience. Pharmacoeconomics 24: 1055–68
Glasziou P, Del Mar C, Salisbury J (2003) Evidence-based medicine workbook. London: British Medical Journal Books
Guyatt GH, Oxman AD, Vist GE et al. for the GRADE working group (2008) GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 336: 924 (see also the GRADE website)
Joint Royal College of Paediatrics and Child Health/Neonatal and Paediatric Pharmacists Group Standing Committee on Medicines (2013) The use of unlicensed medicines or licensed medicines for unlicensed applications in paediatric practice
Kelly MP, Moore TA (2012) The judgement process in evidence-based medicine and health technology assessment. Social Theory and Health 10:1–19
Michie S, Johnston M (2004) Changing clinical behaviour by making guidelines specific. British Medical Journal 328: 343–5
Nuffield Council on Bioethics (2007) Public health: ethical issues. London: Nuffield Council on Bioethics
Sackett DL, Straus SE, Richardson WS (2000) Evidence-based medicine: how to practice and teach EBM. Edinburgh: Churchill Livingstone
Schünemann HJ, Mustafa R, Brozek J et al. for the GRADE working group (2016) GRADE Guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health. Journal of Clinical Epidemiology 76: 89–98
Scottish Intercollegiate Guidelines Network (2019) SIGN 50. A guideline developer's handbook, revised edition. Edinburgh: Scottish Intercollegiate Guidelines Network
Tannahill A (2008) Beyond evidence – to ethics: a decision making framework for health promotion, public health and health improvement. Health Promotion International 23: 380–90
Weightman A, Ellis S, Cullum A et al. (2005) Grading evidence and recommendations for public health interventions: developing and piloting a framework. London: Health Development Agency