NICE process and methods

1 Introduction

1 Introduction

This is not the current manual. From January 2015, guidelines were developed using Developing NICE guidelines: the manual.

The National Institute for Health and Clinical Excellence (NICE) is the independent organisation responsible for providing national guidance on the promotion of good health and the prevention and treatment of ill health. NICE also produces quality standards that set out what high-quality care in the NHS should look like, and support tools for providers and commissioners in the NHS and local authorities. From 2013, NICE will produce quality standards for social care, as part of its expanding remit.

1.1 NICE guidance

NICE develops guidance across a number of different areas and on a range of topics.

All types of NICE guidance are developed using the best available evidence and involving stakeholders in a transparent and collaborative manner. Stakeholders include national organisations that represent the public, patients and carers, practitioners, community interests and companies that have an interest in the guidance in development.

NICE guidance is:

  • based on a rigorous assessment of the evidence base

  • developed by independent advisory bodies, with input from the public, patients, carers, service users, health professionals and other professional groups

  • developed using a transparent process and methods

  • subject to a consultation with stakeholders

  • regularly reviewed.

For more information about the core principles of NICE guidance development, please see our Strategic Plan.

1.1.1 Other products

NICE Pathways

NICE Pathways are a practical online resource for healthcare professionals to use on a day-to-day basis. A pathway presents recommendations from the guidance in a set of interactive topic-based diagrams. It contains all the recommendations from and links to related NICE guidance and other NICE products (for example, relevant quality standards and implementation tools).

Effective Interventions Library

The Effective Interventions Library (EIL) is a facility made available through the NICE public health pathways that links recommendations to supporting evidence about effective interventions. Within an EIL record, users are given summary information about key intervention characteristics such as effectiveness, cost and cost effectiveness, and considerations made by the committee. They are also directed, via links, to the original evidence reviews, study publications and considerations.

1.2 Equality and social value judgements

NICE is committed to promoting equality, eliminating unlawful discrimination and considering the implications of its guidance for human rights. It aims to comply fully with all legal obligations to:

  • promote race and disability equality, and equality of opportunity between men and women

  • eliminate unlawful discrimination on grounds of age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion or belief, sex and sexual orientation in the way it carries out its functions and in its employment policies and practices.

NICE's revised equality scheme sets out how it is meeting these obligations on equality and discrimination and what it still needs to do. In line with NICE's equalities scheme, this manual includes explicit consideration of how the development of guidance will consider equalities issues at the scoping, development, and validation stages, and how an audit trail of this activity will be maintained.

All NICE guidance, and the procedures NICE uses to develop its guidance, follow the principles set out in Social value judgements: principles for the development of NICE guidance. Second edition.

1.3 Who is this manual for?

This manual describes the methods used by the Centre for Public Health Excellence (CPHE) in NICE to develop and update public health guidance. It is aimed at:

  • external agencies undertaking evidence reviews and economic modelling for CPHE

  • other contractors engaged in the guidance production process (for example, those conducting fieldwork)

  • members of NICE's public health advisory committees (PHACs) that develop public health guidance

  • staff in CPHE and other NICE teams that contribute to the development and dissemination of public health guidance and related products.

However, it is also likely to be useful and of interest to stakeholders (for more information on stakeholders, see The NICE public health guidance development process (third edition, 2012).

This third edition has been prepared by CPHE staff, drawing on experience of the first 7 years of public health guidance development at NICE. It is based on international criteria of quality for guidance development, as outlined by the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument.

The structure of this manual follows the methods for development of NICE public health guidance from inception to publication. An overview of the processes for public health guidance development is described in The NICE public health guidance development process (third edition, 2012).

The manuals define the philosophical and methodological principles that govern the production of guidance. They also set out the operational framework that contractors, advisory committees and the CPHE team will follow. The manuals are prescriptive to ensure the:

  • aims and objectives of guidance production are as clear as possible

  • procedures used are as systematic, transparent, auditable and accountable as possible.

1.4 Framework for public health guidance

NICE's methods and processes for developing public health guidance are based on a clear set of values and principles. The range of activities and topics covered is inclusive. It is based on a conceptual framework for public health (see appendix A). This comprises 4 vectors – population, environment, society and organisations – linked to human behaviour. They explain the patterns of potentially preventable diseases. Social differences in the population are linked to patterns of mortality and morbidity. These differences manifest themselves in a number of key areas including:

  • conditions and diseases such as cardiovascular disease, cancer, obesity, diabetes, vaccine-preventable infections

  • behaviours such as smoking, drug or alcohol misuse, sexual activity, physical activity

  • other factors affecting health such as environment, work, housing and transport

  • accidents and injuries

  • child and maternal health, mental health, oral health.

In any of these areas, public health activities may be direct (for example, providing contraceptive services or smoking cessation services) or indirect (for example, creating safe open spaces for physical activity as part of general work to upgrade the environment). Traditional public health issues (such as ensuring the health of pregnant and breastfeeding women) and other issues associated with the wider determinants of health are all covered.

In summary, NICE public health guidance considers a variety of topics and approaches – see the NICE website for a list of published public health guidance and public health guidance in development.

1.4.1 Determinants of health

NICE public health guidance recognises the wide spectrum of determinants of health. The determinants work through the 4 vectors – population, environment, society and organisations (see appendix A). Human behaviour and the vectors interact via causal pathways to determine the health of individuals and populations. Patterns of illness can occur in whole populations or subpopulations. Both illnesses and the resulting patterns have causes. The task of developing guidance involves describing how health is affected and finding preventive mechanisms, both primary and secondary. The conceptual framework is used to define the scope of a topic and to articulate the mechanisms of cause and intervention.

Public health guidance may be aimed at population, community, organisational, group, family or individual level, as appropriate.

1.4.2 Equality and equity

1.4.2.1 Health difference and health inequality

Health difference is a simple difference in health state or status, however measured, between individuals or groups such as social classes, socioeconomic groups, men and women, ethnic groups or geographical communities. Health inequalities are not exclusively biological in origin but are also the consequence of human activity. Where inequalities arise as a consequence of human actions, they can be changed if the causes are changed.

1.4.2.2 Health inequity

A health inequity is an unnecessary, avoidable, unfair and unjust difference in someone's health or healthcare. 'Health inequity' should not be used interchangeably with the term 'health inequality' because the differences in health or healthcare that people experience are not necessarily unfair or unjust. Health inequity is concerned with social justice, values or politics, while inequalities in health are a matter of fact.

Health inequities, like health inequalities, can be eradicated or reduced because they are products of human action. However, addressing them can have considerable political implications because of the value judgement involved: not all people will judge the same health difference to be unfair.

1.4.2.3 Equality legislation

NICE public health guidance is developed within the legal framework relating to equalities. Addressing health inequalities has been a fundamental element of public health practice since its inception as a branch of medicine in the 19th century. Considering evidence on inequalities has been part of the work of CPHE since its foundation, and recommendations and research recommendations have been developed to attempt to redress these issues.

The Equality Act (2010) imposes a duty on public bodies to consider the effect of their activities in relation to equality. NICE considers issues of equality in its guidance production processes, so that unlawful discrimination does not arise from NICE guidance and so that opportunities for promoting equality can be identified. When developing public health guidance, under the public sector equality duty, NICE considers:

  • age

  • disability

  • gender reassignment

  • pregnancy and maternity

  • race

  • religion or belief

  • sex

  • sexual orientation.

Public health guidance also takes account of marriage and civil partnership and characteristics not explicitly protected by the Act such as socioeconomic differences and groups of people in certain situations including:

  • asylum seekers

  • people who are homeless

  • children and young people who are looked after

  • gypsies and travellers.

This list is not exhaustive, but gives an indication of areas where CPHE works to ensure public health guidance proactively reduces health inequalities and promotes equality.

Two legal concepts are important when considering equality: relevance and proportionality. Relevance assesses how much an issue affects equality. Proportionality assesses an appropriate outcome. The weight given to equality in a function should be proportionate to its relevance for that function.

The outcome of an intervention can affect health equity in different ways:

  • An intervention might improve the health of people in different groups to the same degree, so that any differences in health between those groups will remain after the intervention.

  • An intervention may be more effective in 1 group than in another. If it is more effective in the more disadvantaged group, the net outcome will be a reduction in inequity. If it is more effective in the less disadvantaged group, the net outcome will be an increase in health inequity.

  • An intervention can improve the population's overall health while increasing inequities. For example, historically health education campaigns to prevent smoking have tended to be more effective among those who are better off.

The ideal outcome is to benefit health in all groups, while reducing health inequities. However this may in practice be difficult to achieve. It may be justified in certain circumstances to increase overall population health while relative differences between groups remain the same (or get slightly worse). It is also possible that the information needed to achieve the most equitable outcomes is not available. These issues will be considered by the PHACs when developing recommendations. They will also be highlighted in the considerations section of final issued guidance.

Note that conforming to equality legislation will not necessarily mean that all health inequality and equity issues will be dealt with. Hence, the PHACs' examination of these issues may have to go beyond the legal definition as stated under the Equality Act (2010) as described earlier.

1.4.2.4 Health gradient

In whatever way health or disease is measured, there tends to be a gradient on which the most socially and economically advantaged groups have better health and lower rates of illness and death than disadvantaged groups. In advanced western societies such as the UK, the shape of the gradient tends to be relatively smooth with mortality and morbidity increasing steadily as social disadvantage increases. Over time, the gradient as a whole tends to shift upwards because overall the health of most groups is improving. However, the degree and rate of improvement tend to be greater in higher social groupings, meaning that relative differences, and therefore the degree of inequities and inequalities, also tend to increase.

1.4.2.5 Universal and targeted interventions

Universal interventions that produce overall health improvement can exacerbate relative health differences, since the well-off tend to make more use of (and derive greater benefit from) available services. This tendency has implications for public health interventions. Greater efforts are required to help people at the lower end of the social gradient improve their health faster than the rest of the population to combat health inequity. In other words, special efforts need to be made with disadvantaged groups. The problem is that targeted interventions tend to be both more difficult to implement and less cost effective than universal programmes.

Targeted interventions can reduce health inequalities. However, producing interventions to target relative need is difficult, because the social differences that constitute the gradient are not well described in the epidemiological, sociological or intervention literature. Lack of detail makes it very difficult to accurately focus interventions on particular sections of the population.

Efforts at health improvement should not be aimed solely at the most disadvantaged or socially excluded groups: this approach might lead to negligible benefits for the whole population. This is important because the graded characteristic of health differences means that there are still considerable levels of ill health and premature death in the midrange of the gradient. So public health guidance should focus on all levels of the gradient but include elements targeted at the most disadvantaged groups.

Factors that lead to general health improvement – improvements in the environment, good sanitation and clean water, better nutrition, high levels of immunisation, good housing – do not necessarily reduce health inequity. This is because the determinants of good health are not necessarily the same as the determinants of inequities in health (Graham and Kelly 2004). It is necessary, therefore, to distinguish between the causes of health improvement and the causes of health inequities.

In developing guidance, the decision to focus on universal, targeted or hybrid approaches will be made on a case-by-case basis. In some areas of public health work, universal approaches designed to produce overall health improvement will be suitable; in other cases a focus on the most disadvantaged will be suitable, and in others a hybrid approach along the gradient will be needed. Decisions about whether to take a targeted, universal or hybrid approach to equity will be made at the point when the topic is scoped.

1.4.2.6 Evidence for addressing inequalities

Evidence used as the basis for producing guidance should be assessed to see if it meets equality and diversity criteria as follows:

  • Do the evidence questions reflect the scope (that would have been equality-assessed, see section 3.3)?

  • Is the search strategy sensitive to evidence about social differences in the population (see sections 4.3.1 and 4.5)?

  • Is there evidence about inequalities from stakeholders (see section 4.4.5)?

  • Are evidence-review criteria inclusive (see section 5.8.1)?

  • Has the relevant data on equalities and diversity been appropriately extracted and presented in the review evidence statements (see section 5.8.2)?

  • What is the state of the evidence base in relation to health inequalities (see section 5.8.3)?

These questions should be dealt with at appropriate points throughout the guidance process, including: development of search strategies, the assembly of evidence, data extraction and the review and presentation of evidence.

1.5 Types of knowledge and evidence

NICE public health guidance draws on evidence and knowledge from across a spectrum of sources, using different methodologies and approaches. Sources include:

  • organisations

  • practitioners

  • the policy community, gained from the wider policy context

  • research, gathered systematically with a planned design

  • service users and carers.

The guidance may have many different audiences including different practitioner groups, decision and policy makers and commissioners of services. This includes those working in local authorities and other public sector organisations, and in the community, voluntary and private sectors. It includes employers as well as the general public. It may address a wide range of topics and issues, and usually does more than just make recommendations about effective and cost-effective interventions. As well as 'what works', it will often need to address when, why, how and for whom an approach does (and does not) work (Pawson 2006).

NICE public health guidance needs, therefore, to be informed by a wide variety of evidence and other forms of information (Lomas et al. 2005) (see chapter 3). This will include knowledge gathered using explicit, systemic and replicable scientific and social scientific methods. It will also include models, theories, expert testimony, mapping/practice reviews, consultation and practice.

NICE public health guidance is developed using methods and processes that can incorporate these different types of knowledge and evidence at various stages.

1.5.1 Best available scientific evidence

Scientific evidence is drawn from a range of disciplines and research traditions including clinical medicine, epidemiology, health economics, medical sociology, health psychology, medical anthropology, nutrition, sports science, nursing, education, political science and health education and promotion. This evidence is selected and appraised according to well-defined criteria. It is summarised using methods that are appropriate for public health and social scientific evidence, as described in this manual.

The fundamental issues taken into account are:

  • What is the most appropriate type of evidence to answer the question (see below and chapter 3 for further information)?

  • How can the most relevant evidence (published and unpublished) be identified (see chapter 4)?

  • How can the quality and applicability of evidence be assessed (see chapter 5)?

  • How can evidence from different kinds of research be synthesised and, in particular, how can quantitative and qualitative data be combined (see chapter 5)?

The randomised controlled trial (RCT) is normally the most appropriate source of evidence for judging the 'efficacy' of clearly circumscribed interventions that are implemented in ideal circumstances. However, such evidence is not always available or appropriate: it may not be feasible to conduct RCTs for some complex, large-scale, multi-agency and multi-faceted interventions, policies and services; and in some cases it may be unethical to do so. Further, given the complexity of causal chains in public health, the external validity of some RCT findings often has to be enhanced by observational studies to determine the 'effectiveness' of interventions in real-life situations. For evaluating large-scale interventions, observational studies may be the only feasible option (Victora et al. 2004). The Medical Research Council (MRC) has produced guidance on evaluating complex interventions (Craig et al. 2008) and using natural experiments to evaluate population health interventions (Craig et al. 2011).

In public health, social scientific as well as clinical and epidemiological evidence is used to examine outcomes, context, process and implementation (as well as barriers to and facilitators for interventions). There is little academic consensus about how best to synthesise these different approaches and there is still less agreement about how to use these disciplines to develop guidance.

The following types of evidence are of particular importance:

  • social scientific data as adjuncts to studies of clinical or public health effectiveness

  • social scientific trials and experimental and quasi-experimental designs

  • social scientific and other empirical information about context and process

  • social scientific theories and models.

For each of the above, different methods of appraisal and synthesis will be appropriate.

1.6 Stakeholders

Stakeholders are central to the development of NICE public health guidance (for more information on stakeholders, see The NICE public health guidance development process, third edition 2012). Guidance is subject to scrutiny and validation by stakeholders throughout the development process to ensure the resulting recommendations are realistic and appropriate.

1.7 Quality assurance principles

In addition to the broader values outlined above, CPHE operates to NICE's quality assurance principles that are designed to ensure that guidance is credible, robust and relevant.

  • Guidance development processes are governed by clear, published statements of methods and process (including a standard timeline) that are developed and updated at regular, predetermined intervals.

  • Standard operating procedures are prepared for each principal step in the guidance development process. These procedures are developed in consultation with the staff who will operate them and are reviewed at regular, predetermined intervals.

  • Guidance publications are authorised for publication, on behalf of the Board, by the Guidance Executive.

  • The Senior Management Team and the Board receive bimonthly reports that identify variations from the planned programme, the reasons for the variations and the remedial action taken.

  • Each member of staff is aware of his or her personal responsibility for endeavouring to assure the quality of their work, through the application of standard processes and methods, and through independent thought and action, when necessary and appropriate.

  • The potential risks associated with the development of a piece of guidance are assessed and reported to the Audit Committee together with risk minimisation and handling strategies.

  • A set of clear publication standards is applied to the presentation of all work, including review in every case by a professional editor. (This does not include supporting documents such as fieldwork reports and evidence reviews compiled by external contractors.)

  • Dissemination of publications takes place after analysis of the extent and needs of the audience for each piece of guidance.

  • Training is provided for staff to enable them to apply these principles in their daily practice.

1.8 References and further readinghttp://

Craig P, Dieppe P, McIntyre S et al. on behalf of the MRC (2008) Developing and evaluating complex interventions: new guidance. London: Medical Research Council

Craig P, Cooper C, Gunnell D et al. on behalf of the MRC (2011) Using natural experiments to evaluate population health interventions: guidance for producers and users of evidence. London: Medical Research Council

Davidson K, Goldstein M, Kaplan RM et al. (2003) Evidence-based behavioral medicine: what it is and how do we achieve it? Annals of Behavioral Medicine 26: 161–71

Dixon-Woods M, Agarwal S, Young B et al. (2004) Integrative approaches to qualitative and quantitative evidence. London: Health Development Agency

Flay BR (1986) Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs. Preventive Medicine 15: 451–74

Graham H, Kelly MP (2004) Health inequalities: concepts, frameworks and policy. London: Health Development Agency

Kelly MP, Morgan A, Bonnefoy J et al. (2007) The social determinants of health: developing an evidence base for political action. Final report to the World Health Organization Commission on the Social Determinants of Health, from the Measurement and Evidence Knowledge Network

Kelly MP, Stewart E, Morgan A et al. (2009) A conceptual framework for public health: NICE's emerging approach. Public Health 123. e14–20

Kelly MP, Morgan A, Ellis S et al. (2010) Evidence based public health: A review of the experience of the National Institute of Health and Clinical Excellence (NICE) of developing public health guidance in England, Social Science and Medicine, 71:1056–62

Lomas J, Culyer T, McCutcheon C et al. (2005) Conceptualizing and combining evidence for health system guidance: final report. Ottawa: Canadian Health Services Research Foundation

Michie S, Berentson-Shaw J, Pilling S et al. (2007) Turning evidence into recommendations: protocol for a study of guideline development group. Implementation Science 2: 29

National Institute for Health and Clinical Excellence (2007) Behaviour change at population, community and individual levels. NICE public health guidance 6. London: National Institute for Health and Clinical Excellence

National Institute for Health and Clinical Excellence (2008) Social value judgements: principles for the development of NICE guidance, Second edition.. London: National Institute for Health and Clinical Excellence

Nutbeam D (1998) Evaluating health promotion – progress, problems and solutions. Health Education International 13: 27–44

Ogilvie D, Hamilton V, Egan M et al. (2005) Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? Journal of Epidemiology and Community Health 59: 804–8

Ogilvie D, Egan M, Hamilton V et al. (2005) Systematic reviews of health effects of social interventions: 2. Best available evidence: how low should you go? Journal of Epidemiology and Community Health 59: 886–92

Pawson R (2006) Evidence based policy: a realist perspective. London: Sage

Popay J, editor (2005) Moving beyond effectiveness in evidence synthesis: methodological issues in the synthesis of diverse sources of evidence. London: National Institute for Health and Clinical Excellence

Victora C, Habicht J, Bryce J (2004) Evidence-based public health: moving beyond randomized trials. American Journal of Public Health 94(3): 400–5

Weightman A, Ellis S, Cullum A et al. (2005) Grading evidence and recommendations for public health interventions: developing and piloting a framework. London: Health Development Agency

Weiss CH (1995) Nothing as practical as good theory: exploring theory-based evaluation for comprehensive community initiatives for children and families

In: Connell JP, Kubisch A, Schorr LB et al. New approaches to evaluating community initiatives: concepts, methods and context. Washington DC: Aspen Institute