Appendix B: Summary of the methods used to develop this guidance

Guidance

Introduction
Key questions
Reviewing the evidence of effectiveness
Economic appraisal
Fieldwork
How the PDG formulated the recommendations

Appendix B: Summary of the methods used to develop this guidance

Introduction

The reports of the reviews and economic appraisal include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the Programme Development Group (PDG) meetings provide further detail about the Group's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E.

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and facilitated the development of recommendations by the PDG.

The overarching question was: 'What nutritional interventions are effective in improving the health of pre‑conceptual, pregnant and postpartum mothers and children (up to 5 years) and reducing nutrition‑related health inequalities?

The subsidiary questions relating to each intervention/programme studied were:

1. What is the aim/objective?

2. What is the content and how does it influence effectiveness?

3. How does the way that it is carried out influence its effectiveness?

4. Does effectiveness depend on the job title/position of the person delivering the intervention? What are the significant features of an effective leader?

5. How does effectiveness vary according to factors such as the age, sex, class or ethnicity of the target audience?

6. Does the intensity (or length or frequency) of the intervention influence effectiveness or duration of effect?

7. How much does the intervention cost (in terms of money, people, time)?

8. Does the site/setting influence effectiveness?

9. What are the facilitators and barriers to implementation?

10. How acceptable is the intervention to the recipients?

11. What evidence is there on cost effectiveness and does the intervention offer value for money?

These questions were refined further in relation to the topic of each review (see reviews for further details).

Reviewing the evidence of effectiveness

Eight reviews of effectiveness and 3 expert papers were conducted to inform the development of this guidance.

Identifying the evidence

The following searches were carried out for each effectiveness review, as follows.

Review 1

The following databases were searched for systematic reviews (from 1995 onwards): the Cochrane Database of Systematic Reviews (CDSR), and the Database of Abstracts of Reviews of Effects (DARE), the Health Technology Assessment Database and the Ongoing Reviews Register. A number of websites were also scanned/searched to identify relevant reviews. Where no relevant systematic reviews existed, the following databases were searched for randomised controlled trials (RCTs; from 1990 onwards): CENTRAL, EMBASE, PsycINFO, CINAHL and MEDLINE. For other types of UK study, the following databases were searched: MEDLINE, EMBASE, CINAHL and PsycINFO.

Review 2

The following databases were searched for systematic reviews (from 1995 onwards): CDSR and DARE. Where no relevant systematic reviews existed, the following databases were searched for RCTs (from 1990 onwards): the Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE, CINAHL and PsycINFO. For other types of UK study (from 1990 onwards) the following databases were searched: MEDLINE, EMBASE, CINAHL and PsycINFO databases were searched.

Review 3

The following databases were searched for systematic reviews (from 1995 onwards): CDSR, DARE, National Research Register (including CRD Ongoing Reviews), Health Technology Assessment Database, SIGN Guidelines, National Guideline Clearinghouse, National Coordinating Centre for Health Technology Assessment, HSTAT, the DH Research Findings Electronic Register, TRIP, Clinical Evidence, and Health Evidence Bulletins Wales. The following databases were searched for RCTs and UK studies of other types (from 1990 onwards): CENTRAL, EMBASE, PsycINFO, CINAHL and MEDLINE. A search was also conducted of NICE web pages (published appraisals).

Review 4

The following databases were searched for systematic reviews (from 1995 onwards): CDSR (Issue 2 2006), DARE, the Health Technology Assessment Database and Ongoing Reviews Register. The following databases were searched for RCTs (from 1990 onwards): MEDLINE, EMBASE, CENTRAL, CINAHL and PsycINFO.

Review 5

The following databases were searched for systematic reviews (from 1995 onwards): CDSR, DARE, National Research Register (including CRD Ongoing Reviews), National/Health Technology Assessment Database, SIGN Guidelines, National Guideline Clearinghouse, DH Research Findings Electronic Register, TRIP Clinical Evidence and Health Evidence Bulletins Wales. The following databases were searched for RCTs (from 1990 onwards): CENTRAL, EMBASE, PsycINFO, CINAHL and MEDLINE. In addition, a search was carried out of the National Coordinating Centre for Health Technology Assessment NICE web pages (published appraisals).

Review 6

Review 7

The following databases were searched from 1966 to 2006: MEDLINE, EMBASE, CINAHL, CCTR, CDSR, DARE and AMED. The search was not limited by study type, but was restricted to studies in developed countries and published in English language. Reference lists of identified articles were also checked.

Review 8

A non‑systematic review was conducted. Studies of corroborative evidence such as surveys, qualitative studies, cohort studies, case‑control studies, case‑series and expert opinions were identified. The following databases were searched (from 1966 to 2006): MEDLINE, EMBASE, CINAHL, CCTR, CDSR, DARE and AMED. In addition, a 'snowball' search of the internet was carried out and included the websites run by the following: DH, Health Education Authority, MAFF, FCA, DEFRA, WHO and UNICEF. A hand/document search from reference studies and a search of the grey literature was also conducted.

Expert papers

The 2 expert papers on the safe storage of expressed breast milk and growth monitoring draw on published research in addition to expert opinion.

The expert paper on nutrition and breastfeeding draws on nationally published data from surveys and published scientific evidence on maternal nutrition and breast milk volume and composition.

Further details of the databases, search terms and strategies are included in the review reports.

Selection criteria

Details of the inclusion and exclusion criteria are given each review. However, in general:

Review 1 included: systematic reviews from 1990 onwards, RCTs from 1990 onwards published worldwide in English and non‑randomised studies (cohorts, qualitative studies and surveys) conducted from 1990 and published in the UK. It focused on interventions for non‑pregnant women of childbearing age who were planning a pregnancy or who might become pregnant. The interventions had to start prior to conception but could continue or stop at any time during the pregnancy.
Review 2 included: systematic reviews published since 1995; RCTs from 1990 onwards published worldwide in English and non‑randomised UK studies from 1990 onwards, with a particular focus on nutrition interventions aimed at low‑income women during pregnancy.
Review 3 included: systematic reviews from 1995 onwards, RCTs from 1990 onwards published worldwide in English, and UK studies of all interventions aiming to improve the nutrition of women during the first year after birth. Studies had to examine women up to 6 weeks after giving birth (postpartum) and living in developed countries, from any socio‑economic background. Where data were available, the review also considered interventions on disadvantaged population subgroups.
Review 4 included: systematic reviews from 1995 onwards, RCTs from 1990 onwards published worldwide in English, and other UK studies. It focused on interventions to:
- promote the initiation, and increase the duration of, breastfeeding
- reduce the risk of contamination of feeding equipment
- ensure breast milk is safely stored and reheated
- reduce the risks associated with the reconstitution of formula milk.
Review 5 included: systematic reviews from 1995 onwards, RCTs from 1990 onwards and non‑randomised studies conducted in the UK and published from 1990 onwards. It focused on interventions to promote safe and healthy feeding practices for infants and young children who are moving from an exclusively milk‑based diet to solid food.
Review 6 included: systematic reviews from 1995 onwards published worldwide in English, RCTs from 1990 onwards conducted in developed countries, and other study types conducted in the UK and published from 1990 onwards. It focused on interventions aimed at children aged 2 to 5 years old, their parents and carers, and staff looking after 2‑ to 5‑year‑olds in nurseries and other day care settings.
Review 7 included: all types of studies conducted in the UK, and systematic reviews and RCTs carried out in developed countries. Only papers published in English between 1966 and 2006 were considered. It focused on interventions promoting vitamin D intake, in line with the Committee on Medical Aspects of Food and Nutrition Policy (COMA) recommendations on vitamin D. The interventions had to be aimed at 1 of the following:
- women who were planning a pregnancy, were pregnant or had given birth in the previous year
- infants and young children (from birth up to age 5 years)
- vulnerable groups (with a particular emphasis on black and minority ethnic groups).
Review 8 included: primary studies (cohort and case‑control studies, case‑series and expert opinions). It focused on corroborative evidence on interventions to improve the nutrition of infants/children aged 6 months to 5 years. (The corroborative evidence related to the process and context of interventions.)

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual 'Methods for development of NICE public health guidance' (see appendix E). Each study was described by study type and graded (++, +, -) to reflect the risk of potential bias arising from its design and execution.

Study type

Meta‑analyses, systematic reviews of randomised controlled trials (RCTs) or RCTs (including cluster RCTs).
Systematic reviews of, or individual, non‑RCTs, case‑control studies, cohort studies, controlled before‑and‑after (CBA) studies, interrupted time series (ITS) studies, correlation studies.
Non‑analytical studies (for example, case reports, case series).
Expert opinion, formal consensus.

Study quality

++ All or most of the criteria have been fulfilled. Where they have not been fulfilled the conclusions are thought very unlikely to alter.

+ Some criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are thought unlikely to alter the conclusions.

- Few or no criteria fulfilled. The conclusions of the study are thought likely or very likely to alter.

The studies were also assessed for their applicability to the UK.

Summarising the evidence and making evidence statements

The review data was summarised in evidence tables (see evidence tables and the synopsis).

The findings from the reviews were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements reflect the strength (quantity, type and quality) of evidence and its applicability to the populations and settings in the scope.

Economic appraisal

The economic appraisal consisted of 2 economic effectiveness reviews and an economic appraisal.

Reviews of economic evaluations

The following databases were searched: MEDLINE, EMBASE, CINAHL, CCTR, CDSR, DARE and NHSEED. The search strategy combined relevant terms relating to:

pre‑conceptual, pregnant and postpartum women
children up to age 5 years.

The search incorporated a sensitive health economics filter and focused on interventions rather than being restricted to outcomes.

In selecting studies for the review, the main exclusion criteria were as follows:

primary studies set in developing or low‑income countries
studies published before 1990
papers in a language other than English
papers not held at the British Library
abstracts.

A total of 24 articles were included in the pre‑conceptual, pregnant and postpartum women review. Nine articles were included in the children 0 to 5 years review. These were assessed for their methodological rigour and quality using the critical appraisers' checklists provided in appendix B of the Methods for development of NICE public health guidance' (table 3.1). Each study was categorised by study type and graded using a code (++), (+) or (-), based on the potential sources of bias.

Cost‑effectiveness analysis

An economic model was constructed to incorporate data from the reviews of effectiveness and cost effectiveness. The results are reported in: 'Modelling the cost‑effectiveness of breastfeeding support'. The economic models and results are available on the guideline's webpage.

Fieldwork

Fieldwork was carried out to evaluate the relevance and usefulness of NICE guidance for practitioners and the feasibility of implementation. It was conducted with practitioners and commissioners who are involved in primary care, maternity and children's services. This included those working in ante‑ and postnatal care in hospitals and communities, GP practices, community pharmacies and nutrition and dietetic services in the NHS. It also included those working in children's centres and trusts and in early years and pre‑school settings in the public, private and voluntary sectors.

The fieldwork comprised:

Workshops carried out in Birmingham, Brighton, Derby, East London, Leeds, Lowestoft, Manchester and Plymouth with: health visitors, midwives, maternity services managers, dietitians, public health nutritionists, infant and breastfeeding coordinators, breastfeeding peer‑support workers, children's centre managers and coordinators, nursery nurses, managers and cooks at day nurseries, public health coordinators and managers, and 5 a Day coordinators.
Interviews carried out by telephone with commissioners for children's and family services and a range of professionals including: dentists, pharmacists, GPs, obstetricians, paediatricians, children's services managers and midwives specialising in teenage pregnancies.

The workshops and interviews were commissioned to ensure there was ample geographical coverage. The main issues arising from these 8 workshops and 24 interviews are set out in appendix C under 'Fieldwork findings'. The full fieldwork report is available on the NICE website.

How the PDG formulated the recommendations

At its meetings held between March 2006 and September 2007, the PDG considered the evidence of effectiveness and cost effectiveness to determine:

whether there was sufficient evidence (in terms of quantity, quality and applicability) to form a judgement
whether, on balance, the evidence demonstrates that the intervention is effective or ineffective, or whether it is equivocal
where there is an effect, the typical size of effect.

The PDG developed draft recommendations through informal consensus, based on the following criteria:

Strength (quality and quantity) of evidence of effectiveness and its applicability to the populations/settings referred to in the scope.
Effect size and potential impact on population health and/or reducing inequalities in health.
Cost effectiveness.
Balance of risks and benefits.
Ease of implementation and the anticipated extent of change in practice that would be required.

The PDG also considered whether a recommendation should only be implemented as part of a research programme, where evidence was lacking.

Where possible, recommendations were linked to an evidence statement(s) (see appendix C for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

The draft guidance, including the recommendations, was released for consultation in July 2007. At its meeting in September 2007, the PDG considered comments from stakeholders and the results from fieldwork, and amended the guidance. The guidance was signed off by the NICE Guidance Executive in February 2008.