Appendix B Summary of the methods used to develop this guidance | Obesity: working with local communities | Guidance

Introduction

The reviews, primary research, commissioned reports and economic modelling include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the Programme Development Group (PDG) meetings provide further detail about the Group's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E.

Key questions

The key questions were established as part of the scope. They formed the starting point for consideration of the reviews of evidence and were used by the PDG to help develop the recommendations. The key questions were:

What are the essential elements of a local, community-wide approach to preventing obesity that is sustainable, effective and cost effective?
What barriers and facilitators may influence the delivery and effectiveness of a local, community-wide approach (including for specific groups)?
Who are the key leaders, actors and partners and how do they work with each other?
What factors need to be considered to ensure local, community-wide approaches are robust and sustainable?
What does effective monitoring and evaluation look like?
Can the cost effectiveness of local, community-wide obesity interventions be established and, if so, what is the best method to use?

Reviewing the evidence

Effectiveness reviews

One review of effectiveness was conducted (review 2).

Identifying the evidence

A number of databases were searched in July 2010 for interventions published in English from 1990 onwards. See the review for details.

General health and topic-specific websites and other sources of grey literature were also searched including:

Scrutiny committee reports (searched via an Internet search engine)
ZeTOC database (British Library)
ISI proceedings (Web of Science)
Conference Proceedings Citation Index (Web of Science).

Selection criteria

Studies were included in the effectiveness review if they:

demonstrated core features of a whole-system approach (as identified in review 1) to preventing obesity or smoking
covered whole populations or communities and reported on outcome measures or other indicators for an intervention
used comparative study designs
were published from 1990 onwards in English.

Studies were excluded if they:

did not report on the outcomes listed
only presented a single component of an intervention or strategy
did not focus on obesity prevention, improving physical activity or diet, or smoking prevention.

Other reviews

One review was undertaken to define a 'whole-system approach' (review 1) and one review of qualitative data was undertaken to consider the barriers and facilitators to such an approach (review 3).

Identifying the evidence

For reviews 1 and 3, the databases and websites searched were the same as for the effectiveness review (see above).

Selection criteria

Studies were included in review 1 if they considered:

the theory, key elements and relationships of a whole-system approach
a whole-system approach in relation to obesity or smoking prevention.

Qualitative studies were included in review 3 if they focused on:

any 'whole-community' programme in the UK
'whole-community' obesity and smoking prevention programmes, including those delivered in schools or workplaces in Organisation for Economic Co-operation and Development (OECD) countries.

Studies were excluded from review 3 if they focused on:

people's opinions about eating and exercise and their understanding of the issues around obesity, for example, food choices
community engagement, unless there were elements specific to obesity prevention
relationships between members of a single agency (for example, a primary care team)
a single setting (even where the intervention was part of a multi-agency initiative) or a single aspect of health (for example, physical activity or diet).

Quality appraisal

For review 1, included papers were assessed according to whether they provided a coherent account of the concepts and approaches taken and their relationship to each other. (Those that provided more information along these lines were considered better 'quality'.)

For the effectiveness review (review 2), included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual Methods for the development of NICE public health guidance (see appendix E). Each study was graded (++, +, −) to reflect the risk of potential bias arising from its design and execution.

Study quality

++ All or most of the checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are very unlikely to alter.

+ Some of the checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are unlikely to alter the conclusions.

− Few or no checklist criteria have been fulfilled. The conclusions of the study are likely or very likely to alter.

For review 3, the qualitative research studies were assessed using a thirteen-question checklist to determine:

the clarity of descriptions
the appropriateness of the aims and methods
the evidence for the findings
logical and theoretical coherence.

Summarising the evidence and making evidence statements

The review data was summarised in evidence tables (see full evidence reviews).

The findings from the evidence reviews were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by the public health collaborating centre (see appendix A). The statements reflect their judgement of the strength (quality, quantity and consistency) of evidence and its applicability to the populations and settings in the scope.

Commissioned report

Primary, qualitative research was commissioned (September 2011) to understand how local teams can work together effectively to prevent obesity in local communities. The opinions and experiences of the 93 participants are reported in 'Implementing community-wide action to prevent obesity: opinions and experiences of local public health teams and other relevant parties'.

Cost effectiveness

There was a review of economic evaluations and an economic modelling report.

Review of economic evaluations

The obesity-related Reference Manager databases were searched for economic evidence as part of reviews 1 and 2. In addition, selected new searches were undertaken in economic bibliographic databases (NHS EED and EconLit). As a result, four economic evaluations were selected and summarised narratively.

The generic tool for economic evaluations (Drummond and Jefferson 1996) was used for quality assessment.

Economic modelling report

An economic logic model was constructed to explore the circumstances in which a collaboration of two or more local organisations could usually be expected to be cost effective. The model aimed to deduce the direction of change of interventions, but not the magnitude of that change.

The results are reported in the economic modelling report 'Cost effectiveness analysis in partnership working for reducing obesity and other long-term conditions.'

How the PDG formulated the recommendations

At its meetings from July 2011 to February 2012, the Programme Development Group (PDG) considered the evidence, expert reports, primary research and cost effectiveness to determine:

whether there was sufficient evidence (in terms of strength and applicability) to form a judgement
where relevant, whether (on balance) the evidence demonstrates that the intervention or programme/activity can be effective or is inconclusive
where relevant, the typical size of effect (where there is one)
whether the evidence is applicable to the target groups and context covered by the guidance.

The PDG developed draft recommendations through informal consensus, based on the following criteria:

strength (type, quality, quantity and consistency) of the evidence
the applicability of the evidence to the populations or settings referred to in the scope
effect size and potential impact on the target population's health
impact on inequalities in health between different groups of the population
equality and diversity legislation
ethical issues and social value judgements
cost effectiveness (for the NHS and other public sector organisations)
balance of harms and benefits
ease of implementation and any anticipated changes in practice.

The PDG noted that effectiveness can vary according to the context.

Where evidence was lacking, the PDG also considered whether a recommendation should only be implemented as part of a research programme.

Where possible, recommendations were linked to an evidence statement(s) (see appendix C for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

The draft guidance, including the recommendations, was released for consultation in May 2012. At its meeting in July 2012 the PDG amended the guidance in light of comments from stakeholders. The guidance was signed off by the NICE Guidance Executive in October 2012.