Appendix B: Summary of the methods used to develop this guidance | Cardiovascular disease prevention | Guidance

Introduction

The reviews, expert reports and economic analysis include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the PDG meetings provide further detail about the Group's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E.

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and were used by the PDG to help develop the recommendations.

The overarching questions were:

Which multiple risk-factor interventions are effective and cost effective in preventing the onset of cardiovascular disease (CVD) within a given population (primary prevention)?
How does effectiveness and cost effectiveness vary between different population groups?

The subsidiary question was:

What barriers and facilitators influence the effectiveness of multiple risk-factor programmes aimed at reducing CVD (or the risk factors associated with CVD) among a given population (including subgroups experiencing health inequalities, where the data allows)?

These questions were made more specific for each review (see reviews for further details).

Single risk factors were considered using expert testimony. See appendix C.

Reviewing the evidence

Three reviews of effectiveness (reviews 1,2 and 3), one qualitative review (review 4), one primary study of barriers and facilitators (review 5) and one review of cost effectiveness (review 6) were conducted.

Identifying the evidence

The following databases were searched for randomised controlled trials (RCTs); controlled before-and-after trials; cohort studies; case–control studies; before-and-after studies; and interrupted time series (from 1970 onwards):

ASSIA (Applied Social Science Index and Abstracts)
CINAHL (Cumulative Index of Nursing and Allied Health Literature)
Cochrane Database of Systematic Reviews (CDSR)
Cochrane Library (Wiley)
Database of Abstracts of Reviews of Effects (DARE)
DH-Data
EMBASE
Health Management Information Service (HELMIS)
Health Technology Assessment (HTA)
HMIC (Health Management Information Consortium)
King's Fund Database
MEDLINE
MEDLINE In Process
PsycINFO

The following websites were also searched:

Centre for the Evaluation of Public Health Interventions, London School of Hygiene & Tropical Medicine
Cochrane Public Health Group
Health evidence
The Campbell Collaboration
The Evidence for Policy and Practice Information and Coordinating Centre

Further details of the databases, search terms and strategies used are included in the review reports.

Selection criteria

Studies were included in the effectiveness reviews if they:

Involved a population at least the size of one covered by a UK primary care trust.
Were based in an Organisation for Economic Co-operation and Development (OECD) country, another developed country or within a World Health Organization region.
Included primary prevention strategies to tackle at least two of the key risk factors for CVD.

Studies were excluded if they were:

Confined to populations clinically diagnosed as being at high risk of CVD or diagnosed with CVD.
Published before 1970.
Not published in English.

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual 'Methods for the development of NICE public health guidance'. Each study was graded (++, +, –) to reflect the risk of potential bias arising from its design and execution.

Study quality

++ All or most of the checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are very unlikely to alter.

+ Some of the checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are unlikely to alter the conclusions.

– Few or no checklist criteria have been fulfilled. The conclusions of the study are likely or very likely to alter.

Economic analysis

The economic analysis consisted of a review of economic evaluations (review 6) and a cost effectiveness analysis.

Review of economic evaluations

The same protocol was used to conduct the literature reviews for all phases of the review. In a minor departure from the protocol, the list of included study designs was extended to include cost-consequences.

The following databases were searched from 1970 to August 2008:

ECONLIT
EMBASE
MEDLINE
NHS EED database (Cochrane Library, Wiley).

The search was limited to articles published from 1970 onwards and in the English language.

In addition to the general bibliographic database searches, specific searches were conducted for each programme found during the general searches to ensure all published evaluations, particularly economic evaluations, were identified.

Study quality was assessed using an evidence form based on the 'Methods for the development of NICE public health guidance' (second edition 2009) and adapted to reflect the parameters of this review. It was supplemented with questions from the Drummond checklist (Drummond MF [1996] Guidelines for authors and peer reviewers of economic submissions to the BMJ. London: BMJ).

The selection criteria were the same as for the effectiveness reviews (see pages 53 to 54). The following study types were included: cost–benefit, cost-effectiveness and cost–utility analyses.

Modelling

An economic model was constructed to incorporate data from the reviews of effectiveness and cost effectiveness. The results are reported in Prevention of cardiovascular disease at population level: modelling strategies for primary prevention of cardiovascular disease.

Fieldwork

Fieldwork was carried out to evaluate how relevant and useful NICE's recommendations are for practitioners and how feasible it would be to put them into practice. It was conducted with practitioners and commissioners who are involved in activities relevant to cardiovascular disease. They included those working in the food industry (such as food retailers, food producers and trade associations), local planning departments, local authority catering representatives and representatives from the NHS and PCTs.

The fieldwork comprised:

Eight half-day workshops.
Nine case studies (interviews involving practitioners, commissioners and professionals). Each reviewed recommendations made in the following specific areas:
- health impact assessments
- CVD prevention programmes (two case studies)
- take-away planning
- training caterers
- public sector food provision
- physically active travel
- the dairy industry
- reformulation of food.
Telephone interviews covering the following topics: mass media, government departments, and key food industry producers.

The studies were commissioned to ensure there was ample geographical coverage. The main issues arising from these studies are set out in appendix C under fieldwork findings. See the full fieldwork report on prevention of cardiovascular disease at population level.

How the PDG formulated the recommendations

At its meetings between September 2008 and July 2009, the PDG considered the evidence of effectiveness, expert reports and cost effectiveness to determine:

whether there was sufficient evidence (in terms of strength and applicability) to form a judgement
where relevant, whether (on balance) the evidence demonstrates that the intervention or programme/activity can be effective or is inconclusive
where relevant, the typical size of effect (where there is one)
Whether the evidence is applicable to the target groups and context covered by the guidance.

The PDG developed draft recommendations through informal consensus, based on the following criteria:

Strength (type, quality quantity and consistency) of the evidence.
The applicability of the evidence to the populations/settings referred to in the scope.
Effect size and potential impact on the target population's health.
Impact on inequalities in health between different groups of the population.
Equality and diversity legislation.
Ethical issues and social value judgements.
Cost effectiveness (for the NHS and other public sector organisations).
Balance of harms and benefits.
Ease of implementation and any anticipated changes in practice.

Where possible, recommendations were linked to evidence statements (see appendix C). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

The draft guidance, including the recommendations, was released for consultation in October 2009. At its meeting in December 2009, the PDG amended the guidance in light of comments from stakeholders and experts and the fieldwork. The guidance was signed off by the NICE Guidance Executive in May 2010.