Guidance

Appendix B: Summary of the methods used to develop this guidance

Appendix B: Summary of the methods used to develop this guidance

Introduction

The reports of the reviews and economic appraisal include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the PHIAC meetings provide further detail about the Committee's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E.

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and facilitated the development of recommendations by PHIAC. The 2 overarching questions focused on:

  • the use of statins to combat cardiovascular disease (CVD)

  • smoking cessation activities.

Statins

  • What are the most effective and cost-effective methods of identifying and supporting people at increased risk of developing CVD, or who already have CVD?

    • What are the most effective and cost-effective methods of improving access to services, under what circumstances, for whom and when?

    • What type of support is most effective for different groups, under what circumstances and when?

    • Is there a trade-off between equity and efficiency?

Smoking cessation

  • What are the most effective and cost-effective methods of identifying and supporting people aged 16 years and over who want to stop smoking, in particular, pregnant women, manual workers and those from disadvantaged backgrounds?

    • What are the most effective and cost-effective methods of improving access to services, under what circumstances, for whom and when?

    • What type of support is most effective for different groups, under what circumstances and when?

    • Is there a trade-off between equity and efficiency?

Reviewing the evidence of effectiveness

Two reviews of effectiveness were conducted.

Identifying the evidence

The following databases were searched (from 1995 to 2007):

  • AMED (Allied and Complementary Medicine)

  • ASSIA (Applied Social Science Index and Abstracts)

  • British Nursing Index

  • CINAHL (Cumulative Index of Nursing and Allied Health Literature)

  • Cochrane Central Register of Controlled Trails

  • Cochrane Database of Systematic Reviews (CDSR)

  • Database of Abstracts of Reviews of Effectiveness (DARE)

  • EMBASE

  • EPPI Centre Databases

  • HMIC (Health Management Information Consortium – comprises King's Fund and DH-Data databases)

  • MEDLINE

  • PsychINFO

  • SIGLE (System for Information on Grey Literature in Europe)

  • Social Policy and Practice

  • Sociological Abstracts

Other relevant databases (including sources of grey literature) were also searched, along with references from included studies. The following websites were searched:

In addition, information was sought from experts.

Selection criteria

Studies of primary and secondary prevention activities were included in the effectiveness reviews if they aimed to:

  • find and then support adults at increased risk of developing (or with established) CHD (coronary heart disease; note, the statins search included CVD)

  • provide adults at increased risk of developing (or with established) CHD with support services – or improved access to those services (note, the statins search included CVD)

  • find and help people who smoke (aged 16 years and over) to stop or reduce the habit

  • provide people who smoke (aged 16 years and over) with smoking cessation services – or improve their access to those services.

Studies were excluded if the interventions:

  • did not aim to reduce or eliminate premature deaths from CHD or other smoking-related causes

  • tackled the wider determinants of health inequalities (for example, using macro-level policies to tackle poverty and economic disadvantage).

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual 'Methods for development of NICE public health guidance'. Each study was described by study type and graded (++, +, -) to reflect the risk of potential bias arising from its design and execution.

Study type

  • Meta-analyses, systematic reviews of randomised controlled trials (RCTs) or RCTs (including cluster RCTs).

  • Systematic reviews of, or individual, non-randomised controlled trials, case-control studies, cohort studies, controlled before-and-after (CBA) studies, interrupted time series (ITS) studies, correlation studies.

  • Non-analytical studies (for example, case reports, case series).

  • Expert opinion, formal consensus.

Study quality

++ All or most criteria have been fulfilled. Where they have not been fulfilled the conclusions are thought very unlikely to alter.

+ Some criteria fulfilled. Those criteria that have not been fulfilled or not adequately described are thought unlikely to alter the conclusions.

- Few or no criteria fulfilled. The conclusions of the study are thought likely or very likely to alter.

Summarising the evidence and making evidence statements

The review data was summarised in evidence tables (see the full evidence reviews).

The findings from the reviews were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements reflect the strength (quantity, type and quality) of evidence and its applicability to the populations and settings in the scope.

Study of current practice

The mapping review aimed to identify and describe smoking cessation interventions and the provision of statins in disadvantaged areas and among disadvantaged individuals. It looked at:

  • ways of reaching people who need this type of support (proactive case finding)

  • how to encourage those people to keep in touch with services (retention)

  • service accessibility.

Projects and interventions were identified via:

  • telephone interviews

  • documentary analysis

  • questionnaires

  • scanning of selected conference archives and databases (where these were available online).

Work was carried out in 2 phases over a 3-month period. In phase one, semi-structured telephone interviews were carried out with a wide range of national and regional organisations to identify local contacts, interventions and approaches. Selected conference archives and project databases were also scanned. In phase 2, interventions were identified through questionnaires completed by local stakeholders and by analysing local documents. Full details are in the supporting evidence documents.

Economic appraisal

The economic appraisal consisted of a review of economic evaluations, 4 cost-effectiveness reports and a supplementary cost-effectiveness analysis. The cost-effectiveness reports covered:

  • Statins: 1 report focused on disadvantaged people, the other looked at the general population. They focused on how to: identify people at risk, improve or increase their access to services, ensure people who require treatment stay in the system and adhere to the treatment protocol.

  • Smoking cessation: 1 report focused on disadvantaged people, the other looked at the general population. They focused on how to: identify people at risk, improve or increase their access to services, ensure people who require treatment stay in the system and adhere to the treatment protocol.

Review of economic evaluations

The review was conducted using the databases listed for the effectiveness reviews and the following economic databases:

  • Econlit

  • Health Economic Evaluation Database (HEED)

  • NHS Economic Evaluation Database (NHS EED).

The small number of studies involved and the difficulties involved in making direct comparisons across studies (for instance, due to lack of information on the base year used to estimate prices) meant that it was not possible to undertake a quantitative synthesis of the results.

Cost-effectiveness analysis

An economic model was constructed to incorporate data from the reviews of effectiveness and cost effectiveness. The approach was applied to all 4 cost-effectiveness reports. The results are reported in:

  • 'Economic analysis of interventions to improve the use of statins interventions in the general population.'

  • 'Economic analysis of interventions to improve the use of statins in disadvantaged populations.'

  • 'Economic analysis of interventions to improve the use of smoking cessation interventions in the general population.'

  • 'Economic analysis of interventions to improve the use of smoking cessation interventions in disadvantaged populations.'

An additional, supplementary economic analysis was undertaken to answer a number of questions posed by PHIAC.

See the economic analysis reports.

Fieldwork

Fieldwork was carried out to evaluate the relevance and usefulness of NICE guidance for practitioners and the feasibility of implementation. It was conducted with practitioners and commissioners who are involved in smoking cessation services and statin provision. Participants included: strategic health authority directors, primary care trust directors of public health and public health teams, commissioning managers and performance managers, GPs and primary care nurses. They also included community pharmacists, health trainers and managers and representatives from other public and voluntary organisations, including New Deal for Communities.

The fieldwork comprised:

A qualitative study involving a range of different professionals across 4 locations (Coventry, Liverpool, London and Northampton) carried out by Dr Foster Intelligence. The main issues arising from this study are set out in appendix C under fieldwork findings. See the full fieldwork report 'Reducing the rate of premature deaths from CVD and other smoking-related diseases: finding and supporting those most at risk and improving access to services'.

How PHIAC formulated the recommendations

At its meetings in November 2007 and March 2008 PHIAC considered the evidence of effectiveness and cost effectiveness to determine:

  • whether there was sufficient evidence (in terms of quantity, quality and applicability) to form a judgement

  • whether, on balance, the evidence demonstrates that the intervention is effective or ineffective, or whether it is equivocal

  • where there is an effect, the typical size of effect.

PHIAC developed draft recommendations through informal consensus, based on the following criteria.

  • Strength (quality and quantity) of evidence of effectiveness and its applicability to the populations/settings referred to in the scope.

  • Effect size and potential impact on population health and/or reducing inequalities in health.

  • Cost effectiveness (for the NHS and other public sector organisations).

  • Balance of risks and benefits.

  • Ease of implementation and the anticipated extent of change in practice that would be required.

Where possible, recommendations were linked to an evidence statement(s) (see appendix C for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

The draft guidance, including the recommendations, was released for consultation in April 2008. At its meeting in June 2008, PHIAC considered comments from stakeholders and the results from fieldwork and amended the guidance. The guidance was signed off by the NICE Guidance Executive in July 2008.