Guidance

Appendix B Summary of the methods used to develop this guidance

Appendix B Summary of the methods used to develop this guidance

Introduction

The reviews and economic analysis include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the Programme Development Group (PDG) meetings provide further detail about the Group's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E.

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and were used by the PDG to help develop the recommendations. The overarching questions were:

  • What is the effectiveness and cost effectiveness of interventions to encourage young people, especially socially disadvantaged young people, to use contraceptives and contraceptive services (including access to, and information about, contraceptive services)?

  • What are socially disadvantaged young people and their families' perceptions, views and beliefs about contraception and contraceptive services, and where do they get their information about contraception and contraceptive services?

These questions were made more specific for each review (see reviews for further details).

Reviewing the evidence

Effectiveness reviews

Five reviews were conducted:

  • a mapping review

  • 3 settings-based reviews of effectiveness

  • a review of views and barriers.

Identifying the evidence

The following databases were searched in August 2008 for both quantitative and qualitative studies (1995 to 2008) for all of the reviews:

  • Applied Social Science Index and Abstracts (ASSIA)

  • British Nursing Index

  • Cochrane Database of Systematic Reviews

  • Cinahl

  • Cochrane – Central

  • Cochrane DARE

  • Cochrane Health Technology Assessment

  • Embase

  • MEDLINE

  • PsycINFO

  • Science and Social Science Citation Indices

  • Social Care Online

Additionally, the following websites were searched for relevant publications:

  • British Association for Sexual Health and HIV

  • British Medical Association

  • Brook

  • Centre for Reviews and Dissemination

  • Connexions

  • Department for Children, Schools and Families

  • Department of Health

  • Every Child Matters

  • Faculty of Public Health

  • FPA

  • Health Protection Agency

  • Joseph Rowntree Foundation

  • Margaret Pyke Centre

  • Medical Foundation for AIDS and Sexual Health

  • National Electronic Library for Health – Guidelines Finder

  • National Electronic Library for Health – Public Health

  • NICE (and HDA)

  • Royal College of General Practitioners

  • Royal College of Nursing

  • Royal College of Obstetricians and Gynaecologists

  • Royal College of Paediatrics and Child Health

  • Royal Pharmaceutical Society of Great Britain

  • Sex Education Forum

  • Sex Education Forum at the National Children's Bureau

  • SIGN (Scottish Intercollegiate Guidelines Network)

  • Social Care Institute for Excellence

  • South West Public Health Observatories

  • Teenage Pregnancy Unit

  • US National Guidelines Clearinghouse

  • Welsh Assembly Government – Health Promotion Wales

  • World Health Organisation

Selection criteria

Studies were included in the effectiveness reviews if:

  • They included under 25s.

Studies were excluded if:

  • They focused solely on people aged 25 and older. Although a younger age cut off was not explicitly stated, consideration was also given to the Fraser guidelines for competence to consent.

  • They covered sexual health services that do not provide contraceptive services.

  • They covered sterilisation, including vasectomy.

  • They covered abortion (services which do not also provide contraception).

  • They covered use of contraceptive methods for non-contraceptive reasons, for example, for menorrhagia (heavy periods).

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual 'Methods for the development of NICE public health guidance' (see appendix E). Each study was graded (++, +, −) to reflect the risk of potential bias arising from its design and execution.

Study quality

++ All or most of the checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are very unlikely to alter.

+ Some of the checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are unlikely to alter the conclusions.

− Few or no checklist criteria have been fulfilled. The conclusions of the study are likely or very likely to alter.

The evidence was also assessed for its applicability to the areas (populations, settings, interventions) covered by the scope of the guidance. Each evidence statement concludes with a statement of applicability (directly applicable, partially applicable, not applicable).

Summarising the evidence and making evidence statements

The review data was summarised in evidence tables (see full reviews).

The findings from the reviews were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by the public health collaborating centre (see appendix A). The statements reflect their judgement of the strength (quantity, type and quality) of evidence and its applicability to the populations and settings in the scope.

Cost effectiveness

There was a review of economic evaluations and an economic modelling exercise.

Review of economic evaluations

One economic evaluation was identified within the 3 reviews which considered the cost effectiveness of an intensive, school-based intervention for teen mothers to prevent repeat pregnancies. This economic evaluation was poorly reported and appeared to contain some errors within the calculations. No other economic evaluations that met the inclusion criteria were identified by the reviews.

Economic modelling

A number of assumptions were made that could underestimate or overestimate the cost effectiveness of the interventions (see review modelling report for further details).

An economic model was constructed to incorporate data from the reviews of effectiveness and cost effectiveness. The results are reported in: Modelling the cost-effectiveness of interventions to encourage young people, especially socially disadvantaged young people, to use contraceptives and contraceptive services.

Fieldwork

Fieldwork was carried out to evaluate how relevant and useful NICE's recommendations are for practitioners and how feasible it would be to put them into practice. It was conducted with practitioners and commissioners who are involved in contraceptive services, including those working in the NHS, private providers, education and the voluntary sector.

The fieldwork comprised:

  • Focus groups carried out nationally by GHK with practitioners and commissioners working in the NHS, education and the private and voluntary sectors.

  • telephone interviews carried out by GHK.

The 2 studies were commissioned to ensure there was ample geographical coverage. The main issues arising from these 2 studies are set out in appendix C under fieldwork findings.

How the PDG formulated the recommendations

At its meetings in 2009 and 2010, the Programme Development Group (PDG) considered the evidence and cost effectiveness to determine:

  • whether there was sufficient evidence (in terms of quantity, quality and applicability) to form a judgement

  • where relevant, whether (on balance) the evidence demonstrates that the intervention or programme/activity can be effective or is inconclusive

  • where relevant, the typical size of effect (where there is one)

  • whether the evidence is applicable to the target groups and context covered by the guidance.

The PDG developed draft recommendations through informal consensus, based on the following criteria:

  • Strength (type, quality, quantity and consistency) of the evidence.

  • The applicability of the evidence to the populations/settings referred to in the scope.

  • Effect size and potential impact on the target population's health.

  • Impact on inequalities in health between different groups of the population.

  • Equality and diversity legislation.

  • Ethical issues and social value judgements.

  • Cost effectiveness (for the NHS and other public sector organisations).

  • Balance of harms and benefits.

  • Ease of implementation and any anticipated changes in practice.

Where evidence was lacking, the PDG also considered whether a recommendation should only be implemented as part of a research programme.

Where possible, recommendations were linked to evidence statements (see appendix C for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

The guidance was due to be published in November 2010. However it was put on hold pending a Government review of NICE's public health work and while the Department of Health finalised its sexual health framework.

The recommendations have not changed, only factual amendments have been made, for example, where names of organisations have been changed.