Guidance

Appendix B: Summary of the methods used to develop this guidance

Appendix B: Summary of the methods used to develop this guidance

The reviews, expert papers and economic modelling reports include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the Public Health Interventions Advisory Committee (PHIAC) meetings provide further detail about the Committee's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E.

Key questions

The key questions were established as part of two scopes. (The first looked at information provision and the second at environmental changes and provision of resources, including multi‑component interventions.)

The key questions formed the starting point for the reviews of evidence and were used by PHIAC to help develop the recommendations. The overarching questions were:

1. What are the most effective and cost‑effective ways of providing information to change people's knowledge, awareness and behaviour and so prevent the first occurrence of skin cancer attributable to UV exposure?

2. What content do effective and cost‑effective primary prevention messages contain? What is the most effective and cost‑effective content?

3. What factors help to convey information to prevent the first occurrence of skin cancer attributable to UV exposure? What factors hinder the communication of primary prevention messages?

4. What changes to the natural or built environment are effective and cost effective at helping prevent the first occurrence of skin cancer attributable to UV exposure?

5. Which methods of supplying sun protection resources to prevent the first occurrence of skin cancer attributable to UV exposure are effective and cost effective?

6. Which multi‑component interventions (a combination of one or more of: supply of sun protection resources, physical changes to environment and information provision) are effective and cost effective at helping prevent the first occurrence of skin cancer attributable to UV exposure?

7. What factors help or hinder the provision or use of the following to prevent the first occurrence of skin cancer attributable to UV exposure:

  • sun protection resources

  • physical changes to the natural or built environment (such as shelters and other areas of shade in public spaces or school grounds)

  • multi‑component interventions.

These questions were made more specific for each review (see reviews for further details).

Reviewing the evidence

Effectiveness and cost‑effectiveness reviews

Three reviews of effectiveness and cost effectiveness were conducted as described in appendix A (note: review 2 was a synthesis of review 1).

Identifying the evidence

The following databases and websites were searched from 1990 onwards for reviews 1 and 4:

Databases:

  • ASSIA

  • Cochrane Central Register of Controlled Trials (CENTRAL)

  • Cochrane Database of Systematic Reviews (CDSR)

  • CRD Databases (Database of Abstracts of Reviews of Effects {DARE])

  • Cumulative Index to Nursing and Allied Health Literature (CINAHL)

  • EconLIT

  • EMBASE

  • Health Management Information Consortium (HMIC) Database

  • Health Technology Assessment Database (HTA)

  • MEDLINE

  • NHS Economic Evaluation Database (NHS EED)

  • PsycINFO

  • Science Citation Index

  • Social Science Citation Index.

Websites:

A number of additional databases and websites were searched for review 1 and 4. Reference lists and citations were also searched and experts were contacted for review 4.

Further details of the databases, websites, additional search activities, search terms and strategies are included in each of the reviews.

Selection criteria

See the reviews for details of inclusion and exclusion criteria. These varied, but in general:

  • Population:

    • Reviews 1 and 2: studies were included if they covered a population in an Organisation for Economic Co‑operation and Development (OECD) country.

    • Review 4: studies were included from both OECD and non‑OECD countries.

  • Interventions:

    • Reviews 1 and 2: universal and targeted interventions from any setting were included. For example:

      • one‑to‑one or group‑based verbal advice (with or without information resources)

      • mass‑media campaigns

      • leaflets, other information, teaching resources or printed material including posters

      • new media including social networking sites, e‑media and text messaging.

    • Review 4: Interventions were included from any setting if they covered:

      • physical or structural changes to the built or natural environment

      • supply of sun protection resources

      • multi‑component interventions combining either or both of the above with information provision.

  • Comparator:

    • Reviews 1 and 2: current information provision, do nothing or any other intervention listed above.

    • Review 4: no restrictions on type of comparator.

  • Outcomes (reviews 1, 2 and 4):

    • reduction in the incidence of mortality or morbidity from skin cancer, including sunburn

    • change in behaviour or attitudes

    • increase in knowledge and awareness of skin cancer, its causes and how to prevent it

    • costs or cost effectiveness

    • process and implementation details relating to the intervention

    • adverse or unintended effects.

  • Study design (reviews 1, 2 and 4):

    • All randomised controlled trials (RCT) and longitudinal studies were eligible for inclusion. Systematic reviews were not eligible, but were used to identify relevant primary studies via the bibliographies.

Other reviews

Two qualitative evidence reviews (reviews 3 and 5) were conducted.

Both reviews aimed to identify qualitative research on interventions to prevent the first occurrence of skin cancer attributable to UV exposure. They also aimed to synthesise the views on and experiences of (including the barriers to and facilitators for) providing this type of intervention.

Identifying the evidence

The following electronic databases and websites were searched from 1990 for reviews 3 and 5.

Databases:

  • ASSIA

  • Campbell Collaboration Library of Systematic Reviews

  • Centre for Reviews and Dissemination databases (including DARE and HTA)

  • CINAHL

  • Cochrane Library (including CENTRAL)

  • EMBASE

  • HMIC

  • MEDLINE

  • PsycINFO

  • Social Policy and Practice.

Websites:

A number of additional databases and websites were searched. Reference lists and citations were also searched. Further details of the databases, search terms and strategies are included in each of the reviews.

Selection criteria

Studies were included if they:

  • were carried out in OECD countries

  • presented qualitative data

  • were published in English.

Quality appraisal

Included papers for all five reviews were assessed for methodological rigour and quality. The NICE methodology checklists (or adapted versions of these checklists) were used for quantitative interventions and qualitative studies, as appropriate, and as set out in the NICE technical manual 'Methods for the development of NICE public health guidance' (see appendix F and H). Each study was graded (++, +, –) to reflect the risk of potential bias arising from its design and execution.

Study quality

++ All or most of the checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are very unlikely to alter.

+ Some of the checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are unlikely to alter the conclusions.

– Few or no checklist criteria have been fulfilled. The conclusions of the study are likely or very likely to alter.

Summarising the evidence and making evidence statements

The review data was summarised in evidence tables (see full reviews).

The findings from the evidence reviews were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by the external contractors and public health collaborating centres (see appendix A). The statements reflect their judgement of the strength (quality, quantity and consistency) of evidence and its applicability to the populations and settings in the scope.

Economic analysis

There are 2 reports giving the results of the economic model that was constructed for each phase incorporating data from the reviews of effectiveness and cost effectiveness:

Report 1: 'Providing public health information to prevent skin cancer: modelling strategies for primary prevention of skin cancer'

Report 2: 'Economic analysis to inform the development of NICE public health intervention guidance on information, sun protection resources and physical changes to the environment to prevent skin cancer (phase 2)'.

How PHIAC formulated the recommendations

At its meetings in March and July 2009 and May and June 2010, PHIAC considered the evidence reviews, expert reports and economic modelling to determine:

  • whether there was sufficient evidence (in terms of strength and applicability) to form a judgement

  • where relevant, whether (on balance) the evidence demonstrates that the intervention activity can be effective or cost effective or is inconclusive

  • where relevant, the typical size of effect (where there is one)

  • whether the evidence is applicable to the target groups and context covered by the guidance.

PHIAC developed draft recommendations through informal consensus, based on the following criteria.

  • Strength (type, quality, quantity and consistency) of the evidence.

  • The applicability of the evidence to the populations/settings referred to in the scope.

  • Effect size and potential impact on the target population's health.

  • Impact on inequalities in health between different groups of the population.

  • Equality and diversity legislation.

  • Ethical issues and social value judgements.

  • Cost effectiveness (for the NHS and other public sector organisations).

  • Balance of harms and benefits.

  • Ease of implementation and any anticipated changes in practice.

Where evidence was lacking, PHIAC also considered whether a recommendation should only be implemented as part of a research programme.

Where possible, recommendations were linked to evidence statements (see appendix C for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

The draft guidance, including the recommendations, was released for consultation in August 2010. At its meeting in October 2010, PHIAC amended the guidance in light of comments from stakeholders and experts. The guidance was signed off by the NICE Guidance Executive in December 2010.