Guidance

Appendix B: Summary of the methods used to develop this guidance

Appendix B: Summary of the methods used to develop this guidance

Introduction

The reports of the reviews and economic appraisal include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it. The minutes of the PDG meetings provide further detail about the Group's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E.

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and facilitated the development of recommendations by the PDG. The overarching questions were:

1. Which strategies, policies, campaigns, interventions and approaches are effective and cost effective in helping children of different ages (with low levels of physical activity) to become more physically active?

2. What are the barriers and facilitators to children's participation in physical activity?

3. Which physical activity strategies, policies, campaigns, interventions and approaches are effective and cost effective in reducing health inequalities among pre-school and school-age children?

These questions were refined further in relation to the topic of each review (see reviews for further details).

Reviewing the evidence

A total of eight reviews were conducted.

Identifying the evidence

Searches were conducted for studies published from January 1990 to April 2007 (except where stated).

The following databases were searched for reviews 2, 4, 5, 6, and 7. Additional searches for these reviews and details for the other reviews are listed separately.

  • Applied Social Sciences Index and Abstracts

  • ArticleFirst

  • Cambridge Scientific Abstract

  • CINAHL

  • Cochrane Library

  • CSA Environmental Sciences

  • EMBASE

  • ERIC

  • Geobase

  • Global Health

  • ISI Social Science Citation Index

  • MEDLINE

  • PubMed

  • PsycINFO

  • Science Citation Index

  • SIGLE

  • Sociological Abstracts

  • SPORTDiscus

  • TRIS online

  • Web of Science.

Review 1: 'Descriptive epidemiology'

The following databases were searched from 2001 for longitudinal or cohort studies:

  • MEDLINE

  • Metalib (including ArticleFirst, Physical Education Index, PSYCinfo, SPORTDiscus, Web of Science, Zetoc)

  • PubMed.

Web searches were also conducted: a key source was a briefing paper on obesity produced by the NHS.

Review 2: 'Correlates of physical activity in children: a review of quantitative systematic reviews'

In addition to searching the main databases from 2000 to April 2007, manual searches were conducted of the following key peer-reviewed journals:

  • 'International journal of behavioural nutrition and physical activity'

  • 'Journal of physical activity & health'

  • 'Obesity reviews'

  • 'Pediatric exercise science'

  • 'Preventive medicine'

  • 'Sports medicine'.

Primary research articles, reviews and book chapters, as well as research team members' files were also searched. In addition, the websites of four UK and US organisations involved in commissioning, undertaking or cataloguing research on physical activity and young people were searched for 'grey' literature. These were:

Review 3: 'The views of children on the barriers and facilitators to participation in physical activity: a review of qualitative studies'

The following databases were searched to identify non-intervention qualitative studies published since 1990:

  • CINAHL

  • CSA Environmental Sciences

  • EMBASE

  • Environmental Sciences and Pollution Management

  • ERIC

  • Index to Thesis

  • PsycINFO

  • Science Citation Index and SSCI

  • SIGLE (ends 2005)

  • SPORTDiscus

  • TRIS online.

In addition, the websites of four UK and US organisations involved in commissioning, undertaking or cataloguing research on physical activity and young people were searched for 'grey' literature. These were:

Reviews 4, 5, 6 and 7: 'Under eights', 'Children and active travel', 'Adolescent girls', and 'Family and community'

In addition to the main database search, the following were also searched:

  • Environline

  • EPPI Centre Databases

  • HMIC

  • NRR

  • TRANSPORT

  • The 'Journal of physical activity and health'.

Review 8: 'Review of learning from practice: children and active play'

Two PDG members helped to identify a list of relevant references, based on an iterative search of material in the Children's Play Information Service at the National Children's Bureau. This was supplemented by web searches and re-interrogation of the search results from the other reviews. References were screened for relevance by two reviewers.

Selection criteria

Inclusion and exclusion criteria for each review varied. However, in general studies were included as follows.

  • Review 1: Studies conducted in England or the UK (as long as they included England) that questioned children and young people on physical activity in childhood and adulthood.

  • Review 2: Studies classified as review papers and using systematic methodologies, if they looked at the association between quantitatively measured variables and children or adolescents' (<19 years old) physical activity.

  • Review 3: Studies which explored children's, adolescents' (<19 years old) or carers' experiences of sport, play and active travel. Methods and results had to be clearly reported and the study had to be relevant to the UK.

  • Reviews 4, 5, 6 and 7: Intervention studies on children under eight, active travel, adolescent girls, and family and communities, if they reported on physical activity or physical skills. See the reviews for further details.

  • Review 8: Material directly applicable to the UK. It was not limited by quality or study design, but needed to illustrate or describe the opinions and experiences of children, parents and practitioners about how to stimulate – or help stimulate – active play.

Studies were excluded if:

  • they focused on treating obesity

  • they were from less economically developed countries

  • they were studies about ethnic groups that do not have large populations in England

  • the intervention involved the school curriculum/physical education

  • the study involved a change to the built or natural environment.

Quality appraisal

For reviews 3–7, included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual 'Methods for development of NICE public health guidance' (see appendix E). Each study was described by study type and graded (++, +, -) to reflect the risk of potential bias arising from its design and execution.

Study type

  • Meta-analyses, systematic reviews of randomised controlled trials (RCTs) or RCTs (including cluster RCTs).

  • Systematic reviews of, or individual, non-randomised controlled trials, case-control studies, cohort studies, controlled before-and-after (CBA) studies, interrupted time series (ITS) studies, correlation studies.

  • Non-analytical studies (for example, case reports, case series).

  • Expert opinion, formal consensus.

Study quality

++ All or most of the criteria have been fulfilled. Where they have not been fulfilled the conclusions are thought very unlikely to alter.

+ Some criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are thought unlikely to alter the conclusions.

- Few or no criteria fulfilled. The conclusions of the study are thought likely or very likely to alter.

The studies were also assessed for their applicability to the UK.

Summarising the evidence and making evidence statements

The review data was summarised in evidence tables (see full reviews).

The findings from the included papers in each review were synthesised and used as the basis for a number of evidence statements relating to each review question. The evidence statements reflect the strength (quantity, type and quality) of evidence and its applicability to the populations and settings in the scope.

Economic analysis

The economic appraisal consisted of a review of economic evaluations and a cost-effectiveness analysis.

Cost-effectiveness analysis: 'A cost-effectiveness scenario analysis of four interventions to increase child and adolescent physical activity: the case of walking buses, free swimming, dance classes and community sports'

An economic model was constructed to incorporate data from the reviews of effectiveness (reviews 4,5,6,7) and cost effectiveness.

Fieldwork

Fieldwork was carried out to evaluate how relevant and useful NICE's recommendations are for practitioners and how feasible it would be to put them into practice. It was conducted with practitioners and commissioners who are involved in providing physical activity services for children and young people. They included those working in the NHS, education, local authorities and the voluntary and community sector.

The fieldwork comprised three studies conducted by Greenstreet Berman:

  • Ten workshops carried out in Birmingham, Liverpool, London, Newcastle with representatives from a variety of professional groups. These included: local authority sports, health and play promotion/development services and children's services; primary care trusts (health improvement representatives), county sports partnerships, Play England, Youth Sport Trust and Sport England.

  • Thirty-two telephone interviews with representatives from education, parent/carer associations, pre-school organisations and local clubs and associations. Each interview covered five of the draft recommendations.

    • Local clubs: draft recommendations 4, 6, 7, 8, 9,10, 12, 13, 14

    • Parents and carers: draft recommendations 6, 9, 10, 11, 12, 13, 14, 15

    • Education: draft recommendations 4, 6, 7, 8, 9, 10, 11, 12.

  • An online survey of schools, covering the eight draft recommendations that were directly relevant to them (4, 6, 7, 8, 9, 10, 11, 12).

The three studies were commissioned to ensure there was ample geographical and sector coverage. The main issues arising are set out in appendix C fieldwork findings. The full fieldwork report is 'Fieldwork on the promotion of physical activity, active play and sport for pre-school and school age children in family, pre-school, school and community setting'.

How the PDG formulated the recommendations

At its meetings in 2007/2008, the PDG considered the evidence to determine:

  • whether there was sufficient evidence (in terms of quantity, quality and applicability) to form a judgement

  • whether, on balance, the evidence demonstrates that the intervention is effective or ineffective, or whether it is equivocal

  • where there is an effect, the typical size of effect.

The PDG developed draft recommendations through informal consensus, based on the following criteria:

  • Strength (quality and quantity) of evidence of effectiveness and its applicability to the populations/settings referred to in the scope.

  • Effect size and potential impact on population health and/or reducing inequalities in health.

  • Cost effectiveness (for the NHS and other public sector organisations).

  • Balance of risks and benefits.

  • Ease of implementation and the anticipated extent of change in practice that would be required.

The PDG also considered whether a recommendation should only be implemented as part of a research programme where evidence was lacking.

Where possible, recommendations were linked to evidence statements (see appendix C for details of evidence statements). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

The draft guidance, including the recommendations, was released for consultation in August 2008. At its meeting in October 2008, the PDG amended the guidance in light of comments from stakeholders and experts and the fieldwork. The guidance was signed off by the NICE Guidance Executive in December 2008.