Guidance

Summary of the methods used to develop this guideline

Summary of the methods used to develop this guideline

Introduction

The reviews include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the Public Health Advisory Committee (PHAC) meetings provide further detail about the Committee's interpretation of the evidence and development of the recommendations.

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and were used by the PHAC to help develop the recommendations. The overarching questions were:

Question 1: How effective and cost effective are exercise referral schemes? What are the most important factors that influence effectiveness and cost effectiveness?

Question 2: What factors influence referral to an exercise referral scheme?

Question 3: What factors influence attendance at, and successful completion of, an exercise referral scheme?

Question 4: What factors influence longer-term participation in physical activity following attendance on an exercise referral scheme?

The subsidiary questions included:

1. What factors influence the effectiveness of exercise referral schemes (for example, age, gender or socioeconomic status)?

2. How aware are health practitioners of exercise referral schemes?

3. Are there any adverse or unintended effects from exercise referral schemes (for example, unintentional injuries)?

4. Are exercise referral schemes available to, and accessible by, different populations?

5. How are initial assessments and medical records transferred from primary care to physical activity services for people attending exercise referral schemes?

6. What 'exit strategies' are in place for people once they have completed an exercise referral scheme?

These questions were made more specific for each review.

Reviewing the evidence

Review of effectiveness, uptake and adherence

An effectiveness, uptake and adherence review was commissioned by the National Institute for Health Research Health Technology Appraisal programme (NIHR HTA).

This review is an update of a systematic review commissioned by NIHR HTA and carried out by Pavey et al. in 2011. It was specifically commissioned to inform NICE's guidance. NICE set out the parameters and protocols for the review but it is based on the NIHR's methods (summarised below). The review also includes an economic model. For more details, see evidence review 1: a systematic review and economic evaluation of exercise referral schemes in primary care: a short report.

Identifying the evidence

Several databases were searched in September 2013 for randomised control trials published since October 2009 (the date of the previous searches by Pavey et al. 2011).

Selection criteria

Studies were included in the review if they:

  • were based on randomised controlled trials

  • included adults (aged 18 or older) without a medical diagnosis and for whom an exercise referral scheme was deemed appropriate

  • included counselling (face-to-face or by telephone), written materials or supervised exercise training

  • included outcomes on: physical activity, physical fitness, health, adverse events, and uptake and adherence to the scheme.

Studies were excluded if:

  • they focused exclusively on people with a medical diagnosis

  • interventions were not part of an exercise referral scheme

  • interventions did not include a clear assessment of physical activity levels and a clear referral process.

Details can be found in evidence review 1: a systematic review and economic evaluation of exercise referral schemes in primary care: a short report.

Quality appraisal

Included papers were assessed for methodological rigour and quality using the Cochrane risk of bias tool to assess study quality. Study quality was checked against the following factors:

  • method of randomisation

  • allocation concealment

  • blinding

  • numbers of participants randomised, excluded and lost to follow up.

  • whether intent to treat analysis has been performed

  • methods for handling missing data

  • baseline comparability between groups.

Analysis and synthesis

Data from new studies published since 2009 were tabulated and discussed in a narrative review. These were integrated with data from the studies identified and analysed by Pavey et al. in 2011. Meta-analyses were used to estimate a summary measure of effect on relevant outcomes. These were based on intention-to-treat analyses.

Review Manager software was used for the meta-analysis to study fixed and random effects models. Heterogeneity was explored by considering: study populations, methods and interventions; visualisation of results; and, in statistical terms, by the χ2 test for homogeneity and the I2 statistic.

A qualitative thematic analysis of the discussion and conclusion sections of the included randomised controlled trials was undertaken (as per Pavey et al. 2011). The aim was to understand factors that predict uptake of, and adherence to, exercise referral schemes. The results are described in a narrative. A logic model explains the associations between multiple and varied barriers and facilitators to uptake and adherence.

NICE-commissioned review of context, barriers and facilitators

One review of context, barriers and facilitators was conducted using NICE methods and processes, evidence review 2: the factors that influence referral to, attendance at and successful completion of exercise schemes and longer term participation in physical activity.

Identifying the evidence

Several databases and websites were searched in July 2013 for qualitative and grey literature from January 1995 to June 2013. See evidence review 2 for details.

Selection criteria

Studies were included in the review if they:

  • were qualitative and observational that is, they reported the views, perceptions and beliefs of those using and delivering exercise referral schemes

  • mainly covered people aged 19 years and older who were potential or actual users of an exercise referral scheme

  • included exercise referral schemes involving assessments and referrals by health professionals.

Systematic reviews were also identified and 'unpicked' for relevant studies meeting the inclusion criteria.

Studies were excluded if they:

  • mainly focused on people under 19

  • did not include an exercise referral scheme

  • covered only physical activity rehabilitation programmes used to aid recovery from specific health conditions.

See evidence review 2 for details of the inclusion and exclusion criteria.

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in Methods for the development of NICE public health guidance. Each study was graded (++, +, −) to reflect the risk of potential bias arising from its design and execution.

Study quality

++ All or most of the checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are very unlikely to alter.

+ Some of the checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are unlikely to alter the conclusions.

− Few or no checklist criteria have been fulfilled. The conclusions of the study are likely or very likely to alter.

The evidence was also assessed for its applicability to the areas (populations, settings, interventions) covered by the scope of the guidance. Each evidence statement concludes with a statement of applicability (directly applicable, partially applicable, not applicable).

Summarising the evidence and making evidence statements

The review data were summarised in evidence tables (see the review).

The findings from the review were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by the external contractors (see supporting evidence). The statements reflect their judgement of the strength (quality, quantity and consistency) of evidence and its applicability to the populations and settings in the scope.

Cost effectiveness

No formal review of economic studies was conducted because a preliminary search retrieved no additional economic evidence directly related to exercise referral schemes.

Economic modelling

An existing economic model, used for NICE's guideline on physical activity: brief advice for adults in primary care, was updated. This model is a direct update of the models conducted for NICE's guideline on 4 commonly used methods to increase physical activity. See evidence review 1: a systematic review and economic evaluation of exercise referral schemes in primary care: a short report.

The economic model updated 3 groups of parameters:

  • estimates of the relative clinical effectiveness of exercise referral schemes versus not using them

  • costs – these were inflated to 2013 values using Personal Social Services Research Unit inflation indices

  • starting age – this has been changed to 50 (the mean age used in the studies to collect effectiveness data).

Additional analyses conducted before the first committee meeting

The original base case assumption that physical activity offers a 10-year protective effect related to coronary heart disease, stroke and diabetes was based on cohort studies. These studies had follow-up periods of 19 years (for coronary heart disease and stroke) and 12 years (for diabetes). Additional analyses were undertaken to test the model using these different time periods. See appendix 7 of review 1.

The model is particularly sensitive to the feel-good factor ('process utility' gain) attributable to physical activity. In the base case analysis, it is assumed that this lasted for only 1 year. However, it is likely that some people who continue to be physically active at 1 year will carry on being physically active in the longer term (and so continue to benefit from the feel-good factor).

To explore the effect of a gradual fall-off in the number remaining physically active, this 'process utility' has been applied for 10 years. But the model assumes there will be a linear decrease in the number who are physically active over those 10 years and that no-one will benefit from the feel good factor after 10 years.

The additional analysis also explored the effect on the incremental cost-effectiveness ratios (ICER) of combining these 2 less conservative assumptions about the longer-term benefits.

Additional analyses conducted before the second committee meeting

Following the first Public Health Advisory Committee (PHAC) meeting, further additional analyses were undertaken to inform the Committee's discussion at its second meeting. See appendix 8 of review 1.

Incremental cost-effectiveness ratios (ICERs) were conducted for a 'combined scenario analysis' incorporating:

  • costs for providing brief advice in the comparator arm

  • efficacy estimates from the intention-to-treat analysis

  • a 10-year linear fall-off in the 'feel good' factor (process utility) associated with being physically active, applied with the original base-case assumption that the protective effects of exercise are limited to 10 years.

In addition, several sensitivity analyses were undertaken. These:

  • Explored the effect of using EQ-5D data from a study by Murphy et al (2012) as an alternative to the process utility gain estimated by Pavey et al. (2011). The latter was applied in the model used to inform NICE public health guidance 44.

  • Explored the cost-effectiveness of less intensive exercise referral schemes.

Finally, a threshold analysis was undertaken on the intervention cost for exercise referral schemes.

Fieldwork

Fieldwork was carried out to evaluate how relevant and useful NICE's recommendations are for practitioners and how feasible it would be to put them into practice.

It was conducted with those responsible for commissioning, referring to, managing and delivering exercise referral schemes – and 'exit' strategies from those schemes. This included commissioners and practitioners working in primary care and local authorities, and private companies, social enterprises and independent contractors commissioned to deliver the schemes.

The fieldwork comprised:

  • Six group discussions involving 9 to12 participants each, carried out in Birmingham, Leeds and London by Word of Mouth.

  • 10 in-depth one-to-one interviews carried out in Birmingham, Leeds and London by Word of Mouth.

The fieldwork was commissioned to ensure there was ample geographical coverage. The main issues arising from the fieldwork is set out in section 9 under fieldwork findings. Or see the fieldwork report: field testing NICE guideline on exercise referral schemes to promote physical activity.

How the PHAC formulated the recommendations

At its meetings in December 2013 and January 2014, the Public Health Advisory Committee (PHAC) considered the evidence reviews and cost effectiveness to determine:

  • whether there was sufficient evidence (in terms of strength and applicability) to form a judgement

  • where relevant, whether (on balance) the evidence demonstrates that the intervention or programme/activity can be effective or is inconclusive

  • where relevant, the typical size of effect

  • whether the evidence is applicable to the target groups and context covered by the guideline.

The PHAC developed draft recommendations through informal consensus, based on the following criteria:

  • Strength (type, quality, quantity and consistency) of the evidence.

  • The applicability of the evidence to the populations/settings referred to in the scope.

  • Effect size and potential impact on the target population's health.

  • Impact on inequalities in health between different groups of the population.

  • Equality and diversity legislation.

  • Ethical issues and social value judgements.

  • Cost effectiveness (for the NHS and other public sector organisations).

  • Balance of harms and benefits.

  • Ease of implementation and any anticipated changes in practice.

Where evidence was lacking, the PHAC also considered whether a recommendation should only be implemented as part of a research programme.