Guidance

Appendix B Summary of the methods used to develop this guidance

Appendix B Summary of the methods used to develop this guidance

Introduction

The reviews, primary research, expert testimony and economic modelling report include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the Programme Development Group (PDG) meetings provide further detail about the Group's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E.

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and were used by the PDG to help develop the recommendations. The overarching questions were:

Which local interventions are effective and cost effective at promoting and increasing cycling and walking for recreational and travel purposes?

Which local interventions are effective and cost effective at changing population-level norms and behaviour in relation to cycling and walking for recreational and travel purposes?

What factors help or hinder the planning and delivery of walking and cycling-related interventions for recreation or travel purposes?

What factors help or prevent people from walking and cycling for recreation or travel?

What health and other outcomes may be achieved by increasing cycling and walking for travel and recreation?

These questions were made more specific for each review (see reviews for further details).

Reviewing the evidence

Effectiveness reviews

One review of effectiveness was conducted (review 1).

Identifying the evidence

A number of databases were searched in August 2011 for papers relating to walking and cycling published since 1990. See the review for details of the databases searched.

In addition, specific websites were examined and papers from stakeholders and members of the PDG were considered.

Selection criteria

Studies were included in the effectiveness review if they considered the impact of local interventions to raise awareness of, encourage or increase uptake of, walking and cycling for recreational and travel purposes.

Studies were excluded if they covered:

  • national policy, fiscal or legislative changes

  • local interventions which solely aimed to change the physical environment.

Other reviews

A review of barriers and facilitators (review 2) was conducted.

Identifying the evidence

A number of databases were searched in August 2011 for papers relating to walking and cycling published since 1990. See the review for details of the databases searched.

In addition, specific websites were examined and papers from stakeholders and members of the PDG were considered.

Selection criteria

Studies were included if they focused on interventions identified in the scope and addressed barriers and facilitators to walking and cycling.

Studies were excluded if they focused on:

  • National policy, fiscal and legislative changes.

  • Local interventions which solely aimed to change the physical environment (such as traffic-calming measures, provision of cycle parking facilities or construction of cycle routes).

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual 'Methods for the development of NICE public health guidance' (see appendix E). Each study was graded (++, +, –) to reflect the risk of potential bias arising from its design and execution.

Study quality

++ All or most of the checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are very unlikely to alter.

+ Some of the checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are unlikely to alter the conclusions.

– Few or no checklist criteria have been fulfilled. The conclusions of the study are likely or very likely to alter.

The evidence was also assessed for its applicability to the areas (populations, settings, interventions) covered by the scope of the guidance. Each evidence statement concludes with a statement of applicability (directly applicable, partially applicable, not applicable).

Summarising the evidence and making evidence statements

The review data was summarised in evidence tables (see full reviews).

The findings from the reviews were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by the external contractors/public health collaborating centres (see appendix A). The statements reflect their judgement of the strength (quality, quantity and consistency) of evidence and its applicability to the populations and settings in the scope.

Cost effectiveness

There was a review of economic evaluations and an economic modelling exercise.

Review of economic evaluations

Studies were identified by searching the NHS Economic Evaluation Database (NHSEED). An additional search was undertaken using an economics study filter.

The search focused on health economic studies that dealt with:

  • interventions to increase walking and/or cycling and reported relevant health-related outcomes

  • cost–benefit analysis results studies which considered wider outcomes, including travel, congestion and pollution.

Simplified search strategies were also used to search another economic specific database EconLit.

Economic modelling

An economic model was constructed to incorporate data from the reviews of effectiveness and cost effectiveness. The results are reported in: 'Walking and cycling: local measures to promote walking and cycling as forms of travel or recreation: health economic and modelling report'.

How the PDG formulated the recommendations

At its meetings in November 2011 and January, February and March 2012, the Programme Development Group (PDG) considered the evidence, expert reports and cost effectiveness to determine:

  • whether there was sufficient evidence (in terms of strength and applicability) to form a judgement

  • where relevant, whether (on balance) the evidence demonstrates that the intervention or programme/activity can be effective or is inconclusive

  • where relevant, the typical size of effect (where there is one)

  • whether the evidence is applicable to the target groups and context covered by the guidance.

The PDG developed draft recommendations through informal consensus, based on the following criteria:

  • Strength (type, quality, quantity and consistency) of the evidence.

  • The applicability of the evidence to the populations/settings referred to in the scope.

  • Effect size and potential impact on the target population's health.

  • Impact on inequalities in health between different groups of the population.

  • Equality and diversity legislation.

  • Ethical issues and social value judgements.

  • Cost effectiveness (for the NHS and other public sector organisations).

  • Balance of harms and benefits.

  • Ease of implementation and any anticipated changes in practice.

The PDG noted that effectiveness can vary according to the context. For instance, geographical factors such as population density in rural or urban areas influence the likelihood of walking or cycling being a viable option for utility travel.

Where evidence was lacking, the PDG also considered whether a recommendation should only be implemented as part of a research programme.

Where possible, recommendations were linked to evidence statements (see appendix C for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).