Guidance

Appendix B: Summary of the methods used to develop this guidance

Appendix B: Summary of the methods used to develop this guidance

Introduction

The report of the review and economic appraisal include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the PHIAC meetings provide further detail about the Committee's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E.

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and facilitated the development of recommendations by PHIAC. The overarching question was:

What are the most effective and cost effective ways for primary and residential care services to promote the mental wellbeing of older people?

The following subsidiary questions were considered:

1. What is the frequency and duration of an effective intervention?

2. What are the significant features of an effective intervener?

3. Are interventions that engage older people in their design and delivery more effective than those that do not?

4. Are interventions that engage immediate family members or carers more effective than those that do not?

5. Does the intervention lead to any adverse or unintended effects?

6. What are the barriers to and facilitators of effective implementation?

Reviewing the evidence of effectiveness

A review of effectiveness was conducted for interventions to promote mental wellbeing in people aged 65 and over.

Identifying the evidence

The following databases were searched for all study types for the period from January 1993 to February 2007:

  • Age Info

  • Ageline

  • AMED

  • ASSIA

  • British Nursing Index

  • CINAHL

  • Cochrane Central Register of Controlled Trials

  • Cochrane Database of Systematic Reviews (CDSR)

  • Current Controlled Trials

  • Database of Abstracts of Reviews of Effectiveness (DARE)

  • EmBase

  • HMIC

  • Medline

  • National Electronic Library for Health (NELH) – specifically the Specialist Libraries for Later Life and Mental Health

  • National Research Register

  • PsycInfo

  • Research Findings Register

  • SIGLE

  • Social Care Online

  • Social Science Citation Index

  • Sociological Abstracts

The following websites were searched for all study types for the period from January 1993 to February 2007:

Further details of the databases, search terms and strategies are included in the review report.

Selection criteria

Studies were included in the effectiveness reviews if:

  • they included older people, for example, studies of people aged 50–70, but only if results were subdivided by age groups

  • the target population was people aged 65 and older living at home, in the community, in supported housing or in residential care homes

  • they included interventions and activities that promote or sustain mental wellbeing in older people, provided by their carers, families, peers, practitioners, professionals or volunteers.

The wide range of interventions considered included:

  • self-care interventions (for example, health promotion, education, advice and information, exercise and physical activity and dietary advice)

  • psychological interventions (for example, cognitive training, relaxation techniques)

  • social interventions (for example, peer/social support, volunteering, group activity or participation, befriending, leisure activities)

  • environmental interventions (for example, housing adaptations, low-level support, technology, transport).

All study designs were included, and their limitations noted.

Interventions were included that aimed to promote, improve, enhance, sustain and benefit mental wellbeing and that included validated measures and self-reported indicators of outcomes such as: quality of life, autonomy, acceptance, purpose in life, control, affect, resilience, psychological wellbeing, competence, happiness, optimism, personal growth and self-esteem (further details are given in the full review).

Studies were excluded if:

  • they included older people undergoing treatment for a clinically diagnosed physical illness (for example, cancer) or mental illness (for example, dementia)

  • they made assessments for long-term continuing care

  • they included community interventions to improve the physical and social environment not targeted directly at people aged 65 and older, or their carers

  • they were tailored to people in acute or palliative care

  • they were medical or surgical interventions

  • they were related to pre-retirement financial planning schemes

  • they used specific therapeutic interventions (for example, reminiscence therapy) covered by NICE clinical guidelines.

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual 'Methods for development of NICE public health guidance' (see appendix E). Each study was described by study type and graded (++, +, –) to reflect the risk of potential bias arising from its design and execution.

Study type

  • Meta-analyses, systematic reviews of randomised controlled trials (RCTs) or RCTs (including cluster RCTs).

  • Systematic reviews of, or individual, non-randomised controlled trials, case-control studies, cohort studies, controlled before-and-after (CBA) studies, interrupted time series (ITS) studies, correlation studies.

  • Non-analytical studies (for example, case reports, case series).

  • Expert opinion, formal consensus.

Study quality

++ All or most criteria have been fulfilled. Where they have not been fulfilled the conclusions are thought very unlikely to alter.

+ Some criteria fulfilled. Those criteria that have not been fulfilled or not adequately described are thought unlikely to alter the conclusions.

– Few or no criteria fulfilled. The conclusions of the study are thought likely or very likely to alter.

The interventions were also assessed for their applicability to the UK and the evidence statements were graded as follows:

A. Likely to be applicable across a broad range of settings and populations

B. Likely to be applicable across a broad range of settings and populations, assuming they are appropriately adapted

C. Applicable only to settings or populations included in the studies – broader applicability is uncertain

D. Applicable only to settings or populations included in the studies.

Summarising the evidence and making evidence statements

The review data were summarised in evidence tables (see full reviews).

The findings from the review were synthesised and used as the basis for a number of evidence statements relating to the key question. The evidence statements reflect the strength (quantity, type and quality) of evidence and its applicability to the populations and settings in the scope.

Economic analysis

The economic appraisal consisted of a review of economic evaluations and a cost-effectiveness analysis.

Review of economic evaluations

The following databases were searched for the period from January 1993 to February 2007:

  • ECONLIT

  • HEED

  • NHS EED

The search strategies for these reviews were developed by NICE in collaboration with the Centre for Reviews and Dissemination at the University of York. Further detail can be found in the full reviews.

For the health economic and modelling review, studies were identified that included economic evaluation/analyses as well as health economics, cost benefit, cost containment, cost effectiveness, cost utility, cost allocation, socioeconomics, healthcare costs and healthcare finance.

For published studies that met the inclusion criteria the quality of the evidence was established using the Drummond checklist (Drummond MF, Jefferson TO [1996] Guidelines for authors and peer reviewers of economic submissions to the BMJ. BMJ 313: 275–83).

Cost-effectiveness analysis

Interventions identified in the effectiveness review that did not have supporting economic evidence were selected for inclusion in an economic model developed for the assessment of benefits (expressed in quality-adjusted life years; QALYs) relative to their respective costs. Algorithms were applied to the profile of scores covering physical and emotional health used in the identified studies, often measured by means of the SF-36 or SF-12 questionnaires, to derive SF-6D health state utility indices to enable the calculation of cost utility estimates. The results are reported in Public health interventions to promote mental well-being in people aged 65 and older: systematic review of effectiveness and cost-effectiveness.

Fieldwork

Fieldwork was carried out to evaluate the relevance and usefulness of NICE guidance and the feasibility of implementation. Practitioners and commissioners who are involved in health and social care services for older people were involved. They included those working in primary care, public health and health promotion, occupational therapy and community pharmacy in the NHS, leisure services, residential and domiciliary care services and the voluntary sector services for older people and their carers. Fieldwork also included a group discussion with older people and their carers.

The fieldwork included:

  • Group discussions, paired depth interviews and individual depth interviews were conducted at Wakefield, Derbyshire, Leeds, Cheshire, Staffordshire, Barking and Dagenham and Redbridge in February and March 2008 by Dr Foster Intelligence. Those who took part included:

    • PCT and local authority commissioners and directors of services for older people

    • social services managers and staff

    • public health advisers and health promotion specialists

    • occupational therapists working with older people

    • GPs

    • practice nurses

    • community pharmacists

    • residential care managers, activity coordinators and staff

    • domiciliary care managers and staff

    • voluntary sector workers who provide services for or represent the views of older people.

  • A group discussion with older people in residential care and their carers was carried out in Staffordshire by Dr Foster Intelligence.

The main issues arising from these sessions are set out in appendix C under fieldwork findings. The full fieldwork report is 'Occupational therapy and physical activity interventions to promote the mental wellbeing of older people in primary care and residential care: fieldwork report'.

How PHIAC formulated the recommendations

At its meetings in September 2007, November 2007, April 2008 and June 2008, PHIAC considered the evidence of effectiveness and cost effectiveness of interventions to promote the mental wellbeing of older people to determine:

  • whether there was sufficient evidence (in terms of quantity, quality and applicability) to form a judgement

  • whether, on balance, the evidence demonstrates that the intervention is effective or ineffective, or whether it is equivocal

  • where there is an effect, the typical size of the effect.

PHIAC developed draft recommendations through informal consensus, based on the following criteria.

  • Strength (quality and quantity) of evidence of effectiveness and its applicability to the populations/settings referred to in the scope.

  • Effect size and potential impact on population health and/or reducing inequalities in health.

  • Cost effectiveness (for the NHS and other public sector organisations).

  • Balance of risks and benefits.

  • Ease of implementation and the anticipated extent of change in practice that would be required.

Where possible, recommendations were linked to an evidence statement(s) (see appendix C for details). Where a recommendation was inferred from the evidence this was indicated by the reference 'IDE' (inference derived from the evidence).

The draft guidance, including the recommendations, was released for consultation in February 2008. At its meetings in April 2008 and June 2008 PHIAC considered comments from stakeholders and the results from fieldwork and amended the guidance. The guidance was signed off by the NICE Guidance Executive in September 2008.