Guidance

Appendix B: Summary of the methods used to develop this guidance

Appendix B: Summary of the methods used to develop this guidance

Introduction

The reviews and economic analysis include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the PDG meetings provide further detail about the Group's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E.

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and were used by the PDG to help develop the recommendations. The primary questions were:

Question 1: What type of price controls are effective and cost effective in reducing alcohol consumption, alcohol misuse, alcohol-related harm or alcohol-related social problems among adults and young people?

Question 2: Which interventions are effective and cost effective at managing alcohol availability to reduce levels of consumption, alcohol misuse, alcohol-related harm or alcohol-related social problems among adults and young people?

Question 3: Is the control of alcohol promotion (for example, advertising) effective and cost effective in reducing levels of consumption, alcohol misuse, alcohol-related harm or alcohol-related social problems among adults and young people?

Question 4: What are the key factors that increase the risk of an individual misusing alcohol? When are individuals most vulnerable to alcohol misuse?

Question 5: Are alcohol screening questionnaires, biochemical markers or clinical indicators (for example, hypertension, dilated facial capillaries) an effective and cost effective way of identifying adults and young people who currently misuse – or are at risk of misusing – alcohol?

Question 6: Are brief interventions effective and cost effective in managing hazardous and harmful drinking among adults and young people?

Question 7: What are the key barriers to helping adults and young people manage their drinking behaviour (for example, is access to services a problem)? What are the key facilitators?

These questions were made more specific for each review (see reviews for further details).

Reviewing the evidence of effectiveness

Two reviews of effectiveness were conducted.

Identifying the evidence

Relevant literature was identified using an iterative search process. Study types and years were not predefined. The following databases were searched.

  • ASSIA (Applied Social Science Index and Abstracts)

  • Cochrane Library (Cochrane database of systematic reviews, Database of abstracts of reviews of effects, Health technology assessment and Cochrane-controlled trials register)

  • EconLit

  • MEDLINE (including MEDLINE in process)

  • NHS Economic Evaluation Database (NHS EED)

  • Social Science Citation Index

Additional searches (non-systematic) were carried out on the following websites:

Selection criteria

Studies were included in the effectiveness reviews if:

  • people of a range of ages were involved

  • interventions were relevant to the key questions set out in the reviews

  • outcomes such as alcohol consumption, alcohol misuse, alcohol-related harm, social problems, costs and economic impact were reported.

Studies were excluded if:

  • they were not published in English

  • the study population was below the age of 10 years

  • the evidence did not originate in economically developed countries (that is, if it did not come from countries that are members of the Organisation for Economic Cooperation and Development [OECD]).

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual 'Methods for the development of NICE public health guidance' (see appendix E). Each study was graded (++, +, –) to reflect the risk of potential bias arising from its design and execution.

Study quality

++ All or most of the methodology checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are thought very unlikely to alter.

+ Some of the methodology checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are thought unlikely to alter the conclusions.

– Few or no methodology checklist criteria have been fulfilled. The conclusions of the study are thought likely or very likely to alter.

Summarising the evidence and making evidence statements

The review data was summarised in evidence tables (see full reviews).

The findings from the reviews were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by the public health collaborating centre (see appendix A). The statements reflect the collaborating centre's judgement of the strength (quantity, type and quality) of evidence and its applicability to the populations and settings in the scope.

Economic analysis

The economic analysis consisted of 2 cost effectiveness reviews and an economic modelling report .

Review of economic evaluations

The following databases were searched for economic literature, in addition to the searches carried out for the effectiveness reviews:

  • EconLIT

  • NHS Economic Evaluation Database (NHS EED).

Studies were included if:

  • they addressed key questions 1, 2, 3, 5 and 6

  • they were from peer-reviewed journals published in English

  • the study population involved a range of ages (10+ years)

  • they were carried out in OECD countries.

Economic modelling report

A number of assumptions were made which could underestimate or overestimate the cost effectiveness of the interventions (see review modelling report for further details).

An economic model was constructed to incorporate data from the reviews of effectiveness and cost effectiveness. The results are reported in the economic modelling report – Modelling to assess the effectiveness and cost-effectiveness of public health-related strategies and interventions to reduce alcohol attributable harm in England using the Sheffield alcohol policy model version 2.0.

Fieldwork

Fieldwork was carried out to evaluate how relevant and useful NICE's recommendations are and how feasible it would be to put them into practice. It was conducted with commissioners, practitioners and other interested parties who are involved in alcohol services in the NHS, local authorities and the private, voluntary and community sectors. They included: policy makers, applied researchers, economists, trading standards, representatives of licensing boards, retailers and the alcohol industry, and representatives from criminal justice and social welfare.

The fieldwork comprised:

  • five meetings in Birmingham, Bristol, Leicester, Liverpool and London conducted by Liverpool John Moores University with policy makers, commissioners, industry representatives and practitioners

  • an online survey of professionals (14) who could not attend the fieldwork meetings.

The fieldwork meetings and online survey were commissioned to ensure there was ample geographical coverage. The main issues arising are set out in appendix C under fieldwork findings. Also see the fieldwork report – Alcohol-use disorders: preventing the development of hazardous or harmful drinking.

How the PDG formulated the recommendations

At its meeting in July 2009, the PDG considered the evidence of effectiveness and cost effectiveness to determine:

  • whether there was sufficient evidence (in terms of quantity, quality and applicability) to form a judgement

  • whether, on balance, the evidence demonstrates that the intervention is effective, ineffective or equivocal

  • where there is an effect, the typical size of effect.

The PDG developed draft recommendations through informal consensus, based on the following criteria:

  • Strength (quality and quantity) of evidence of effectiveness and its applicability to the populations/settings referred to in the scope.

  • Effect size and potential impact on the target population's health.

  • Impact on inequalities in health between different groups of the population.

  • Cost effectiveness (for the NHS and other public sector organisations).

  • Balance of risks and benefits.

  • Ease of implementation and any anticipated changes in practice.

The PDG noted that effectiveness can vary according to the context. For example, it depends on the enforcement of different regulatory regimes.

Where possible, recommendations were linked to an evidence statement(s) (see appendix C for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

The draft guidance, including the recommendations, was released for consultation in September 2009. At its meeting in December 2009, the PDG amended the guidance in light of comments from stakeholders and experts and the fieldwork. The guidance was signed off by the NICE Guidance Executive in March 2009.