NICE process and methods

Glossary

Glossary

Absolute risk reduction (or increase)

The arithmetic difference between the event rates in 2 groups in a clinical study.

Abstract

A summary of a study, which may be published alone or as an introduction to a full scientific paper.

Adherence

The extent to which a person follows the health advice agreed with healthcare professionals. It may also be referred to as 'compliance'.

Adverse effect

A consequence other than that which was intended. Adverse effects relate specifically to drugs or other treatments or interventions that a person is receiving – they are a toxic reaction.

Appraisal Committee

A standing advisory committee of the Institute. Includes people who work in the NHS, people representing patient and carer organisations, lay members, people from relevant academic disciplines and the pharmaceutical and medical device industries.

Assessment Group

An independent assessment group commissioned by the NHS Research and Development Health Technology Assessment (HTA) programme to produce an independent review of the evidence for technologies being appraised within the multiple technology appraisal (MTA) process.

Assessment report

A critical review of the clinical and cost effectiveness of a health technology or technologies being appraised within the multiple technology appraisal (MTA) process. It is prepared by the Assessment Group. To prepare the report, the Assessment Group carries out a review of the published literature and the submissions from manufacturers and sponsors.

Baseline

Used to describe the initial set of measurements taken at the beginning of a study (after a run-in period, when applicable).

Bias

Systematic (as opposed to random) deviation of the results of a study from the 'true' results.

Blinding

When patients, caregivers, researchers and outcome assessors are kept unaware of the interventions patients have received in a study.

Carer

In this guide the term 'carer' refers to a person who provides unpaid care by looking after a relative, friend or partner who needs support because of ill health, frailty or disability.

Case–control study

A comparative observational study in which the investigator compares people who have experienced an event (for example, developed a disease) with people who have not (controls), and collects data to determine possible causes of the event.

CE mark

The abbreviation of 'Conformité Européene'. This mark indicates that the manufacturer has conformed with all the obligations required by European law applying to health, safety and environmental protection legislation. The CE mark allows a manufacturer to sell their products within the European market.

Citizens Council

A group of 30 people drawn from all walks of life who bring the public's views to NICE decision-making. The Citizens Council tackles challenging questions about values, such as fairness and need.

Class (of drugs in a NICE technology appraisal)

A group of drugs with the same or similar mechanisms of action. These drugs may or may not have the same basic chemical structure. However, there may be differences between drugs within a class, such as the side effects associated with them.

Clinical audit

A quality improvement process that measures patient care and outcomes through a structured or detailed review of care against explicit criteria, and takes action to improve it if necessary.

Clinical effectiveness

The extent to which an intervention produces an overall health benefit, taking into account beneficial and adverse effects, in routine clinical practice. It is not the same as efficacy.

Clinical specialist

In technology appraisals, clinical specialists act as expert witnesses to the Appraisal Committee. They are selected on the basis of specialist expertise and personal knowledge of the technology and/or other treatments for the condition. They provide a view of the technology within current clinical practice, and insights not typically available in the published literature.

Cohort study

A retrospective or prospective follow-up study. People in the study are grouped on the basis of whether or not they have been exposed to a suspected risk factor or intervention. A cohort study can be comparative, but the study investigator has no control over who is or isn't exposed.

Commentator

An organisation that engages in the appraisal process but is not asked to prepare a submission dossier. Commentators are invited to comment on the draft scope document, the assessment report and the appraisal consultation document (ACD). They receive the final appraisal determination (FAD) for information only, and do not have the right of appeal. These organisations are manufacturers of comparator technologies, Healthcare Improvement Scotland, the relevant National Collaborating Centre, related research groups, and other groups when appropriate.

Comorbidity

A disease or condition a patient has in addition to the disease being studied or treated.

Comparator

The standard intervention against which the intervention under appraisal is compared. The comparator can be no intervention, for example best supportive care.

Confidence interval (CI)

A range of values for an unknown population parameter (for example, blood pressure) with a stated 'confidence' (conventionally 95%) that it contains the true value. The range is calculated from sample data, and generally includes the sample estimate. The 'confidence' value means that if the method used to calculate the interval is repeated many times, then that proportion of ranges will actually contain the true value.

Confounding

In a study, confounding occurs when the effect of an intervention on an outcome is distorted because of an association between the population or intervention or outcome and another factor (the 'confounding variable') that can influence the outcome independently of the intervention under study.

Constant proportional trade-off

The proportion of remaining life that a person would trade off for a given quality improvement is independent of the amount of remaining life.

Construct validity

The extent to which a measure correlates with other measures or 'constructs' in a manner consistent with theory (for example, the extent to which a generic measure of quality of life correlates with other established measures of disease severity).

Consultation

The process that allows stakeholders and individuals to comment on initial versions of NICE guidance and other documents (for example, the draft scope) so that their views can be taken into account when the final version is being produced.

Consultee

An organisation that participates in the appraisal of a technology. Consultees can comment on the draft scope, the assessment report or Evidence Review Group report, and the appraisal consultation document (ACD) during the consultation process. Consultee organisations can nominate clinical specialists, commissioning experts and patient experts to present their personal views to the Appraisal Committee. All consultees are given the opportunity to appeal against the final appraisal determination (FAD).

Control

An explicitly defined comparator against which the effects of an intervention are compared in a clinical study.

Cost–benefit analysis

An economic evaluation that expresses both costs and outcomes of an intervention in monetary terms. Benefits are valued in monetary terms using valuations of people's observed or stated preferences, such as the willingness-to-pay approach.

Cost-effectiveness acceptability curves

A graph that plots a range of possible maximum acceptable ICERs on the horizontal axis against the probability (chance) that the intervention will be cost effective at that ICER on the vertical axis. In technology appraisals, cost-effectiveness acceptability curves are a means of representing the uncertainty surrounding the cost-effectiveness estimates in relation to the decision.

Cost-effectiveness analysis

An economic study design in which consequences of different interventions are measured using a single outcome, usually in 'natural' units (for example, life-years gained, deaths avoided, heart attacks avoided, or cases detected). Alternative interventions are then compared in terms of cost per unit of effectiveness.

Cost-effectiveness frontier

A region on a plot that shows the probability that the technology with the highest expected net benefit is cost effective.

Cost-effectiveness model

An explicit mathematical framework which is used to represent clinical decision problems and incorporate evidence from a variety of sources to estimate costs and health outcomes.

Cost-effectiveness plane

A graphical illustration of cost effectiveness. The horizontal axis represents the difference in effect between the intervention and the comparator. The vertical axis represents the difference in cost.

Decision problem

A clear description of the interventions, patient populations, outcome measures and perspective adopted in a health technology evaluation, relating specifically to the decision(s) that the evaluation is designed to inform.

Director of the Centre for Health Technology Evaluation

The Director of the Centre for Health Technology Evaluation is responsible for the delivery of the technology appraisal programme. The Director is also responsible for ensuring that appraisals are conducted in accordance with the published appraisal process and methodology.

Discounting

Costs and benefits incurred today are usually valued more highly than costs and benefits occurring in the future. Discounting health benefits reflects society's preference for benefits to be experienced in the present rather than the future. Discounting costs reflects society's preference for costs to be experienced in the future rather than the present.

Dominance

An intervention is dominated if it has higher costs and worse outcomes than an alternative intervention.

Effectiveness

See 'Clinical effectiveness'.

Efficacy

The extent to which an intervention is effective when studied under controlled research conditions.

End point

In a research study, an event or outcome that can be measured and constitutes 1 of the target outcomes of the trial.

Epidemiological study

The study of a disease within a population, which includes defining its incidence and prevalence and examining the roles of external influences (for example, infection or diet) and interventions on the disease.

Equity

Fair distribution of resources or benefits.

European Medicines Agency

A decentralised agency of the European Union responsible for the scientific evaluation of medicines developed by pharmaceutical companies for use in the European Union.

Evidence

Information on which a decision or guidance is based. Evidence is obtained from a range of sources, including randomised controlled trials, observational studies and expert opinion (of clinical professionals and/or patients/carers).

Evidence Review Group

An independent assessment group commissioned by the NHS Research and Development Health Technology Assessment (HTA) programme to produce an independent assessment of the evidence submitted by the manufacturer or sponsor of a technology being appraised within the single technology appraisal (STA) process.

Evidence Review Group report

A critical assessment of the evidence submitted by the manufacturer of a technology being appraised within the single technology appraisal (STA) process. It is prepared by the Evidence Review Group.

Extended dominance

The incremental cost-effectiveness ratio (ICER) for a given treatment alternative is higher than that of the next, more effective, alternative (that is, it is dominated by the combination of 2 alternatives and should not be used to calculate appropriate ICERs).

External validity

The degree to which the results of an observation, study or review are likely to hold true in a population or clinical practice setting outside of the study population/setting. See also 'Internal validity'.

Extrapolation

In data analysis, predicting the value of a parameter outside the range of observed values.

Forest plot

A common way of presenting the results of a systematic review and meta-analysis. The estimates of treatment effects, along with their confidence intervals, are plotted relative to a vertical line indicating no difference between the intervention and control in the included study. From this plot, an impression of the distribution of the estimates of effect in all included studies can be gained.

Generalisability

The extent to which the results of a study conducted in a particular patient population and/or a specific context will apply for another population and/or in a different context.

Health-related quality of life

A combination of a person's physical, mental and social wellbeing.

Health technology

Any method used by those working in health services to promote health, prevent and treat disease, and improve rehabilitation and long-term care. Technologies in this context are not confined to new drugs or medical technologies.

Healthcare Resource Groups (HRGs)

These groups provide a way of categorising the treatment of patients so that the use of resources can be monitored and evaluated. Each HRG refers to a group of health-related activities or services that have been judged to consume a similar level of resources.

Heterogeneity

Used in meta-analyses and systematic reviews to describe when the results or estimates of effects of a treatment from separate studies seem to be very different (for example, the size of treatment effects may vary across studies, or some studies may indicate beneficial treatment effects whereas others suggest adverse treatment effects). Such difference in results may occur by chance, because of variation in study quality or because of variation in populations, interventions, or methods of outcome measurement in the included studies.

Inclusion criteria (literature review)

Explicit criteria used to decide which studies should be considered as potential sources of evidence.

Incremental cost-effectiveness ratio (ICER)

The ratio of the difference in the mean costs of a technology compared with the next best alternative to the differences in the mean outcomes.

Indication (specific)

The use of a technology as licensed by the Medicines and Healthcare products Regulatory Agency (MHRA).

Indirect comparison

An analysis comparing interventions that have not been compared directly within a head-to-head randomised trial.

Intention-to-treat (ITT) analysis

An analysis of the results of a randomised controlled trial in which the data are analysed for all study participants as if they had remained in the group to which they were randomised, regardless of whether or not they remained in the study until the end, crossed over to another treatment or received an alternative intervention.

Intermediate outcome

Outcomes that are related to the outcome of interest but may be more easily assessed within a clinical study (for example, blood pressure reduction is related to the risk of a stroke).

Internal validity

The degree to which the results of a study are likely to approximate the 'truth' for the participants recruited in a study. It refers to the integrity of the study design and is a prerequisite for applicability (external validity) of a study's findings. See also 'External validity'.

Life-years gained

Average years of life gained per person as a result of the intervention.

Marketing authorisation

An authorisation from the Medicines and Healthcare products Regulatory Agency (MHRA) or European Commission to market a medicinal product.

Medicines and Healthcare products Regulatory Agency (MHRA)

The Executive Agency of the Department of Health that protects and promotes public health and patient safety by ensuring that medicines, healthcare products and medical equipment meet appropriate standards of safety, quality, performance and effectiveness, and are used safely.

Meta-analysis

A statistical technique for combining (pooling) the results of a number of studies that address the same question and report on the same outcomes to produce a more precise summary estimate of the effect on a particular outcome.

Mixed treatment comparison

An analysis that compares 2 or more interventions using a combination of direct evidence (from head-to-head trials of the interventions of interest) and indirect evidence (trials that do not compare the interventions of interest directly).

Multiple technology appraisal (MTA)

The name given to the NICE process in which appraisals of more than 1 technology, or a single technology for a broad set of indications, are conducted.

National Institute for Health Research (NIHR)

The NIHR commissions and funds NHS, social care and public health research, and makes this research available to support decision-making by professionals, policy makers and patients.

Natural history of a disease

The progression of a disease when untreated.

Net benefit

The net benefit can be expressed in health (for example, using quality-adjusted life years [QALYs]) or monetary terms. Net health benefit is the difference between the total expected QALYs and the health expected to be forgone elsewhere (the total expected costs divided by the maximum acceptable incremental cost-effectiveness ratio [ICER] value). The net monetary benefit is the difference between the monetary value of total expected QALYs (expected QALYs multiplied by the maximum acceptable ICER value) and total expected costs.

NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC)

One of the activities of NETSCC is the management of the NIHR health technology assessment (HTA) programme. This programme produces independent research information about the effectiveness, costs and broader impact of healthcare treatments and tests for those who plan, provide or receive care in the NHS. Technology assessment reports are commissioned by the HTA programme on behalf of NICE to inform its national clinical guidance to the NHS.

Non-reference-case analysis

An analysis that does not use methods specified in the reference case considered by the Institute to be the most appropriate for the Appraisal Committee's purpose.

Observational study

A retrospective or prospective study in which the investigator observes the natural course of events with or without control groups (for example, cohort and case–control studies).

Opportunity cost

The opportunity cost of investing in a healthcare intervention is the other healthcare programmes that are displaced by its introduction. This may be best measured by the health benefits that could have been achieved had the money been spent on the next best alternative healthcare intervention.

Outcome

The measure of the possible results of treatment with a preventive or therapeutic intervention. Outcome measures can be either intermediate or final end points. See also 'Intermediate outcome'.

Pairwise comparisons

Comparisons that compare each of the technologies of interest in a series of separate analyses. For example, if there are 3 treatments (A, B and C) being compared, they could be compared in a single combined analysis (that is, A compared with B compared with C) or as a series of pairwise comparisons (that is A compared with B, A compared with C, and B compared with C).

Parameter

A measurable or quantifiable characteristic. For example, the relative treatment effect of a technology may be a parameter in an economic model.

Parameter uncertainty

Uncertainty about the mean values of parameters (for example, health outcomes, utilities and resource use) included in the model.

Patient expert

Acts as an expert witness to the Appraisal Committee. Patient experts have used the technology either personally or as part of a representative group. They provide a view on the risks and benefits of the technology from personal experience as a patient or carer, and an understanding of the wider range of patient and/or carer views.

Perspective (in economic evaluation)

The viewpoint from which an economic evaluation is conducted. The viewpoint may be that of the patient, hospital/clinic, healthcare system or society.

Primary research

A study generating original data rather than analysing data from existing studies (which is called secondary research).

Quality-adjusted life year (QALY)

An index of survival that is adjusted to account for the patient's quality of life during this time. QALYs incorporate changes in both quantity (longevity/mortality) and quality (morbidity, psychological, functional, social, and other factors) of life. Used to measure benefits in cost–utility analysis.

Quality of life

See 'Health-related quality of life'.

Random effects model

In meta-analysis, a model allowing for heterogeneity between studies. The simplest models allow for a single random effect term, and more complicated models can allow for different levels of heterogeneity.

Randomisation

Allocation of participants in a research study to 2 or more alternative groups using a chance procedure such as computer-generated random numbers. This approach is used to attempt to ensure there is an even distribution of participants with different characteristics between groups and reduces bias and confounding.

Randomised controlled trial (RCT)

A comparative study in which people are randomly allocated to intervention and control groups and followed up to examine differences in outcomes between the groups.

Reference case

When estimating clinical and cost effectiveness, the reference case specifies the methods considered by NICE to be the most appropriate for the Appraisal Committee's purpose and consistent with an NHS objective of maximising health gain from limited resources.

Relative risk (RR)

The number of times more likely or less likely an event is to happen in 1 group compared with another (calculated as the risk of the event in group A divided by the risk of the event in group B). The RR is usually expressed as the risk of the event in the intervention group divided by the risk of the event in the comparator group. In this case, an RR of less than 1 indicates that there is less risk of the event with the intervention than the comparator.

Relative treatment effect

The effect of a treatment relative to another treatment or control, for example measured by relative risk (RR).

Remit

The brief given to the Institute by the Department of Health and Welsh Assembly Government when a technology is referred to NICE for appraisal.

Sensitivity analysis

A way of representing uncertainty in the results of economic evaluations. Uncertainty may arise from missing data, imprecise estimates or methodological controversy. Sensitivity analysis also allows for exploring the generalisability of results to other settings. The analysis is repeated using different assumptions to examine the effect on the results.

One-way simple sensitivity analysis (univariate analysis)

Each parameter is varied individually to isolate the consequences of the parameter on the results of the study.

Multi-way simple sensitivity analysis (scenario analysis)

Two or more parameters are varied at the same time and the overall effect on the results is evaluated.

Threshold sensitivity analysis

The critical value of parameters above or below which the conclusions of the study will change are identified.

Probabilistic sensitivity analysis

Probability distributions are assigned to the uncertain parameters and are incorporated into evaluation models based on decision analytical techniques (for example, Monte Carlo simulation).

Single technology appraisal (STA)

The name given to the NICE process in which appraisals of single technologies for 1 indication are conducted.

Structural uncertainty

Uncertainty relating to the range of assumptions and judgements necessary in constructing a model. This can include design features of the model (for example, the assumed standard pathway of care) as well as judgements about the relevance of evidence, assumptions about appropriate distributions for parameters and alternative methods of estimation.

Synthesis of evidence

A generic term to describe methods used for summarising (comparing and contrasting) evidence into a clinically meaningful conclusion to answer a defined clinical question. This can include systematic review (with or without meta-analysis), and qualitative and narrative summaries.

Systematic review

Research that summarises the evidence on a clearly formulated question according to a predefined protocol. Systematic and explicit methods to identify, select and appraise relevant studies, and to extract, collate and report their findings are used. Statistical meta-analysis may or may not be used.

Technology

See 'Health technology'.

Technology assessment

The process of evaluating the clinical, economic and other evidence on the use of a technology to formulate guidance on its most efficient use.

Time horizon

The time span used in the NICE appraisal that reflects the period over which the main differences in health effects and use of healthcare resources between interventions are expected to be experienced.

Time trade-off

A method used to measure utility (for example, health states). The utility value is measured by finding the point at which the respondent cannot choose between 2 scenarios. For chronic illness, the choice is between the illness for a period of time and perfect health for a shorter time, both followed by death. For short-term illness, the choice is between the illness for a period of time and a worse health state for a shorter time, both followed by the same specified outcome.

Treatment options

The choice of interventions that are available for a specific condition.

Treatment sequence

Used to describe when the intervention being evaluated and the comparator are used in succession in the management of a condition.

Utility

A measure of the strength of a person's preference for a specific health state in relation to alternative health states. The utility scale assigns numerical values on a scale from 0 (death) to 1 (optimal or 'perfect' health). Health states can be considered worse than death and thus have a negative value.

Variable

A measurement that can vary within a study (for example, the age of participants). Variability is present when differences can be seen between different people or within the same person over time, with respect to any characteristic or feature that can be assessed or measured.

ISBN: 978-1-4731-0119-7