4 Evidence and interpretation
The Appraisal Committee (appendix A) considered evidence from a number of sources (appendix B).
4.1 Clinical effectiveness
4.1.1 The assessment report included 48 randomised controlled trials (RCTs) that compared the efficacy of CPAP with placebo or usual care or dental devices. Submissions were obtained from three manufacturers. Fisher & Paykel Healthcare included one RCT that compared CPAP with placebo. This study was excluded by the Assessment Group on the grounds of study quality. Respironics UK did not carry out a review of clinical effectiveness, and ResMed (UK) presented the findings of a Cochrane systematic review carried out in 2006.
4.1.2 The Assessment Group identified 23 RCTS that compared CPAP with placebo or usual care using the Epworth Sleepiness Scale (ESS). A meta-analysis of these studies identified a statistically significantly greater reduction in daytime sleepiness with CPAP compared with placebo or usual care (weighted mean difference in ESS score −2.7; 95% confidence interval [CI] −3.5 to −2.0).
4.1.3 The Assessment Group undertook a series of meta-analyses that compared the effect of CPAP on levels of daytime sleepiness in different populations. This showed a statistically significantly greater reduction in daytime sleepiness with CPAP compared with placebo for moderate and severe categories of OSAHS. For mild OSAHS (meta-analysis of 3 studies; AHI=5 to 14 episodes per hour) a weighted mean difference in ESS score of −1.5 (95% CI −3.4 to 0.4) was found. For moderate OSAHS (meta-analysis of 7 studies; AHI=15 to 30 episodes per hour) a weighted mean difference in ESS score of −2.0 (95% CI −3.0 to −1.1) was found. For severe OSAHS (meta-analysis of 13 studies; AHI=over 30 episodes per hour) a weighted mean difference in ESS score of −3.4 (95% CI −4.6 to −2.3) was found.
4.1.4 The Cochrane systematic review submitted by ResMed (UK) included 36 RCTs. A meta-analysis of the parallel-group studies found a statistically significantly greater reduction in daytime sleepiness with CPAP compared with placebo or usual care (weighted mean difference in ESS score −3.8; 95% CI −4.6 to −3.1). A meta-analysis of the crossover studies showed a statistically significantly greater reduction in daytime sleepiness with CPAP compared with placebo or usual care (weighted mean difference in ESS score −1.9; 95% CI −2.6 to −1.3). The RCT included by Fisher & Paykel Healthcare compared CPAP with placebo, and reported improvements in sleepiness (measured by the ESS, a subjective measure) and in wakefulness (measured by the 'modified maintenance of wakefulness' test, an objective measure) with CPAP.
4.1.5 The Assessment Group identified six RCTs that compared the effects on daytime sleepiness (ESS score) of CPAP and dental devices. A meta-analysis of these studies did not identify a statistically significant difference between the two treatments (weighted mean difference in ESS score −0.9; 95% CI −2.1 to 0.4). The Assessment Group also found that removing individual trials from the analysis, use of a fixed-effect model or conducting subgroup analysis by study design (parallel group or crossover) did not reveal any statistically significant differences in daytime sleepiness between CPAP and dental devices.
4.1.6 A meta-analysis from the Cochrane systematic review identified statistically significant improvements in the AHI and in minimum oxygen saturation during sleep in favour of CPAP compared with dental devices.
4.1.7 The Assessment Group identified six RCTs that measured daytime mean arterial blood pressure. A meta-analysis of these RCTs found that CPAP was associated with a reduction in arterial blood pressure compared with placebo or usual care (mean difference −2.1 mmHg; 95% CI −4.3 to 0.0 mmHg). However, when these RCTs were analysed by severity of OSAHS classified by baseline ESS score, a statistically significant treatment effect in favour of CPAP was identified only for severe OSAHS (mean difference −4.2 mmHg; 95% CI −6.4 to −2.0 mmHg), but not for mild (mean difference 1.1 mmHg; 95% CI −2.9 to 5.1 mmHg) or moderate (mean difference −3.4 mmHg; 95% CI −7.9 to 1.2 mmHg) OSAHS.
4.1.8 The Assessment Group identified six RCTs that evaluated the effect of CPAP treatment on driving ability, using the Steerclear computerised driving simulator program. A meta-analysis of the four studies that reported the number of obstacles hit did not identify a statistically significant difference between CPAP treatment and placebo (weighted mean difference −5.74; 95% CI −14.75 to 3.27). A meta-analysis of the two RCTs that reported percentage of obstacles hit did not show a statistically significant difference in favour of CPAP (weighted mean difference 0.00; 95% CI −3.35 to 3.35).
4.1.9 The Assessment Group identified one trial that used a computerised simulated driving program that assessed night-time driving ability. CPAP resulted in statistically significant improvements in standard deviation of position on the road (p = 0.03), minutes spent driving accurately (p = 0.02) and deterioration in driving ability (p = 0.007) compared with placebo.
4.1.10 The Assessment Group identified two studies that estimated the impact of CPAP on the rate of road traffic accidents. The first study (a meta-analysis of eight before-and-after studies) estimated that use of CPAP reduced the odds of a road traffic accident by 85% (odds ratio 0.15). The second study showed reduced odds of a road traffic accident with CPAP treatment (odds ratio 0.33). A meta-analysis of these two studies identified an 83% reduction in road traffic accidents (odds ratio 0.17). The ResMed (UK) submission reported the findings of a 3-year case–control study that estimated the annual number of road traffic accidents among 640 people with OSAHS as 0.18 per year before and as 0.06 per year after initiation of CPAP therapy.
4.1.11 The Assessment Group found that there were not enough studies for meta-analyses to be performed for most of the quality-of-life outcome measures reported. Six studies were identified that used the SF-36 instrument for measuring health-related quality of life. Meta-analyses of these studies did not find a statistically significant difference between CPAP and placebo or usual care for any of the subscales, but there was a trend towards an improvement in favour of CPAP in the subscales for vitality and physical role. Individual studies showed statistically significant improvements in the subscales for bodily pain, emotional role, mental health, general health, physical function, social function and vitality. Four studies were identified that used the Functional Outcomes of Sleep Questionnaire. A statistically significant benefit in favour of CPAP was identified for the activity level and social outcome subscales, but not for the general productivity, intimacy and sexual activity or vigilance subscales, or total scores. Only two studies reported the Sleep Apnoea Quality of Life Index total score: one reported a significant benefit with CPAP compared with placebo, but in the other study there was no statistically significant difference.
4.1.12 There was no statistically significant difference between CPAP and dental devices when the results from two studies reporting scores on the Functional Outcomes of Sleep Questionnaire and two studies reporting the Sleep Apnoea Quality of Life Index were pooled. Three studies used the SF-36 instrument to compare CPAP with dental devices, but the findings were not consistent. One of these studies reported a benefit with CPAP compared with dental devices in both the physical and mental component summary scores, one study reported no difference in the total SF-36 score, and the remaining study identified a statistically significant benefit in favour of CPAP only for the bodily pain subscale.
4.1.13 The ResMed (UK) submission identified seven studies that reported short- and longer-term rates of adherence with CPAP therapy. The mean rate of adherence across the studies was 71% (range 64–83%) up to 12 months and 79% (range 68–90%) at 12 months or more. The Assessment Group reported on one of these studies, which was an observational study carried out over 6 years in a Scottish cohort. The adherence rates were 84% at 1 year, 74% at 2 years, 73% at 3 years and 68% after 4 years.
4.1.14 Statements from the patient experts and clinical specialists asserted that there are enough trained medical staff in secondary care to investigate the expected numbers of patients with likely OSAHS, but further training of support staff for CPAP provision will be necessary over the coming years. There are currently insufficient resources and training in primary care for the continued supervision of patients being treated with CPAP for OSAHS.
4.2 Cost effectiveness
4.2.1 Four published economic evaluations were identified by the Assessment Group, all of which compared CPAP with a 'do nothing' alternative. The resulting incremental cost-effectiveness ratios (ICERs) were: (1) US $3354 (approximately £1688; currency conversions were calculated in August 2007) per quality-adjusted life year (QALY) gained from a third-party payer perspective and US $314 (£158) per QALY gained from a societal perspective; (2) €7861 (£5348) per QALY gained over a 5-year time horizon and €4938 (£3359) per QALY gained for a lifetime time horizon; (3) £8300 per QALY gained at 1 year and £5200 per QALY gained at 2 years; (4) Can $9809 (£4654) per QALY gained for the high‑cost estimate and Can $3523 (£1672) per QALY gained for the low-cost estimate.
4.2.2 Fisher & Paykel Healthcare and Respironics UK did not submit its own cost-effectiveness analyses. The Assessment Group therefore evaluated only the economic model submitted by ResMed (UK).
4.2.3 ResMed (UK) submitted an economic model comparing fixed and auto-titrating CPAP devices with a 'do nothing' alternative. The model included people with severe OSAHS with the following health states: event free, cardiovascular event, stroke and road traffic accident. People remained in one of the four health states for 1 year before moving to another state. People who had a cardiovascular event or road traffic accident in 1 year could have a stroke, cardiovascular event or road traffic accident in a later year. People who had experienced a stroke were considered unable to drive and therefore could not experience a subsequent road traffic accident, but they could experience a subsequent stroke or cardiovascular event. There was no limit to the total number of events each person could undergo in subsequent years. No complications or symptoms were included, and the model had a 14-year time horizon and was from a UK NHS perspective. Utility estimates were obtained from a published study reporting EQ-5D data. The results of the ResMed (UK) model showed that both fixed and auto-titrating CPAP devices dominated 'non-treatment' after a minimum of 2 years of treatment (that is, CPAP was associated with more QALYs and lower costs than 'non-treatment').
4.2.4 The Assessment Group provided an economic model comparing CPAP with dental devices and with lifestyle management. The base-case model included people with moderate OSAHS and included the following health states: OSAHS, OSAHS post-coronary heart disease (CHD), OSAHS post-stroke, and death. People remained in one of the health states for 1 year, and could remain in the initial OSAHS state until death or until they experienced a road traffic accident, stroke or CHD event, which could result in disability. The OSAHS post-CHD and OSAHS post-stroke states incorporated the increased mortality and morbidity associated with having these events. People could remain in the post-stroke or post-CHD state until death. No complications or symptoms were included, and the model had a lifetime time horizon and was from a UK NHS perspective.
4.2.5 Health effects in the model included decreased utility associated with ESS score, cardiovascular events, stroke and road traffic accidents, and effects on mortality associated with cardiovascular events, stroke and road traffic accidents. The Assessment Group developed its own mapping algorithm to transform ESS data into utility scores. For this, the Assessment Group used three sets of individual patient data that measured ESS score and SF-36 and/or EQ-5D profile in the same people. A simple linear regression model was fitted to predict absolute utility scores from absolute ESS scores, controlling for baseline utility and ESS scores. This utility mapping was then applied to data on mean difference in ESS score between CPAP and placebo (23 studies) and between CPAP and dental devices (6 studies).
4.2.6 The base-case ICERs for men were £2000 per QALY gained for dental devices compared with lifestyle management, and £3899 per QALY gained for CPAP compared with dental devices. The ICERs for women were similar. The Assessment Group undertook a series of subgroup analyses based on baseline severity of OSAHS as measured by the ESS. This analysis excluded road traffic accidents and cardiovascular events. The resulting ICERs for CPAP compared with lifestyle management were £20,585 per QALY gained for mild OSAHS, £9391 per QALY gained for moderate OSAHS and £4413 per QALY gained for severe OSAHS. Dental devices were extendedly dominated by CPAP for moderate OSAHS, and there were no data for comparisons of dental devices with CPAP for mild or severe OSAHS.
4.2.7 Only two of the Assessment Group's subgroup and scenario analyses resulted in pronounced changes to the base-case ICERs. When the lifespan of the device was changed from 7 to 5 years and an auto-titrating device plus humidifier was used instead of a fixed-pressure device, the ICER was £16,362 per QALY gained. When cardiovascular events and road traffic accidents were excluded in the analysis for the total population (all severities of OHAHS), the ICER was approximately £8000 per QALY gained.
4.3 Consideration of the evidence
4.3.1 The Appraisal Committee reviewed the data available on the clinical and cost effectiveness of CPAP, having considered evidence on the nature of the condition and the value placed on the benefits of CPAP by people with OSAHS, those who represent them, and clinical specialists. It was also mindful of the need to take account of the effective use of NHS resources.
4.3.2 The Committee sought the opinion of the clinical specialists and patient experts on the impact of OSAHS on people's lives and on the experience of using CPAP devices. The Committee heard that OSAHS can be a debilitating condition that affects both the individual and their immediate family. It can also have a wider impact through the increased risk of road traffic accidents caused by people with untreated or undiagnosed OSAHS.
4.3.3 The Committee considered the alternative treatment options available for people with OSAHS, namely lifestyle advice, dental devices and surgery. The Committee heard that advice to lose weight is often not effective, partly because people with severe OSAHS lack the energy to engage in weight loss activities. The experts stated that dental devices can be uncomfortable and of limited effectiveness for most people, even those with mild OSAHS. The Committee also discussed the use of surgical procedures to correct craniofacial features associated with OSAHS, and understood that such forms of surgery were not usually considered to be viable treatment options because of lack of efficacy.
4.3.4 The Committee noted that the clinical effectiveness of CPAP was usually assessed in RCTs by measuring the ESS score (self-reported episodes of daytime sleep or dozing). The Committee heard from the clinical specialists that the ESS score does not cover all effects of OSAHS, and may underestimate the effectiveness of CPAP. However, the Committee was persuaded, on the basis of the RCT evidence and statements from the clinical specialists and patient experts that CPAP treatment is associated with considerable reductions in daytime sleepiness compared with placebo or lifestyle management. The Committee noted that most of the available RCT evidence did not identify a statistically significant reduction in daytime sleepiness for CPAP compared with dental devices. It heard from the clinical specialists that the reason for this could be that dental devices are not normally used for treating severe OSAHS, so such comparisons were likely to be weighted towards the less severe end of the condition, where generally the treatment effects are smaller. The Committee noted that the absolute effect of CPAP increased with increasing severity of OSAHS.
4.3.5 The Committee reviewed the evidence on the impact of CPAP on quality of life. The Committee noted that most of the available evidence did not identify a statistically significant difference in favour of CPAP. However, the Committee noted that studies that included dimensions specifically related to sleepiness found evidence of improvements in quality of life with CPAP therapy.
4.3.6 The Committee discussed the variable levels of adherence to CPAP therapy. It heard that adherence or under-use can be monitored and that it is routine practice for those under-using a CPAP device to be asked to return it, so that it can be used for another person.
4.3.7 The Committee reviewed the available evidence on the cost effectiveness of CPAP for the treatment of severe OSAHS, namely the analyses from one of the manufacturers (ResMed (UK)) and from the Assessment Group, and noted that the base-case ICERs in both analyses were below £5000 per QALY gained.
4.3.8 The Committee reviewed the results of the sensitivity analyses available. It understood that there was a lack of conclusive evidence for a beneficial effect of CPAP on blood pressure across all severity grades, and that the effect on cardiovascular events was inferred in the economic model via this effect of lowering of blood pressure. However, the Committee noted that excluding the effect of CPAP on cardiovascular events in the model did not lead to significant changes in the ICER.
4.3.9 The Committee noted that there was a lack of evidence on which to base the lifespan of the device, and that reducing the lifespan of the device from 7 to 5 years increased the ICER considerably in both economic models. However, the Committee was persuaded by the clinical specialists and patient experts that the assumed 7-year lifespan is plausible.
4.3.10 The Committee considered the regression mapping to predict utility values from ESS scores used in the Assessment Group model. The Committee was concerned that the utility values were not based on clinical trial evidence and also, given the reservations about the ESS, that these utility values were associated with some degree of uncertainty. However, the Committee agreed that in the absence of other plausible utility values, the approach taken by the Assessment Group was appropriate.
4.3.11 The Committee noted that the data on the likely beneficial effect of CPAP treatment on the rate of road traffic accidents came from observational studies and were not consistently supported by the RCT evidence on driving performance. Furthermore, it heard from the clinical specialists that people diagnosed with OSAHS but not yet successfully treated are not permitted to drive by the UK Driver and Vehicle Licensing Authority. Therefore, the Committee agreed that it was important also to take into account the sensitivity analyses that excluded road traffic accidents.
4.3.12 The Committee considered the findings of the subgroup analysis for different severity grades of OSAHS. This subgroup analysis was only available excluding cardiovascular events and road traffic accidents. The Committee noted that the ICERs for moderate and severe OSAHS were below £10,000 per QALY gained, even when road traffic accidents were excluded from the economic modelling. It therefore agreed that, for people with moderate or severe OSAHS, CPAP would be an appropriate use of NHS resources and should be recommended as a treatment option. The Committee further agreed that the diagnosis, treatment and monitoring of OSAHS should be carried out by specialists with experience in the management of sleep disorders.
4.3.13 The Committee noted that for people with mild OSAHS, the utility gains from using CPAP were lower than for those with moderate or severe OSAHS. The only analysis available for this subgroup excluded road traffic accidents and resulted in an ICER of £20,585 per QALY gained. The Committee heard from the clinical specialists that usually CPAP was considered not to be appropriate for people with mild symptomatic CPAP, because the inconvenience associated with use of the device would outweigh the benefits in reduction of OSAHS symptoms. Therefore the Committee concluded that other options, such as providing advice and support on lifestyle management, including helping people to lose weight, stop smoking and/or decrease alcohol consumption, should be considered first for this group. However, the Committee also heard that there may be people with mild severity grading who have considerable OSAHS symptoms that affect their quality of life and ability to go about their daily activities, and therefore could benefit from CPAP treatment. It agreed that, based on the ICER of £20,585 per QALY gained, CPAP should only be available as a treatment option for people with mild symptomatic OSAHS if lifestyle advice and other relevant treatment options have been unsuccessful or are considered inappropriate.
4.3.14 The Committee discussed the use of CPAP therapy for children and adolescents with OSAHS. The Committee heard that OSAHS is less common among children than in adults and that the clinical issues and aetiology in children are different from those encountered in adults. The Committee concluded that the recommendations for CPAP should apply only to adults with OSAHS.
4.3.15 The Committee discussed whether there are important differences between models and makes of CPAP devices, for example between fixed and auto-titrating devices or between those with and without a humidifier. The Committee heard from the clinical specialists and patient experts that the available CPAP devices are broadly similar, and that there is no robust evidence on patient preferences. The Committee therefore concluded that decisions on the make and type of CPAP device to be used should be made on the basis of the requirements of the individual.