Guidance
5 Outcomes
The Diagnostics Advisory Committee considered evidence from a number of sources but primarily the assessment performed by the External Assessment Group.
How outcomes were assessed
5.1 The assessment consisted of a systematic review of the evidence on clinical-effectiveness data for the 3 depth of anaesthesia monitors compared with standard clinical monitoring. The outcome measures included consumption of anaesthetic agents, time to extubation, time to discharge from the recovery room, probability of awareness during surgery, patient distress and other sequelae resulting from awareness during surgery, morbidity including post-operative cognitive dysfunction, and mortality.
Clinical effectiveness
Bispectral Index
5.2 A Cochrane review on 'Bispectral Index for improving anaesthetic delivery and post-operative recovery' provided a basis for assessing the clinical effectiveness of BIS. It included 31 randomised controlled trials of BIS monitoring compared with standard clinical practice. All of the trials included in the Cochrane review were conducted in adults. The External Assessment Group identified 11 randomised controlled trials that were published after the Cochrane review and compared the clinical effectiveness of the BIS monitor with standard clinical monitoring. Five of these trials were conducted in children aged 2 to 18 years. Two of the trials were conducted in populations with known risk factors for awareness during surgery (for example, patients undergoing cardiac or airway surgery). These 11 trials were used to supplement the Cochrane review. The method of administering general anaesthesia varied across the 11 trials. Five trials used inhaled anaesthetic (predominantly sevoflurane) for both induction and maintenance of general anaesthesia. Three other trials used intravenous anaesthesia (propofol) for both induction and maintenance of general anaesthesia (total intravenous anaesthesia). The remaining 3 trials used both intravenous and inhaled anaesthesia. Two used propofol for the induction of anaesthesia and sevoflurane for the maintenance of anaesthesia. Muscle relaxants were used in 7 of the trials.
5.3 A total of 6 trials identified by the External Assessment Group reported awareness during surgery as an outcome and 3 of these trials reported this as the primary outcome. The 3 trials that did not report awareness as the primary outcome had no cases of awareness during surgery. These 3 trials were not designed to detect awareness during surgery, and it is likely that the sample sizes were insufficient to detect this uncommon outcome. In the 3 trials that did report awareness as the primary outcome, there were 29 cases of confirmed or possible awareness during surgery with BIS monitoring and 30 cases with the comparators used in the studies. One trial, monitoring inhaled anaesthesia in patients classified as being at high risk of awareness during surgery, reported 19 definite or possible cases of awareness in the group with BIS monitoring (n=2861) compared with 8 definite or possible cases in the group with clinical monitoring, which included a structured protocol with audible alarms for monitoring end-tidal anaesthetic concentration (n=2852). This difference was not statistically significant. The use of structured protocols is not considered part of standard clinical monitoring in the NHS. A second trial, in patients at increased risk of awareness receiving total intravenous anaesthesia, reported 8 cases of confirmed or possible awareness in the group with BIS monitoring (n=2919) compared with 21 cases in the standard clinical monitoring group (n=2309). The lower incidence of confirmed awareness in the group with BIS monitoring was statistically significant. A third trial, monitoring inhaled or intravenous anaesthesia in patients not classified at greater risk, reported 2 cases of awareness during surgery in the group with BIS monitoring (n=67) compared with 1 case in the group with standard clinical monitoring (n=61). Statistical significance was not reported. This trial measured awareness with explicit recall using a modified Brice interview and awareness with implicit recall using a word recognition test. The sample size of this study was small and may have contributed to the inconclusive results.
5.4 The Cochrane review on BIS included a meta-analysis of awareness during surgery with recall, which included 4 trials in patients at high risk of awareness during surgery. This meta-analysis was updated by the External Assessment Group to include 2 further trials in patients at high risk of awareness during surgery. After the addition of these 2 trials, the odds ratio increased from 0.33 to 0.45, indicating a statistically significant difference between groups favouring BIS. However, there was a large amount of heterogeneity between the trials.
5.5 Six trials identified by the External Assessment Group reported anaesthetic consumption as an outcome and 2 of these reported it as the primary outcome. Three of the trials showed a statistically significant reduction in the use of inhaled anaesthetic in the group with BIS monitoring compared with the group with standard clinical monitoring. The other 3 trials reported use of intravenous anaesthetic. Two of these trials reported a higher maintenance dose of anaesthetic with BIS monitoring compared with standard clinical monitoring, but there was no statistically significant difference between the 2 groups. The third trial reported a 25.3% reduction in the consumption of intravenous anaesthetic (propofol) with BIS monitoring compared with standard clinical monitoring. No statistical significance was reported in the trial.
5.6 The Cochrane review of BIS included a meta-analysis of anaesthetic consumption, with separate analyses for inhaled anaesthetic consumption and intravenous anaesthetic consumption. When these meta-analyses were updated by the External Assessment Group, the mean difference (in MAC equivalents) in inhaled anaesthetic consumption was slightly reduced from −0.16 to −0.15 but remained statistically significant. The mean difference in intravenous anaesthetic consumption was also slightly reduced from −1.44 mg/kg/h to −1.33 mg/kg/h but remained statistically significant.
5.7 Of the 11 trials identified by the External Assessment Group, 5 reported time to extubation as a secondary outcome. All 5 trials showed that time to extubation was reduced by 0.5 to 5 minutes with BIS monitoring compared with standard clinical monitoring. Two of these trials reported statistically significant results.
5.8 Five trials identified by the External Assessment Group reported the time to discharge from the recovery room as a secondary outcome, and 4 of these trials were conducted in children. All of the trials showed that the time to discharge was shorter by 6.7 to 30 minutes in the group with BIS monitoring than in the group with standard clinical monitoring. These results were reported as statistically significant in all trials. However, the point at which the time to discharge began varied across the trials. One trial reported the time to discharge from the end of surgery and 2 others reported time to discharge from the end of general anaesthesia.
5.9 In the Cochrane review, 12 trials were included in the meta-analysis of the time to discharge from the recovery room. The mean difference in the Cochrane review was −7.63 minutes in favour of BIS. The External Assessment Group did not update the Cochrane review for this outcome because of heterogeneity between studies.
5.10 One trial conducted in children receiving inhaled anaesthesia reported post-operative nausea and vomiting as a secondary outcome. There was no significant difference between BIS monitoring and standard clinical monitoring in the number of children with nausea (n=5 [10%] and n=6 [11%] respectively, p=0.95) or with vomiting (n=2 [4%] and n=3 [6%] respectively, p=0.88). The Cochrane review did not report post-operative nausea and vomiting.
5.11 The evidence on long-term cognitive dysfunction following general anaesthesia was limited to 1 study (reported in a conference abstract) of patients over 60 years of age. This study reported a reduction in post-operative cognitive dysfunction at 7 days and 3 months with BIS monitoring, although the difference at 7 days was not statistically significant.
E-Entropy
5.12 Seven randomised controlled trials comparing the clinical effectiveness of the E-Entropy monitor with standard clinical monitoring were included in the systematic review conducted by the External Assessment Group. Two of these studies were conducted in children (aged 3 to 12 years). None of the trials was conducted in populations with known risk factors for awareness during surgery.
5.13 The method of administering general anaesthesia varied across trials. Two trials used inhaled anaesthetic (sevoflurane) and 3 trials used intravenous anaesthetic (propofol), for both induction and maintenance of general anaesthesia. Two trials used intravenous anaesthesia for induction followed by an inhaled anaesthetic for maintenance of general anaesthesia. All but 1 trial used muscle relaxants.
5.14 There was 1 case of awareness during surgery in the 6 trials that reported this outcome. This occurred in the standard clinical monitoring group. Sample sizes were small in all of the trials, so uncommon events such as awareness during surgery may not have occurred or have been detected.
5.15 Four trials showed a statistically significant reduction in the consumption of inhaled anaesthetic with E-Entropy monitoring compared with standard clinical monitoring, although 1 of these trials showed no reduction in the total amount of anaesthetic consumed. By contrast, no statistically significant reduction in the consumption of intravenous anaesthetic was found in a trial reporting the consumption of intravenous anaesthetic as a primary outcome. However, 2 trials that reported the consumption of intravenous anaesthesia as a secondary outcome did show lower propofol consumption with E-Entropy monitoring compared with standard clinical monitoring that was statistically significant.
5.16 Three trials reported time to extubation as a secondary outcome. All showed that time to extubation was shorter by approximately 3 to 4 minutes with E-Entropy monitoring compared with standard clinical monitoring. Two of these trials reported this reduction in time to extubation as statistically significant. Two trials reported that the time to discharge from the operating room to the recovery room was reduced by approximately 3 to 4 minutes with E-Entropy monitoring compared with standard clinical monitoring. Both trials reported that this result was statistically significant. Only 1 trial reported the time to discharge from the recovery room. The group with E-Entropy monitoring was discharged sooner than the group with standard clinical monitoring, but the difference was not statistically significant.
5.17 One trial conducted in patients receiving intravenous anaesthesia reported post-operative nausea and vomiting as a secondary outcome. There was no statistically significant difference in the number of patients with nausea and vomiting in the group with E-Entropy monitoring and in the group with standard clinical monitoring.
Narcotrend-Compact M
5.18 Four randomised controlled trials comparing the clinical effectiveness of the Narcotrend-Compact M monitor with standard clinical monitoring were included in the systematic review conducted by the External Assessment Group. All of these were conducted in adults. None reported risk factors in the study populations for awareness during surgery.
5.19 The method of administering general anaesthesia varied across trials. Three trials used total intravenous anaesthesia (propofol-remifentanil or propofol-fentanyl) and 1 other trial had a mix of patients receiving intravenous anaesthesia and inhaled anaesthetic (propofol-remifentanil and desflurane-remifentanil) for general anaesthesia. Three trials used muscle relaxants.
5.20 There were no cases of awareness during surgery in any of the trials reporting the clinical effectiveness of the Narcotrend-Compact M monitor.
5.21 Of 3 trials that reported consumption of the anaesthetic propofol, 2 showed a statistically significant reduction in consumption with Narcotrend-Compact M monitoring compared with standard clinical monitoring. The third trial showed no difference in propofol consumption between the 2 groups.
5.22 In 1 trial that reported time to extubation as a primary outcome, no difference was found between the group with Narcotrend-Compact M monitoring and the group with standard clinical monitoring. Two trials that reported time to extubation as a secondary outcome showed a statistically significant reduction of 1.4 to 6 minutes with Narcotrend-Compact M monitoring compared with standard clinical monitoring.
5.23 Two trials reported a statistically significant reduction in the time to arrival at the recovery room in the group with Narcotrend-Compact M monitoring compared with the group with standard clinical monitoring.
Cost effectiveness
5.24 A systematic review of the evidence on cost effectiveness for the 3 technologies was undertaken by the External Assessment Group. One study was identified that evaluated the cost effectiveness of standard clinical monitoring in combination with BIS monitoring compared with standard clinical monitoring alone. The cost per patient of BIS monitoring included the cost of the sensors and the monitor. An incidence of awareness during surgery of 0.04% was used for standard clinical monitoring in combination with BIS monitoring and 0.18% was used for standard clinical monitoring alone. The study concluded that the addition of BIS monitoring to standard clinical monitoring was not cost effective. However, the study did not include health-related quality of life and its methodology was of uncertain quality.
5.25 No studies were identified that included E-Entropy or Narcotrend-Compact M monitoring and met the inclusion criteria for the systematic review on cost effectiveness.
5.26 An economic model was developed by the External Assessment Group to assess the cost effectiveness of using a monitor to assess the depth of anaesthesia plus standard clinical monitoring compared with standard clinical monitoring alone. The model evaluated costs from the perspective of the NHS and personal social services. Outcomes were expressed as quality-adjusted life years (QALYs). Both costs and outcomes were discounted using a 3.5% annual discount rate. Separate economic analyses were conducted for each of the 3 technologies. No analyses were conducted to directly compare the technologies.
5.27 A decision tree model was developed to evaluate the outcomes and costs resulting from the use of depth of anaesthesia monitors as opposed to standard clinical monitoring alone. The relevant clinical outcomes included in the model were those associated with excessively deep levels and inadequate levels of general anaesthesia in the general surgical population and the population at high risk of awareness. Specifically, these were the risk of experiencing short-term adverse outcomes (such as post-operative nausea and vomiting) and long-term adverse outcomes (such as post-traumatic stress disorder and post-operative cognitive dysfunction), and the risk of experiencing awareness during surgery.
5.28 The model was also used to estimate the costs associated with depth of anaesthesia monitoring and the costs of treating short- and long-term adverse outcomes. It was assumed that the costs of monitoring clinical signs such as blood pressure and heart rate were common to all surgery with general anaesthesia with and without depth of anaesthesia monitoring. Therefore, these were not included in the model. The main costs associated with standard clinical monitoring in the model were costs of anaesthesia, costs of adverse outcomes related to anaesthesia and costs of managing long-term sequelae of awareness during surgery. The costs associated with post-operative nausea and vomiting were also included. No impact of short-term adverse outcomes on quality of life was included in the model because, by definition, these are expected to be of short duration.
5.29 Three separate models were developed, 1 for each monitoring system. However, the model structures were the same, with only the values for the parameters varying. The models used different values for the risks associated with standard clinical monitoring (without a depth of anaesthesia monitor) corresponding to the results in the respective trials. As a result, no direct comparisons of the monitors were performed.
5.30 For each monitor, 4 analyses were performed; 2 each for the population at general risk of adverse outcomes from anaesthesia and for the population at high risk of adverse outcomes from anaesthesia. For each of the 2 populations, 2 analyses were performed; 1 for patients receiving total intravenous anaesthesia and 1 for a general mix of patients regardless of the type of anaesthesia.
5.31 Unit costs for depth of anaesthesia monitors included the acquisition cost of the monitor (annual cost assuming a 5-year effective life and converted to an average cost per patient based on assumptions of patient throughput) and recurring costs arising from the single-use sensors. The cost of the monitors varied from £4867 for the BIS monitor to £10,825 (the midpoint of a range of prices for Narcotrend-Compact M). Sensor costs varied more widely, with costs per patient of £14.08 for BIS, £8.68 for E-Entropy and £0.56 for Narcotrend-Compact M.
5.32 The cost-effectiveness estimates in the following sections were, in most cases, derived using data from BIS monitoring for estimating the impact on awareness during surgery and its sequelae, and for long-term adverse outcomes of anaesthesia overdosing. No robust evidence was identified on the effect of the E-Entropy or Narcotrend-Compact M monitors on awareness during surgery and its sequelae, or for long-term adverse outcomes of anaesthesia overdosing. Therefore, the effect estimates derived from studies using the BIS monitor were applied to E-Entropy and Narcotrend-Compact M in the modelling.
Patients at high risk of adverse outcomes from anaesthesia receiving total intravenous anaesthesia
5.33 The base-case analysis for patients at high risk of adverse outcomes from anaesthesia receiving total intravenous anaesthesia resulted in incremental cost-effectiveness ratios (ICERs) of £21,940, £14,421 and £5681 per QALY gained for BIS, E-Entropy and Narcotrend-Compact M monitoring respectively, compared with standard clinical monitoring alone.
5.34 Sensitivity analyses showed that the ICERs for BIS, E-Entropy and Narcotrend-Compact M monitoring were sensitive to changes in the probability of awareness during surgery. When the probability of awareness was 0.0006, the ICER for BIS monitoring was £82,903 per QALY gained and, with a probability of 0.0119, the ICER was £8027 per QALY gained compared with standard clinical monitoring alone. The corresponding ICERs for E-Entropy monitoring were £56,429 per QALY gained and £4834 per QALY gained respectively. The corresponding ICERs for Narcotrend-Compact M monitoring were £25,656 per QALY gained and £1123 per QALY gained respectively.
5.35 The ICER for BIS monitoring was also sensitive to changes in the probability and duration of post-traumatic stress disorder, the effectiveness of the BIS module, the quality-of-life decrement applied to post-traumatic stress disorder and the unit cost of the sensors.
5.36 In contrast to BIS monitoring, the ICER for E-Entropy monitoring was robust to changes in the unit cost of the sensors. The ICER for E-Entropy monitoring was sensitive to changes in the relative risk of awareness and changes in the quality-of-life decrement applied to post-traumatic stress disorder.
5.37 The sensitivity analysis for Narcotrend-Compact M monitoring showed that the ICER was robust to most changes in the parameters. However, the ICER was sensitive to changes in the probability of awareness and the decrement applied to post-traumatic stress disorder.
Patients at general risk of adverse outcomes from anaesthesia receiving total intravenous anaesthesia
5.38 The base-case analysis for patients at general risk of adverse outcomes from anaesthesia receiving total intravenous anaesthesia resulted in ICERs of £33,478 and £31,131 per QALY gained for the use of BIS and E-Entropy monitors respectively, compared with standard clinical monitoring alone. Monitoring with the Narcotrend-Compact M monitor dominated standard clinical monitoring in this population (that is, it was more effective and less costly than standard clinical monitoring).
5.39 As in patients at high risk of adverse outcomes from anaesthesia receiving total intravenous anaesthesia, the ICERs for BIS monitoring and E-Entropy monitoring were sensitive to changes in the probability of awareness. When the probability was 0.0023, the ICER for BIS monitoring was £25,778 per QALY gained compared with standard clinical monitoring alone. When the probability was 0.001, the ICER increased to £44,491 per QALY gained. The corresponding ICERs for E-Entropy monitoring were £23,936 and £41,419 per QALY gained respectively. The ICERs were also sensitive to changes in the probability of post-traumatic stress disorder and the quality-of-life decrement applied to post-traumatic stress disorder. The ICER for E-Entropy monitoring was also sensitive to changes in the effectiveness of the E-Entropy module.
5.40 The sensitivity analysis showed that the ICER for Narcotrend-Compact M monitoring in this general risk population was robust to changes in parameters. Narcotrend-Compact M monitoring dominated standard clinical monitoring by generating improved outcomes at reduced costs.
Patients at high risk of adverse outcomes from anaesthesia receiving either intravenous or inhaled anaesthesia
5.41 The base-case analysis for patients at high risk of adverse outcomes from anaesthesia receiving intravenous or inhaled anaesthesia resulted in ICERs of £29,118, £19,367 and £8,033 per QALY gained for the use of BIS, E-Entropy and Narcotrend-Compact M monitors respectively, compared with standard clinical monitoring alone.
5.42 Sensitivity analyses showed that the ICERs for BIS, E-Entropy and Narcotrend-Compact M monitoring were most sensitive to changes in the probability of awareness. When the probability was 0.0119, the ICER for BIS monitoring compared with standard clinical monitoring alone was £11,591 per QALY gained, rising to £93,139 per QALY gained when the probability was 0.0006. The corresponding ICERs for E-Entropy monitoring were £7290 and £63,483 per QALY gained respectively. The corresponding ICERs for Narcotrend-Compact M monitoring were £2290 and £29,010 per QALY gained respectively.
5.43 Changes in the relative risk of awareness with the BIS module, probability of developing post-traumatic stress disorder, the duration of post-traumatic stress disorder and the decrement in quality of life applied to post-traumatic stress disorder all led to large variations in the ICER for BIS monitoring, ranging from £22,207 to £61,433 per QALY gained compared with standard clinical monitoring alone.
5.44 The ICER for E-Entropy monitoring was also sensitive to an increase in the relative risk of awareness with the Entropy module, giving an ICER of £41,635 per QALY gained compared with standard clinical monitoring alone when the odds ratio was increased from 0.45 to 0.81. As in the population receiving total intravenous anaesthesia, the ICER was sensitive to changes in the probability of post-traumatic stress disorder and the decrement in quality of life applied to post-traumatic stress disorder.
5.45 The ICER for Narcotrend-Compact M monitoring was also sensitive to changes in the effectiveness of the Narcotrend-Compact M monitor, the proportion of patients who develop post-traumatic stress disorder and the quality-of-life decrement applied to post-traumatic stress disorder.
Patients at general risk of adverse outcomes from anaesthesia receiving either intravenous or inhaled anaesthesia
5.46 The base-case analysis for patients at general risk of adverse outcomes from anaesthesia receiving intravenous or inhaled anaesthesia resulted in ICERs of £47,882 and £19,000 per QALY gained for the use of BIS and E-Entropy monitors respectively, compared with standard clinical monitoring alone. Monitoring with the Narcotrend-Compact M monitor dominated standard clinical monitoring in this population (that is, it was more effective and less costly than standard clinical monitoring).
5.47 Sensitivity analysis showed that the ICER for BIS monitoring in this population was sensitive to changes in the probability of awareness with ICERs of £38,163 and £60,911 per QALY gained for probabilities of 0.0023 and 0.001 respectively, compared with standard clinical monitoring alone. The ICER was also sensitive to changes in the relative risk of awareness with the BIS monitor, changes in the probability of developing post-traumatic stress disorder, the duration of post-traumatic stress disorder and the unit costs of the sensors.
5.48 For E-Entropy monitoring, sensitivity analyses showed that the largest variation in the ICER from the base case of £19,000 per QALY gained was caused by changes in sevoflurane consumption, with ICERs ranging from £6494 to £31,567 per QALY gained, compared with standard clinical monitoring alone. When the probability of awareness was 0.0023 and 0.001 the ICERs were £14,881 and £24,521 per QALY gained respectively, compared with standard clinical monitoring alone.
5.49 The ICER for E-Entropy monitoring was also sensitive to changes in the probability of post-traumatic stress disorder, the decrement in quality of life applied to post-traumatic stress disorder and changes in the unit cost of the sensors.
5.50 The sensitivity analysis showed that the ICER for Narcotrend-Compact M monitoring in this population was generally robust to changes in the parameters. However, the ICER was sensitive to a change in the consumption of desflurane (−0.156 to −0.056), resulting in an ICER of £2534 per QALY gained compared with standard clinical monitoring alone.
5.51 Scenario analyses were performed to investigate the impact of varying the assumed number of patients per monitor per year (1000 patients) in the base-case analyses. These analyses showed that the number of patients per monitor only had a substantial effect on the ICERs at low patient numbers (less than 500 patients). This applied for all 3 monitors.