4 Evidence and interpretation
The Appraisal Committee (Appendix A) considered evidence from a number of sources (see Appendix B).
4.1 Clinical effectiveness
4.1.1 The Assessment Report and the submissions reviewed the literature and focused on two aspects separately: the diagnostic performance of SPECT, and its long-term prognostic value. Much of the evidence consisted of non-randomised open observational (both prospective and retrospective) studies, with several studies using a comparative design.
Diagnostic performance
4.1.2 The diagnostic performance of SPECT was expressed as sensitivity and specificity. Sensitivity is the proportion of true-positives that are correctly identified by the test. Specificity is the proportion of true-negatives that are correctly identified by the test.
4.1.3 The Assessment Report reviewed 21 studies with 100 or more patients that evaluated the sensitivity and specificity of both SPECT and sECG in the diagnosis of CAD compared with CA. Median sensitivity values for SPECT were higher than those for sECG in all studies (SPECT: 81% for the largest subcategory of studies, with a range of 63–93%; sECG: 65% for the largest subcategory of studies, with a range of 42–92%). However, the results were not pooled because of the heterogeneity across the different studies. Median specificity values were similar for SPECT (65%, range 10–90%) and sECG (67%, range 41–88%).
4.1.4 The submission from the professional groups reviewed the diagnostic performance of SPECT only (compared with CA) from 62 studies. Because of differences in inclusion criteria, only two of these studies were also included in the Assessment Report analysis. There was considerable variation in study size, quality and design, but weighted means for sensitivity and specificity were reported to be 86% and 74%, respectively. The manufacturer's submission quoted one publication with sensitivity and specificity for SPECT reported as 91% and 89%, respectively, and the American College of Cardiologists/American Heart Association Task Force guideline, with average sensitivity and specificity reported as 89–90% and 70–76%, respectively.
Long-term prognostic value
4.1.5 For the long-term prognostic value of SPECT, the Assessment Report included a systematic review of 46 observational studies.
4.1.6 In the 20 studies that provided general prognostic information, cardiac event rates (defined as cardiac mortality or non-fatal MI) were significantly higher for patients with abnormal SPECT scans than for those with normal scans. An abnormal SPECT result was associated with an annual cardiac event rate of 6.7%, whereas a normal scan was associated with an annual cardiac event rate of 0.7% (data from metaanalyses of 15,000 and 20,963 patients, respectively). Furthermore, the extent and size of a perfusion defect can predict the likelihood of future cardiac events.
4.1.7 The proportion of normal angiograms was lower in patients who were referred to CA after a positive SPECT than in patients referred directly for CA (two studies: 33% versus 43% [4688 patients], and 18% versus 33% [6800 patients], respectively).
4.1.8 However, the rate of subsequent revascularisations was lower for the SPECT-CA strategy (13–27%) than for the direct CA strategy (16–44%) (data from three studies with a combined total of approximately 11,000 patients).
4.1.9 In studies where it was possible to analyse the contribution of different clinical parameters to the prediction of clinical outcomes, it was found that SPECT provided independent prognostic information for predicting MI, and had an additional value over clinical and sECG data that was maintained at long-term follow-up.
4.1.10 In several studies that investigated whether an abnormal SPECT scan was a predictor of cardiac death, the relative risk or odds ratios were calculated depending on study design. In all studies an abnormal SPECT scan was described as an independent, main or statistically significant predictor of cardiac death. In four studies, with patient numbers ranging from 176 to 947, the relative risk ranged between 1.1 and 17.6. In two studies, with patient numbers of 248 and 1182, the odds ratios were reported to be 2.8 and 4.8, respectively.
4.1.11 SPECT also provided independent prognostic information in the following subgroups: women (five studies), patients post-MI (four studies), patients who had undergone PCI or CABG (three studies), medically treated patients with left main and/or three-vessel CAD (one study), patients hospitalised with angina who had a normal or non-diagnostic sECG (one study), and patients with diabetes (two studies).
4.1.12 Two studies found ECG-gated SPECT to be more sensitive than non-ECG-gated SPECT, but with slightly lower specificity. Also, ECG-gated SPECT provided incremental prognostic information in patients with known or suspected CAD that was better than perfusion data alone. One study compared SPECT with attenuation-corrected SPECT and reported that attenuation correction had a significant impact on the assessment of the severity and extent of MI.
4.1.13 The search strategy used in the Assessment Report did not identify any studies evaluating the role of SPECT in the context of rapid access chest pain clinics or in pre-operative risk assessment of patients undergoing major surgery who were potentially at risk of coronary events. However, the submission from the professional groups lists 20 studies on SPECT in pre-operative risk assessment, and emphasises the acknowledged role of SPECT for this indication.
4.1.14 In summary, as studies reviewed in the Assessment Report were carried out under a number of different clinical settings investigating different outcomes, it was not possible to summarise the effectiveness of SPECT in simple quantitative estimates. However, the evidence from the reviewed studies consistently suggested that SPECT provided valuable independent and incremental information predictive of outcome that helped to risk-stratify patients and influence the way in which their condition was managed.
4.1.15 The submissions from the professional groups and the manufacturer included reviews of a larger number of papers and, because of differences in the inclusion criteria, there was little overlap between the studies included in each of the three reviews. Despite the differences in the evidence base of the three reviews, similar conclusions were drawn.
4.2 Cost effectiveness
4.2.1 The Assessment Group, the manufacturer and the professional group reviewed published cost-effectiveness studies. The Assessment Group and the manufacturer also provided new economic models.
4.2.2 The systematic review in the Assessment Report included studies that compared both costs and outcomes of SPECT with alternative diagnostic strategies. The comparison of different publications was complicated by the multitude of strategies considered, differences in study designs and populations, in treatment comparisons, in costing methods and different ways in which outcomes were measured. Overall, it was concluded that direct CA (without any prior tests) was cost effective when the prevalence of disease was high. At low levels of prevalence, strategies involving SPECT and/or sECG were considered to be a better use of resources than a strategy of direct CA. Furthermore, strategies involving SPECT were often found to be dominant or provided additional benefits that might be considered worth the additional cost compared with the sECG-CA strategy.
4.2.3 The new economic models provided by the Assessment Group and the manufacturer used similar designs; decision tree models were constructed for the diagnostic performance of different strategies and Markov models were used to estimate the long-term costs and benefits. They both used a hypothetical cohort of 1000 patients (to start at the age of 60), with the assumption that effectiveness of therapy (CABG, PCI, medical management) lasts for 10 years. The time horizon was 25 years with an annual cycle time.
4.2.4 The diagnostic strategies considered in both models were:
-
sECG, followed by SPECT if sECG was positive or indeterminate, followed by CA if SPECT was positive or non-diagnostic (sECG-SPECT-CA)
-
sECG, followed by CA if sECG was positive or non-diagnostic (sECG-CA)
-
SPECT, followed by CA if SPECT was positive or non-diagnostic (SPECT-CA)
-
direct CA (CA).
4.2.5 The results were presented as incremental cost per true-positive diagnosed, per accurate diagnosis, per life year gained and per quality-adjusted life year (QALY) gained, and – importantly – were calculated for different levels of prevalence of CAD.
4.2.6 The key results were as follows:
-
As prevalence of CAD increased, total cost increased and total number of QALYs gained decreased for each diagnostic strategy.
-
At all prevalence levels of CAD the ordering of diagnostic strategies was the same, with sECG-SPECT-CA being least costly and least effective, and having the lowest average cost per QALY. This implies that an incremental cost is paid for some incremental benefit when SPECT is not included.
-
CA was the most costly strategy in both models and for all prevalence levels of CAD, and (as the reference standard) was defined as the most effective strategy.
-
Most incremental cost-effectiveness ratios (ICERs) were less favourable in the manufacturer's model than in the Assessment Report model. However, all ICERs calculated were less than £24,000, apart from the ICER for direct CA compared with SPECT-CA at low and 30% prevalence of CAD.
4.2.7 When compared with sECG-CA at low prevalence of CAD, the ICER for SPECT-CA (£8723) was more favourable than the ICER for direct CA (£21,538). Conversely, at high prevalence of CAD, the more favourable strategy was direct CA with an ICER of £1962, whilst SPECT-CA had an ICER of £3242.
4.2.8 When direct CA was compared with the SPECT-CA strategy, a high ICER was seen at low prevalence (£42,225). However, as prevalence increased, direct CA became increasingly more cost-effective. At 80% prevalence of CAD, the move to the direct CA from SPECT-CA involved a modest extra cost per additional QALY gained (£942 in the Assessment Report and £4482 in the manufacturer's submission).
4.2.9 Several sensitivity analyses showed that the results varied considerably depending on the sensitivity or specificity values entered for SPECT and sECG. When the impact of the additional independent information provided by SPECT was explored by increasing the proportion of SPECT positives whose condition could be satisfactorily managed medically, ICERs generally improved. When the time horizon was less than 15 years, all ICERs became less favourable. In the subgroup analysis for women, the SPECT-CA strategy dominated both the sECG-CA and CA strategies.
4.2.10 In summary, when compared with sECG-CA, SPECT-CA has more favourable ICERs than direct CA at low levels of prevalence of CAD. At higher prevalence levels, the sECG-CA and CA strategies lead to more favourable ICERs than SPECT-CA.
4.3 Consideration of the evidence
4.3.1 The Committee reviewed the evidence available on the clinical and cost effectiveness of MPS for the diagnosis and management of CAD, having considered evidence on the value placed by users on the benefits of MPS for the diagnosis and management of CAD, from people with CAD, those who represent them, and clinical experts. It was also mindful of the need to ensure that its advice took account of the effective use of NHS resources.
4.3.2 The Committee considered the evidence submitted on the diagnostic performance of SPECT indicating that, overall, it is more sensitive than sECG. However, the Committee appreciated that considerable uncertainty remains over the true values for sensitivity and specificity of SPECT. In particular, trials that assessed these values were subject to referral bias, in that only SPECT-positive cases were referred for CA, which was assumed to be the 'gold standard'. Additionally the Committee was aware that, contrary to SPECT, CA does not always provide the fullest evaluation of the patient with CAD, particularly where information relating to myocardial perfusion and function are considered important for the establishment of prognosis and management.
4.3.3 The Committee heard from the clinical experts that SPECT is of value at all levels of likelihood for CAD, because it provides highly accurate diagnostic and prognostic information. The experts indicated that, if SPECT and sECG were equally accessible in the NHS, there would be a case for the preferential use of SPECT in certain groups of patients. However, because of the currently limited availability of SPECT in the UK, the committee believed that its use should be particularly directed to patient groups for whom it provides the greatest additional benefit in terms of initial diagnosis of suspected CAD and in the management and prediction of prognosis in those with established CAD.
4.3.4 The Committee also recognised that there are circumstances where the information from sECG is important, as in the evaluation of the overall exercise performance of patients with CAD. sECG is therefore likely to remain a commonly used investigation in most circumstances.
4.3.5 The Committee reviewed the cost-effectiveness modelling. They noted that because the difference in QALYs derived between the different investigational strategies was small, and the disutility of CA was not included in the models, the conclusions of cost–utility differences between diagnostic strategies (see Section 4.2.4) should be interpreted with caution. However, the Committee considered that, overall, SPECT was cost effective across a wide range of clinical situations.
4.3.6 The Committee further considered that, in terms of both clinical effectiveness and cost effectiveness, the absolute 'value' of SPECT as an appropriate diagnostic tool depends on the likelihood of the presence of CAD in the target population under investigation. Thus the diagnostic strategy SPECT-CA is clearly preferred on cost-effectiveness grounds in individuals with a lower likelihood of CAD and consequently lower risk of future coronary events. However, as the likelihood of CAD increases, differences in the incremental cost effectiveness for the different testing strategies decrease. Thus, at higher likelihood of CAD and of possible intervention (CABG or PCI), a strategy where direct CA is preferred over SPECT-CA could be considered more appropriate.
4.3.7 The Committee heard from the experts that SPECT enables the redirection of patients into medical rather than surgical management. SPECT may therefore postpone or completely avert the need for CA in some clinical situations. The Committee also recognised the significance of the disutility associated with CA, which would favour SPECT and had been omitted from the economic models reviewed. It concluded that full consideration of these aspects is likely to improve the cost effectiveness of SPECT.
4.3.8 The Committee was advised by the experts that SPECT scanning may be particularly useful as an initial diagnostic tool in people for whom sECG poses particular problems of poor sensitivity or difficulties with interpretation. This includes women, patients with cardiac conduction defects (such as left bundle branch block) and people with diabetes. SPECT also has an important role in assessing the presence of CAD in patients for whom treadmill exercise is difficult or impossible, and in the full evaluation of patients following MI or reperfusion interventions.
4.3.9 The Committee considered that increased provision of SPECT within the NHS over that currently available was desirable on the basis of this evidence. However, it recognised that more widespread use of SPECT would require an implementation strategy that may take several years to fulfil and would need a significant increase in the availability of both equipment and trained staff. The Committee therefore concluded that the increased use of SPECT should initially be targeted at those groups for whom it provides the greatest benefit in terms of cost effectiveness, as expressed in Section 1.