QAngio XA 3D QFR and CAAS vFFR imaging software for assessing coronary stenosis during invasive coronary angiography

CAAS vFFR

3.5 Of the 4 studies reporting the diagnostic accuracy of CAAS vFFR only 1 (ILUMIEN I) reported a 2 x 2 table of diagnostic accuracy, and only 1 presented a Bland-Altman plot (FAST; Masdjedi et al. 2019) from which data were extracted to calculate diagnostic accuracy. Two of the studies were conference abstracts and only reported sensitivity and specificity without confidence intervals (Jin et al. 2019 and FAST EXTEND). There was notable heterogeneity across this small number of studies.

3.6 The EAG noted that the meta-analyses of the CAAS vFFR studies should be interpreted with caution because imputation of data (replacing missing data with substituted values) was needed. This was for 2 studies on the prevalence of FFR results below and above the cut-off for revascularisation decisions (0.80 or less), and because of the high heterogeneity across studies. The results of these bivariate meta-analyses are summarised in table 1.

3.7 Only 1 study, reported as a conference abstract, directly compared CAAS vFFR with QAngio QFR. It concluded that diagnostic performance of CAAS vFFR was poorer than for QAngio QFR, with area under the curves of 0.719 (95% confidence interval [CI] 0.621 to 0.804) for CAAS vFFR and 0.886 (95% CI 0.807 to 0.940) for contrast QFR (cQFR).

QAngio QFR

3.8 The EAG did a meta-analysis of the included studies, focusing on the diagnostic accuracy of QAngio QFR to detect lesions or vessels needing intervention (defined as having an FFR of 0.80 or less). Two approaches were used. The primary analysis consisted of a meta-analysis of reported diagnostic accuracy data. The secondary analysis used a data extraction approach in which FFR and QAngio QFR values from published plots were extracted and used to calculate diagnostic accuracy. This second approach allowed for a wider range of analyses.

3.9 The EAG identified 26 studies with sufficient diagnostic accuracy data to be included in the primary meta-analysis. Both univariate and bivariate meta-analyses of sensitivity and specificity were done and compared. These were divided into 3 modes of QAngio QFR: fixed-flow QFR (fQFR), contrast QFR (cQFR) and studies in which the type of QAngio QFR was not specified. Most studies included in the primary analysis used FFR as the reference standard, using a cut-off of 0.8, although 1 study used iFR as the reference standard. The EAG noted that there was no conclusive evidence of a significant difference between cQFR and fQFR.

3.10 In the univariate meta-analysis for the random-effect analysis, QAngio QFR at a cut-off of 0.80 had good diagnostic accuracy to predict FFR (also at a cut-off of 0.80). cQFR had a sensitivity of 85% (95% CI 78% to 90%) and specificity of 91% (95% CI 85% to 95%); fQFR had a sensitivity of 82% (95% CI 68% to 91%) and specificity of 89% (95% CI 77% to 95%). Studies that did not specify the mode of QAngio QFR had a sensitivity of 84% (95% CI 78% to 89%) and specificity of 89% (95% CI 87% to 91%).

3.11 Summary positive predictive values were 77% (95% CI 69% to 83%) for fQFR, 85% (95% CI 80% to 89%) for cQFR and 80% (95% CI 76% to 84%) for non-specified QAngio QFR (see figure 27 in the appendix of the diagnostics assessment report). Summary negative predictive values were 92% (95% CI 89% to 94%) for fQFR, 91% (95% CI 85% to 94%) for cQFR and 91% (95% CI 87% to 93%) for non-specified QAngio QFR.

3.12 The results of the bivariate meta-analysis were almost identical to the univariate analyses, with no conclusive evidence of a significant difference between fQFR and cQFR. The results of this analysis are summarised in table 2.

Abbreviations: QFR, quantitative flow ratio; cQFR, contrast QFR; fQFR, fixed-flow QFR

3.13 The mean difference between QAngio QFR and FFR was almost exactly zero for all 3 modes of QAngio QFR testing. For fQFR the mean difference was 0 (95% CI -0.05 to 0.06), for cQFR the mean difference was -0.01 (95% CI -0.06 to 0.04) and for non-specified QAngio QFR the mean difference was 0.01 (95% CI -0.03 to 0.05). FFR and QAngio QFR were highly correlated in all studies, with correlation coefficients of 0.78 (95% CI 0.72 to 0.82) for fQFR, 0.78 (95% CI 0.70 to 0.85) for cQFR and 0.79 (95% CI 0.73 to 0.83) for non-specified QAngio QFR.

3.14 The secondary analysis allowed for a wider range of analyses, such as considering different QAngio QFR and FFR cut-offs, and the effect of using a grey zone, in which people with intermediate QAngio QFR values go on to have confirmatory FFR.

3.15 A bivariate meta-analysis of diagnostic accuracy using data extracted from figures gave summary estimates for sensitivity and specificity of 84.6% (95% CI 80.7% to 87.8%) and 87.2% (95% CI 83.4% to 90.3%), respectively. This was similar to the results from the primary analysis when cQFR and non-specified QFR were combined.

3.16 QFR, as measured by QAngio, was highly correlated with FFR (r=0.80). In 50% of people, QFR and FFR differed by no more than 0.04. In 95% of people, values differed by no more than 0.10.

Grey-zone analysis

3.17 In the grey-zone analysis:

3.18 This strategy increased diagnostic accuracy compared with using QAngio QFR alone. The sensitivity was 93.1% (95% CI 90.1% to 94.9%) and the specificity was 92.1% (95% CI 88.3% to 94.5%). A total of 20.1% of people were in the grey zone and would have confirmatory FFR. However, only 30.4% of people with QAngio QFR results in the grey zone had results that differed from their FFR.

Invasive coronary angiography

3.19 The EAG identified 5 studies included in the meta-analysis that also reported 2 x 2 table data on the diagnostic accuracy of using 2D or 3D invasive coronary angiography alone. These studies used 50% diameter stenosis as the cut-off and FFR of 0.80 or less as the reference standard. Given the small number of studies, and because 2D and 3D invasive coronary angiography may have very different performance, no bivariate meta-analysis of these data was done. However the results of the individual studies showed that the diagnostic accuracy of invasive coronary angiography was inferior to QAngio QFR.

3.20 To inform the economic analysis, the EAG did an additional pragmatic search for studies that compared 2D invasive coronary angiography with FFR assessment. Data extracted from these studies showed that compared with QAngio QFR, the correlation of 2D invasive coronary angiography with FFR was much weaker (correlation coefficient -0.432). A bivariate meta-analysis of these extracted data produced summary sensitivity and specificity estimates of 62.6% (95% CI 51.5% to 72.5%) and 61.6% (95% CI 53.1% to 69.4%), respectively.

Test failure

3.21 The most reported (15 studies) causes of exclusion were issues with image acquisition and quality (for example, lack of at least 2 projections with a 25 degree angle in between, or poor image quality). The second most reported reason for exclusion was anatomical features of arteries (for example, excessive overlapping or foreshortening, ostial lesions, severe tortuosity).

3.22 Exclusion rates for QAngio QFR were higher overall in retrospective studies (median 28%, range 6% to 92%) compared with prospective studies (median 17%, range 7% to 52%). This may be partly explained by the fact that invasive coronary angiography images in retrospective studies were less likely to have been collected following manufacturer instructions.

3.23 There were only 2 retrospective CAAS vFFR studies that reported exclusion rates, and these were both high at 63% and 65%. In both studies most exclusions were because of angiographic image processing issues (rather than directly because of CAAS vFFR).

Variability

3.24 There were 8 studies that reported outcomes data on reproducibility of QAngio QFR readings between 2 different analysts (inter-observer variability). QAngio QFR was found to have a moderate to high level of inter-observer reliability.

3.25 There were 8 retrospective studies that reported outcomes data on intra-observer reproducibility of QAngio QFR readings. The time gap between initial and repeated measurements was reported in 4 studies and ranged from 3 days to 2 weeks. Most studies reported a high level of intra-observer reliability for QAngio QFR.

Timing

3.26 There were 6 studies of QAngio QFR that reported the time needed to complete QFR analysis. Time to QFR data acquisition ranged from an average of 2 minutes 7 seconds to 10 minutes (standard deviation 3 minutes). One study of 268 patients reported that time to image acquisition significantly decreased with the number of invasive coronary angiographies analysed, from 5 minutes 59 seconds to 2 minutes 7 seconds between the first and last 50 patients.

Morbidity, mortality and major adverse events

3.27 There were 3 cohort studies that reported mortality or major clinical outcomes in eligible patients with QAngio QFR measurements. All found that a clinically significant QAngio QFR predicted a higher incidence of long-term major cardiovascular adverse events. No data were reported for CAAS vFFR.

Subsequent use of invasive pressure-wire FFR

3.28 Five studies included in the diagnostic accuracy review retrospectively derived a grey-zone strategy based on their diagnostic accuracy results to model a potential reduction in adenosine and FFR use. These results are summarised in table 3.

Abbreviations: FFR, fractional flow reserve; QFR, quantitative flow ratio

Simulation study of clinical effectiveness

3.29 Because of the lack of published data on QAngio's clinical effectiveness, the EAG did a simulation study to investigate its possible effect on coronary outcomes compared with FFR.

3.30 The sample population was taken from data extracted from published Bland-Altman figures. Only cQFR or non-specified QAngio QFR data were used, for 3,193 people, each with an FFR measurement and its associated QAngio QFR measurement. To predict coronary outcomes, the results of the recent IRIS-FFR registry report were used. This represented 5,846 people who either had revascularisation (stent or bypass surgery) or were deferred (continued with current management without surgery) based on their measured FFR result. The IRIS-FFR study used major adverse cardiovascular events as its primary outcome.

3.31 Three strategies for deciding whether to revascularise were investigated:

3.32 If using the FFR only strategy 40.2% of people would have revascularisation. Using the QAngio QFR only strategy 42.0% would have revascularisation, and using the grey-zone strategy 43.2% would have revascularisation. Using QAngio QFR therefore moderately increased the revascularisation rate, and using it with a grey zone increased it further.

3.33 These simulations suggest that using FFR may prevent slightly more major adverse cardiovascular events, at around 1 event per 1000 people, but the overlap in simulated distributions means it is highly uncertain whether the difference is genuine. By contrast, the simulation suggests that QAngio QFR increases the number of revascularisations done, without substantially improving the number of major adverse cardiovascular events prevented. Overall these simulations suggested that there was little conclusive clinical difference between using QAngio QFR and FFR to make revascularisation decisions.

Model inputs

3.38 The prevalence of functionally significant stenosis in the population was based on studies that reported values of FFR and cQFR or non-specified QFR. It was assumed that the population in these QAngio QFR studies reflected the UK population. This suggested a prior likelihood of functionally significant stenosis of 40.2%, based on the proportion of people in the studies who had an FFR measurement of 0.80 or less.

3.39 The proportion of positive or negative test results when using the QAngio QFR, CAAS vFFR or invasive coronary angiography (strategies 3, 5 and 1) was based on the estimated accuracy of the 3 tests. The diagnostic accuracy estimates for these 3 tests are shown in table 4.

Abbreviations: ICA, invasive coronary angiography

3.40 The diagnostic accuracy of QAngio QFR in strategy 4 was based on the joint distribution of QFR and FFR measurements in the extracted individual-level patient data. The probabilities of QAngio QFR test results being positive (QFR less than 0.78), negative (QFR more than 0.84) or inconclusive (QFR of 0.78 to 0.84) are shown in table 5.

Abbreviations: FFR, fractional flow reserve; QFR, quantitative flow ratio

3.41 The rates of FFR and iFR procedural complications applied in the base-case analysis are summarised in table 6.

3.42 The rate of procedural deaths associated with revascularisation was sourced from UK audit data, which gives a 0.99% death risk for non-emergency coronary artery bypass graft and 0.17% for percutaneous coronary intervention. The mortality rate associated with revascularisation was estimated as a weighted average of the mortality rates for percutaneous coronary intervention and coronary artery bypass graft. This was relative to the proportion of percutaneous coronary interventions and coronary artery bypass graft procedures. In the base case, 87% of revascularisation procedures were assumed to be percutaneous coronary intervention, and 13% were coronary artery bypass graft.

3.43 The reported 1-year and long-term (up to 3 years) cumulative incidence of major adverse cardiovascular events in the IRIS-FFR registry for deferred lesions was used in the model to estimate the baseline risk of major adverse cardiovascular events for the first year and subsequent years. The baseline risk of major adverse cardiovascular events used in the model for people in the group with the highest FFR values (0.91 or more) was 0.64% in the first year and 0.32% per year in subsequent years. The hazard ratios were 1.06 (95% CI, 0.99 to 1.13), 1.09 (95% CI, 1.05 to 1.14), 1.07 (95% CI, 1.06 to 1.09) per 0.01 decrease in FFR for cardiac death, myocardial infarction, and unplanned or urgent revascularisation, respectively.

3.44 The treatment effect of revascularisation on major adverse cardiovascular events in people with stable coronary artery disease is highly uncertain. The ISCHEMIA trial, a randomised, parallel, open-label clinical trial comparing revascularisation with optimal medical therapy did not find evidence that revascularisation reduced the risk of major adverse cardiovascular events. Therefore, in the base-case analysis, the diagnostic tests did not benefit major adverse cardiovascular events outcomes. Scenario analyses were done to explore the impact of this assumption.

Health-related quality of life

3.45 By identifying the appropriateness for revascularisation, the tests can have health benefits through greater symptom relief and, therefore, higher health-related quality of life (HRQoL). Because the base-case analysis assumed that there was no treatment effect of revascularisation on major adverse cardiovascular events, the improvement in symptom relief was the only benefit. The HRQoL effects of revascularisation were based on the FAME trials. Both were randomised, parallel, open-label clinical trials. FAME I compared invasive coronary angiography with FFR for guiding percutaneous coronary interventions in patients with multivessel coronary artery disease. FAME II compared clinical outcomes, safety and cost-effectiveness of FFR-guided percutaneous coronary intervention with optimal medical treatment alone in patients with stable coronary artery disease. These trials showed that HRQoL improved significantly from baseline after percutaneous coronary intervention.

3.46 In the diagnostic model a one-off procedural disutility was applied for people having invasive FFR or iFR and for those who had revascularisation. In the prognostic model, a one-off utility decrement was also applied for people who had a non-fatal myocardial infarction or needed an unplanned revascularisation. A separate utility decrement was applied to the post-myocardial infarction health state, to reflect a decrease in HRQoL for those with a history of myocardial infarction.

3.47 The base-case analysis made an assumption that the quality-adjusted life year (QALY) loss applied for FFR or iFR was representative of both types of pressure wire procedures. The QALY loss estimates associated with each procedure in the diagnostic model are summarised in table 7.

Abbreviations: ICA, invasive coronary angiography; FFR, fractional flow reserve; iFR, instantaneous wave-free ratio; PCI, percutaneous coronary intervention; CABG, coronary artery bypass graft; QALY, quality-adjusted life year

Costs

3.48 The base-case costs of QAngio QFR with a throughput of 200 people per year was £430.61 per person tested. This was based on the purchase of vouchers for 100 people, which covered the cost of the software license and the training and certification of up to 4 QAngio QFR users, in addition to a staff cost per person tested of £7.76. The base-case costs of CAAS vFFR with a throughput of 200 people per year was £172.18 per person tested. This included staff training and annual maintenance and was based on the purchase of a perpetual license which allows analysis of as many people as needed per year. The model did not consider a cost for invasive coronary angiography because all people who entered the diagnostic model had this test.

3.49 The unit cost for FFR and iFR was estimated as the difference between the activity weighted average of the healthcare resource group codes for complex and standard cardiac catheterisation (£436.80).

Assumptions

3.50 The following assumptions were applied in the base-case analysis:

Base-case results

3.51 The deterministic and probabilistic cost-effectiveness results for the base-case analysis, expressed in terms of net health benefit at a maximum acceptable incremental cost-effectiveness ratio (ICER) of £20,000 per QALY gained, are shown in tables 9 and 10, respectively. The incremental net health benefit was calculated for each strategy compared with invasive coronary angiography alone. The results were consistent for both the deterministic and probabilistic analysis.

NHB and INHB are measured at a maximum acceptable ICER of £20,000 per QALY gained. Incremental NHB is relative to ICA alone. Abbreviations: ICA, invasive coronary angiography; FFR, fractional flow reserve; QFR, quantitative flow ratio; vFFR, vessel FFR; QALY, quality-adjusted life year; NHB, net health benefit; INHB, incremental NHB

NHB and INHB are measured at a maximum acceptable ICER of £20,000 per QALY gained. Incremental NHB is relative to ICA alone. Abbreviations: ICA, invasive coronary angiography; FFR, fractional flow reserve; QFR, quantitative flow ratio; vFFR, vessel FFR; QALY, quality-adjusted life year; NHB, net health benefit; INHB, incremental NHB

3.52 Strategy 2 (invasive coronary angiography with FFR) had the highest net health benefit and the highest probability of being cost effective, although the differences between all the strategies were small. Strategy 1 (invasive coronary angiography alone) was the cheapest and had the lowest QALY gain, while strategy 5 (invasive coronary angiography with vFFR) was the most expensive and had the highest QALY gain.

Analysis of alternative scenarios

3.53 Results from the scenario analyses showed that the base-case results were generally robust when alterations were made to the sources of data used in the model and when different assumptions were made. However, sometimes these alterations resulted in significant changes to the net health benefit rankings of the different strategies.

3.54 In the base case, the diagnostic accuracy estimates for vFFR were based on the FAST EXTEND study (sensitivity 97.0% and specificity 74.0%), the largest study of vFFR (330 patients). Using accuracy estimates from ILUMIEN I reduced the cost effectiveness of vFFR, but estimates from Jin et al. 2019 increased it. This resulted in vFFR being the second most cost-effective strategy. This highlighted the substantial uncertainty surrounding the cost effectiveness of vFFR in strategy 5.

3.55 When QAngio QFR was considered to have the same diagnostic accuracy as FFR (that is, perfect sensitivity and specificity), the total QALYs and costs for strategy 3 increased by 0.017 QALYs and £6 per person from the base-case scenario. In this scenario strategy 3 became cost effective with the highest net health benefit, largely because of greater total QALYs gained for strategy 3 compared with strategy 2. This difference was mainly because of the procedural disutility associated with FFR or iFR.

3.56 When the procedural disutility of FFR was more than that used in the base case, the net health benefit of strategies 2 and 4 were affected most. The total QALYs for both strategies were reduced, resulting in strategy 2 becoming the second least cost effective and strategy 3 the most cost effective. An FFR disutility of 0.014 QALYs resulted in an equal net health benefit for strategies 2 and 3. This procedural disutility was 2.5 times greater than that associated with percutaneous coronary intervention, but less than half the disutility associated with coronary artery bypass graft.

3.57 In terms of how duration of HRQoL affected cost effectiveness, the benefits need to last for at least 7 years to offset the disutility associated with FFR or iFR in the base case for strategy 2 to remain more cost effective than strategy 3.

3.58 The benefits of revascularisation, in terms of improved HRQoL, suggested that the sensitivity of test results was a more important driver of cost effectiveness than specificity. This was because true positive test results translated into higher QALY gains than mismanagement of false negative test results.

3.59 In a diagnostic-only setting, the large additional costs of repeating diagnostic catheterisation at a subsequent appointment in an interventional laboratory for strategies involving measuring FFR or iFR (strategies 2 and 4) meant that strategies without this testing component were more cost effective. Strategy 3 (QAngio QFR alone) became the strategy with the highest net benefit, followed by strategy 5 (CAAS vFFR alone).