3 Clinical evidence

Guidance

Summary of diagnostic accuracy evidence
Summary of clinical-effectiveness evidence

3 Clinical evidence

3.1

The key clinical outcomes for HeartFlow FFR_CT presented in the decision problem were:

measures of diagnostic accuracy (sensitivity and specificity, positive and negative likelihood ratios, area-under curve) using invasive fractional flow reserve (FFR) as the reference standard
rates of diagnostic coronary angiography, percutaneous coronary intervention and coronary artery bypass surgery
adverse events (test-related, major adverse cardiac events, radiation exposure and so on)
quality of life
mortality.

Summary of diagnostic accuracy evidence

3.2

The company conducted a literature search on the diagnostic accuracy of FFR_CT and the existing tests in the current treatment pathway for patients with a 10% to 90% pre-test likelihood of coronary artery disease, against a reference standard of invasive FFR testing. This review identified 5 relevant meta-analysis studies and 23 individual studies, 1 of which was unpublished. Based on the 22 published studies, and using FFR as the reference standard, the company presented diagnostic accuracy per-patient results for HeartFlow FFR_CTcompared with:

invasive coronary angiography (ICA)
single-photon emission CT (SPECT)
stress echocardiogram (ECHO)
magnetic resonance imaging (MRI)
coronary CT angiography (CCTA).

If there were multiple studies for a test, the company conducted a meta-analysis; for example, 3 studies were included in the meta-analysis for HeartFlow FFR_CT (Koo et al. 2011, Min et al. 2012 and Nørgaard et al. 2014). The methodology and results of the meta-analyses are reported as academic in confidence.

3.3

The external assessment centre (EAC) reviewed the company's selection of studies and considered that although they addressed the scope in terms of the comparators, reference test and outcomes, most included a mixture of patients with both high (over 90%) and intermediate (10% to 90%) pre-test likelihoods of disease. It also disagreed with the company's decision only to include studies that provided FFR measurements in more than 75% of blood vessels. The EAC considered this criterion not to be reflective of clinical practice, where visual assessment is sometimes used before proceeding with FFR measurements. The EAC also noted that this criterion did not reflect the company's proposed changes to the clinical pathway, where CCTA would be used to decide if HeartFlow FFR_CT should be used.

3.4

To address these concerns, the EAC conducted a diagnostic literature search using extra keywords related to comparators and outcomes. It included only studies in which most patients had an intermediate pre-test likelihood of disease. The EAC identified 7 diagnostic studies, including 3 presented by the company (Bernhardt et al. 2012, Nørgaard et al. 2014 and Stuijfzand et al. 2014) and 3 that the company had identified but excluded (Danad et al. 2013, Kajander et al. 2010 and Ponte et al. 2014). Only 1 of these, Nørgaard et al. 2014, involved HeartFlow FFR_CT.

3.5

Nørgaard et al. (2014) reported on a multicentre study (the NXT trial) involving 2 UK centres, which compared HeartFlow FFR_CT (v1.4) with CCTA for diagnosing myocardial ischaemia in 254 patients with suspected stable coronary artery disease scheduled to have ICA. Most patients in the study (87%) were considered to have an intermediate likelihood of coronary artery disease. Invasive FFR was measured in all vessels (n=484). The study reported the diagnostic performance of HeartFlow FFR_CT and CCTA for diagnosing ischaemia compared with FFR measured during ICA as the reference standard. The diagnostic accuracy of each test was presented on a per-patient and a per-vessel basis compared with the reference standard, an FFR value of ≤0.80. Per-vessel FFR_CT was correlated to FFR (Pearson's correlation coefficient 0.82, p>0.001), with a slight underestimation of FFR_CT compared with FFR. The authors concluded that HeartFlow FFR_CT can identify functionally significant coronary artery disease with high sensitivity and specificity. Furthermore, adding FFR_CT measurements to CCTA led to a marked increase in specificity.

3.6

The EAC identified 6 studies which both used the comparator tests and included patients with an intermediate likelihood of coronary artery disease. Bernhardt et al. (2012) compared the diagnostic performance of 1.5 T and 3 T MRI scanners using FFR as a reference standard in 34 patients with stable angina and suspected or known coronary artery disease. The authors studied an intermediate-risk population with a mean PROCAM score of 42.7 (a risk assessment metric which estimates the 10‑year risk of developing a coronary event). Ponte et al. (2014) compared the diagnostic accuracy of CCTA and MRI for detecting functionally significant coronary artery disease in patients referred with suspected coronary artery disease, using ICA with FFR as the reference standard. The study included 95 patients with a 15% to 85% pre-test likelihood of coronary artery disease. Stuijfzand et al. (2014) evaluated CCTA and transluminal attenuation gradient compared with CCTA alone for diagnosing functionally significant lesions, using invasive FFR as the reference standard. The study included 85 patients (253 vessels) with an intermediate likelihood of coronary artery disease. Neglia et al. (2015) assessed the accuracy of several imaging techniques – CCTA, SPECT and ECHO – in 475 patients with an intermediate likelihood of coronary artery disease. Danad et al. (2013) evaluated the diagnostic accuracy of CCTA in 120 patients with suspected coronary artery disease who had cardiac positron emission topography (PET), CCTA and ICA. CCTA was done using a hybrid PET/CT scanner. Kajander et al. (2010) evaluated the diagnostic accuracy of PET and CCTA in 107 patients with a history of stable chest pain and a 30% to 70% pre-test likelihood of coronary artery disease. All patients had ICA independently of the non-invasive imaging results, and treatment decisions were based on both ICA and FFR.

3.7

Table 1 summarises the EAC's analysis of diagnostic accuracy for HeartFlow FFR_CT and its comparators at both per-vessel and per-patient levels, as shown in table 1. When there was more than 1 diagnostic accuracy study available, the EAC conducted a meta-analysis.

**Table 1 Diagnostic accuracy: HeartFlow FFRCT and comparator tests**
Type of analysis	Index test	N	Sensitivity	Specificity	Positive likelihood ratio	Negative likelihood ratio
Patient based	HeartFlow FFR _CT (Nørgaard, 2014: NXT trial)	254	0.86 0.77 to 0.93	0.79 0.72 to 0.85	4.07 3.02 to 5.49	0.18 0.10 to 0.31
Patient based	CCTA (6 studies)	1,136	0.95 0.92 to 0.97	0.68 0.65 to 0.71	3.18 1.56 to 6.47	0.09 0.05 to 0.16
Patient based	ECHO (Neglia, 2015)	261	0.45 0.33 to 0.57	0.90 0.85 to 0.94	4.52 2.74 to 7.45	0.61 0.49 to 0.76
Patient based	ICA (Nørgaard, 2014)	254	0.64 0.52 to 0.74	0.83 0.76 to 0.88	3.70 2.57 to 5.33	0.44 0.33 to 0.59
Patient based	MRI (2 studies)	129	0.89 0.78 to 0.95	0.91 0.82 to 0.97	8.59 4.12 to 17.9	0.13 0.07 to 0.26
Patient based	SPECT (Neglia, 2015)	293	0.73 0.63 to 0.81	0.67 0.60 to 0.74	2.20 1.74 to 2.79	0.41 0.29 to 0.57
Vessel based	HeartFlow FFR _CT (Nørgaard, 2014)	484	0.84 0.76 to 0.91	0.86 0.82 to 0.89	5.97 4.60 to 7.75	0.18 0.12 to 0.29
Vessel based	CCTA (4 studies)	1,645	0.85 0.81 to 0.89	0.75 0.73 to 0.77	4.15 2.38 to 7.23	0.19 0.12 to 0.32
Vessel based	ICA (Nørgaard, 2014)	484	0.55 0.45 to 0.65	0.90 0.87 to 0.93	5.56 3.92 to 7.89	0.50 0.40 to 0.62
Vessel based	MRI (Bernhardt, 2012)	102	0.87 0.72 to 0.96	0.98 0.92 to 1.00	55.6 7.92 to 390	0.13 0.06 to 0.30

Abbreviations: CCTA, coronary CT angiography; CI, confidence interval; ECHO, stress echocardiogram; FFR_CT, fractional flow reserve CT; ICA, invasive coronary angiography; MRI, magnetic resonance imaging; SPECT, single-photon emission CT.

3.8

The EAC considered that despite the limitations associated with patients having a different reference test in some studies, all contributed to the decision problem and provided data for synthesis. It judged that the Nørgaard (2014) study had a low risk of bias for flow and timing, index and reference test. It noted that there was a risk of patient selection bias because an inclusion criterion was that patients had to have been referred for ICA, but it noted no other risks of bias or applicability concerns. Although it acknowledged that there were no studies directly comparing all the tests, it concluded that HeartFlow FFR_CT has:

similar sensitivity but higher specificity compared with CCTA
higher sensitivity but lower specificity compared with ECHO
similar sensitivity but lower specificity compared with MRI
higher sensitivity and specificity compared with SPECT.

Summary of clinical-effectiveness evidence

3.9

The company conducted a literature search for evidence on the clinical outcomes specified in the decision problem for HeartFlow FFR_CT, and the existing treatments, against any comparator. It identified 16 studies of which 5 included HeartFlow FFR_CT, 1 published (Guar et al. 2014) and 4 unpublished (PLATFORM, Radiation FFR_CT, Real World Usage FFR_CTand FFR_CT RIPCORD).

3.10

The EAC included extra intervention and comparator keywords and identified 11 studies, 4 of which had already been identified by the company: 2 published studies (Hachamovitch et al. 2012 and Douglas et al. 2015) and 2 unpublished studies. The EAC noted that only the 2 unpublished studies fully matched the population, intervention, comparators and outcomes defined in the scope; the other 9 included various comparators but not HeartFlow FFR_CT. The 2 unpublished studies including HeartFlow FFR_CT were PLATFORM (see section 3.18) and Radiation FFR_CT; the company provided both in the form of interim results for the former and an abstract for the latter. Two studies (Real World Usage FFR_CTand FFR_CT RIPCORD) included HeartFlow FFR_CT but were excluded because they did not provide information on patients' pre-test likelihood of coronary artery disease.

3.11

Radiation FFR_CT is a single-centre modelling study, based in Canada, investigating the potential effect of HeartFlow FFR_CT on radiation dose exposure and downstream clinical event rate. In the modelling, a clinical pathway in which CCTA plus FFR_CT was the initial diagnostic test was compared with 3 clinical pathways instead utilising SPECT, ECHO or CCTA as initial diagnostic tests. The model included 100 patients with suspected coronary artery disease, 34% of whom had intermediate disease. Patients were stratified into 3 categories of likelihood of disease: 50% low, 40% moderate and 10% high. No clinical outcomes were measured in this modelled population. The primary outcome was the estimated radiation dose and the secondary outcome was death or myocardial infarction estimates at 1 year after the test. Of the 4 diagnostic pathways studied, ECHO had the lowest radiation dose (5.3 mSv) but had a higher clinical event rate related to both false-positive and false-negative findings. The FFR_CT pathway had lower cumulative radiation exposure (9.4 mSv) than SPECT (26.4 mSv) or CCTA (13.9 mSv) and also had the lowest clinical adverse event rate for low and intermediate-risk patients. For high-risk patients, the lowest clinical event rate was with ICA.

3.12

The PROMISE study (Douglas et al. 2015) is a US‑based multicentre randomised controlled trial involving over 10,000 patients, with a median follow‑up of 25 months. Although the study did not include FFR_CT, the EAC considered it relevant to the decision problem because it provides further evidence on a diagnostic pathway based on CCTA. Patients with a mean pre-test likelihood of coronary artery disease of 53.3% plus or minus 21.4% were randomly assigned to either CCTA or functional imaging as a first-line diagnostic test. The composite primary end point was death, myocardial infarction, hospitalisation for unstable angina, or major procedural complication. Secondary end points included invasive cardiac catheterisation that did not show obstructive coronary artery disease and radiation exposure. Results showed that 164 of 4,996 (3.3%) patients in the CCTA group and in 151 of 5,007 (3.0%) in the functional testing group (adjusted hazard ratio, 1.04; 95% confidence interval, 0.83 to 1.29; p=0.75) achieved the primary end point. CCTA was associated with fewer catheterisations showing no obstructive coronary artery disease than functional imaging (3.4% compared with 4.3%, p=0.02), although more patients in the CCTA group had catheterisation within 90 days of randomisation (12.2% compared with 8.1%). The median cumulative radiation exposure per patient was lower in the CCTA group than in the functional testing group (10.0 mSv compared with 11.3 mSv), but 32.6% of the patients in the functional testing group had no exposure. As such, overall exposure was higher in the CCTA group (mean 12.0 mSv compared with 10.1 mSv; p<0.001).

3.13

The EAC identified 9 published studies containing information on clinical outcomes in comparator diagnostic technologies. Further information about these studies can be found in the assessment report.

Chest pain guideline update: second literature search

3.14

During the assessment of HeartFlow FFR_CTfor this guidance, NICE updated its guideline on chest pain. Because this included new recommendations for investigating chest pain, it became necessary to update the evidence and cost modelling for the HeartFlow FFR_CT assessment. The EAC repeated the evidence searches up to February 2016 and asked the company to identify any recent and ongoing studies. In total, the EAC assessed 7 new studies, 6 of which included HeartFlow FFR_CT.

3.15

Tanaka et al. (2016) is a technical study on a subgroup of the NXT study investigating the association between FFR_CT and invasive FFR in coronary arteries with serial lesions. The authors investigated patients (n=18 patients and 18 vessels) with stable angina and clinically suspected coronary artery disease. There was no clinical follow‑up. The primary outcome was the per-segment correlation between FFR_CT and invasive FFR values, expressed as translesional delta (the difference between the proximal and distal FFR measurement of all sequential lesions). Values of translesional delta for FFR and FFR_CT were 0.10 plus or minus 0.09 and 0.09 plus or minus 0.10 in distal segments, and 0.17 plus or minus 0.10 and 0.22 plus or minus 0.13 in proximal segments respectively. The coefficient of correlation between translesional delta FFR and FFR_CT in each segment was 0.92 (p<0.001). The authors concluded that translesional delta FFR is highly correlated with FFR_CT.

3.16

Thompson et al. (2015) investigated the diagnostic performance of FFR_CT in relation to patients' sex and age, using invasive FFR measurements as the reference standard for a subgroup of the DeFACTO study. Previous evidence from DeFACTO was not considered eligible because it included patients with a high pre-test likelihood of coronary artery disease (Min et al. 2012). Thompson et al. (2015) was included because it reports results based on subgroup analyses for age and sex. The baseline pre-test likelihood did not differ in statistical significance within these subgroups, so it is not expected to bias the results. The authors investigated 252 patients (407 vessels) with stable angina, clinically suspected coronary artery disease and at least 1 coronary stenosis of 30% to 90%. For their analysis, the authors used a clinical rule that included all vessels of diameter more than or equal to 2 mm and assigned an FFR value of 0.90 for vessels with stenoses less than 30% and an FFR value of 0.50 for vessels with stenoses more than 90%. There was no clinical follow‑up. The primary outcome was per-patient and vessel diagnostic performance of FFR_CT. Using this clinical rule, diagnostic performance improved in both sexes with no statistically significant differences between them. There were no differences in the discrimination of FFR_CT after application of the clinical use rule when stratified by age 65 and over or less than 65 years. The authors concluded that FFR_CT had similar diagnostic accuracy and discriminatory power to FFR for ischaemia detection in men and women irrespective of age using a cut‑off point of 65 years.

3.17

The other 4 studies on HeartFlow FFR_CT looked at clinical outcomes. The PLATFORM study (Douglas et al. 2015b and 2016) was presented to the committee as academic in confidence in June 2015 (Douglas et al. 2015a). The study included 584 patients recruited at 11 international centres. They were prospectively assigned to have either functional imaging (n=287) or CCTA/FFR_CT (n=297). Each cohort was subdivided into 2 groups based on the evaluation plan decided before enrolment in the study: non-invasive testing (any form of stress testing or CCTA without FFR_CT) or ICA (invasive testing).

3.18

Douglas et al. (2015b) report the study results at 3‑month follow‑up. The primary end point was the percentage of patients with planned ICA in whom no significant obstructive coronary artery disease (no stenosis more than or equal to 50% by core laboratory quantitative analysis or invasive FFR less than 0.80) was found at ICA within 90 days. Secondary end points included a composite measure of major adverse cardiac events consisting of death, myocardial infarction and unplanned revascularisation, all of which were independently and blindly assessed. Among patients with intended ICA (CCTA/FFR_CT=193; functional imaging=187), no obstructive coronary artery disease was found with ICA in 24 patients (12%) in the CCTA/FFR_CT arm and 137 patients (73%) in the functional imaging arm (risk difference 61%, 95% CI 53 to 69, p<0.0001). Among patients intended for non-invasive testing, the rates of finding no obstructive coronary artery disease with ICA were 13% in the CCTA/FFR_CT arm and 6% in the functional imaging arm (p=0.95). ICA was cancelled in 61% of patients after reviewing the CCTA/FFR_CT results. There were low numbers of MACE and vascular complications in all groups.

3.19

Douglas et al. (2016) report outcomes from the same study at 1 year. The clinical end points measured were MACE and MACE plus vascular events within 14 days of procedure. Quality of life and resource use outcomes were also collected. There were 2 MACE events in each arm of the planned invasive group (risk difference -0.03 [CI -8.6 to 8.5]) and 1 in the planned non-invasive group (risk difference -1.00 [CI -12.7 to 10.7]). Cumulative 1‑year radiation exposure in patients in the intended invasive evaluation cohort was similar between the usual care strategy (mean: 10.4 plus or minus 6.7 mSv) and CCTA/FFR_CT-guided strategy (mean: 10.7 plus or minus 9.6 mSv; p=0.21), whereas in the non-invasive testing cohort it was higher in patients with a CCTA/FFR_CT-guided strategy than usual care strategy (mean: 9.6 plus or minus 10.6 mSv compared with 6.4 plus or minus 7.6 mSv, p<0.001). Functional status and quality of life improved from baseline to 1‑year follow‑up in the planned non-invasive group (p<0.001 for all measures), with greater improvements on the EQ‑5D in patients having CCTA/FFR_CT compared with patients having functional imaging (mean change: 0.12 for CCTA/FFR_CT compared with 0.07 for functional imaging, p=0.02).

3.20

Lu et al. (2015) used a subgroup analysis of the PROMISE trial (n=181) to investigate the added value of FFR_CT compared with CCTA in improving efficiency of referral to ICA. End points for the subgroup analysis were rate of revascularisation and ICA that did not show obstructive coronary artery disease and MACE. Over a median follow‑up of 25 months, the addition of FFR_CTincreased the rate of ICA with revascularisation from 49% to 61%. The rate of angiography without obstructive disease decreased from 27% to 11%. No patient with FFR_CT more than 0.80 had an adverse event which ICA would have prevented.

3.21

Nørgaard (2016) reports on the real-world experience of using CCTA with FFR_CT as gatekeeper to ICA in patients with stable coronary artery disease and intermediate-range coronary lesions (n=189). Patients were followed up for a median of 12 months. The primary end point was the impact of FFR_CT on further downstream diagnostic testing. Other end points were the agreement between FFR_CT and invasive FFR, and the short-term clinical outcome after FFR_CT testing defined as the occurrence of MACE (death and acute myocardial infarction) or an angina episode leading to hospital admission or visit in the outpatient clinic. The authors concluded that FFR_CT testing is feasible in real-world scenarios involving patients with intermediate-range coronary stenosis determined by CCTA. They also concluded that implementing FFR_CT for clinical decision-making may influence the downstream diagnostic workflow, and patients with an FFR_CT more than 0.80 who are not referred for ICA have a favourable short-term prognosis. The authors highlight that in patients with FFR_CT ranging between 0.76 and 0.80, a non-negligible number of false-positive results may be expected.

3.22

The EAC considered that the 1‑year follow‑up data from the PLATFORM study supported the company's claims about resource use, rates of ICA and percutaneous coronary intervention, and quality of life with HeartFlow FFR_CT. Additionally, the 1‑year follow‑up evidence from the PLATFORM supports the company's claim that MACE outcomes are equivalent between the current pathway and one that uses FFR_CT, whereas the PROMISE study showed that MACE outcomes at 1 year were equivalent between CCTA alone and functional testing. The EAC also considered that the evidence from the PLATFORM study showed higher 1‑year radiation exposure in the HeartFlow FFR_CT group in patients intended for non-invasive evaluation. However, this is to be expected because many patients in the non-invasive evaluation had a non-invasive test which did not need the use of radiation. The EAC concluded that the submitted evidence on clinical outcomes supports the value proposition of an FFR_CT-guided strategy compared with standard of care, mainly in patients with planned invasive investigation, with equivalent results between FFR_CT and functional imaging in the non-invasive group.

Committee considerations

3.23

The committee considered that the evidence showed high diagnostic accuracy and increased specificity with HeartFlow FFR_CT compared with CCTA alone. It also noted promising results from the PLATFORM study, in a population which closely matches that in the scope. The evidence was sufficient to conclude that HeartFlow FFR_CT has a high diagnostic accuracy for coronary artery disease, and that its use has the potential to reduce the need for invasive coronary investigations.

3.24

The committee considered the technology to be innovative and understood that its adoption may serve to simplify a complex patient pathway. The committee heard from clinical experts that they had confidence in the diagnostic accuracy of HeartFlow FFR_CT, and that it could provide an effective early rule-out test for coronary artery disease. This would reduce the need for ICA and invasive FFR measurement, and potentially reduce radiation exposure.

3.25

The committee understood that there are differences in the local implementation of the patient pathway for diagnosing coronary artery disease. It was advised by clinical experts that the choice of functional imaging test depends on local access, available expertise and clinician preference. It heard that although HeartFlow FFR_CT has the potential to reduce the number of tests that are done, the other non-invasive functional imaging tests that are part of the current patient pathway offer different functionality and in some cases provide additional information. Overall, the committee concluded that HeartFlow FFR_CT should be considered for use as a non-invasive investigation for diagnosing angina in patients with stable, recent-onset chest pain of suspected cardiac origin, and that it provides the clinician with additional functional information to determine which coronary lesions are responsible for myocardial ischaemia. The committee considered that further clinical studies would be helpful to clarify the wider applicability of HeartFlow FFR_CTin routine clinical practice.

3.26

The committee considered the evidence from the PLATFORM study to be most relevant to the decision problem. It considered that the results demonstrate the potential of FFR_CT to avoid ICA and improve quality of life.

3.27

The committee discussed the relative importance of a per-patient or a per-vessel diagnosis. It heard from experts that per-patient diagnostic accuracy was more important for initial diagnosis, and that a per-vessel assessment provides additional information to inform patient management. The committee concluded that per-patient level figures were the most reliable and relevant to the diagnosis of coronary artery disease.