Lead-I ECG devices for detecting symptomatic atrial fibrillation using single time point testing in primary care

Quality assessment of diagnostic accuracy studies

4.6 The QUADAS-2 tool was used to assess study quality. For patient selection, the EAG judged that all 9 studies had an unclear risk of bias and a high level of concern for applicability (because none were done in a population who had symptoms). For 1 study there was limited information available in the publication; Koltowski et al. (2017) was only available as a conference proceeding.

4.7 The included studies varied in how the devices gave a positive result for atrial fibrillation. This was either based on the lead‑I ECG device's diagnostic algorithm or on clinician interpretation of an ECG trace generated by the devices. The EAG judged that studies in which the device output was interpreted by a trained healthcare professional were more applicable (low concern) than those in which a lead‑I ECG device algorithm alone was used (high concern; Lau et al. 2013, Tieleman et al. 2014 and Vaes et al. 2014). The EAG presented results in 2 sections depending on how atrial fibrillation was identified (by a clinician or by the device's algorithm alone).

Diagnostic accuracy results: Lead-I ECG interpreted by a trained healthcare professional

4.8 Data were included from 4 studies, which assessed Kardia Mobile alone (Haberman et al. 2014; Williams et al. 2015), Kardia Mobile and MyDiagnostick (Desteghe et al. 2017) and Zenicor‑ECG alone (Doliwa et al. 2009).

4.9 Desteghe et al. reported separate accuracy estimates from lead‑I ECGs interpreted by 2 electrophysiologists; only pooled estimates using data from electrophysiologist 1 are shown in table 2 (values were similar when data from electrophysiologist 2 were used). Williams et al. reported separate accuracy estimates from lead‑I ECGs interpreted by a cardiologist or by a GP with a special interest in cardiology. Pooled accuracy estimates in table 3 used data from Williams et al. when the lead‑I ECG interpreter was a cardiologist (interpreters in other studies were cardiologists or electrophysiologists). Pooled accuracy estimates using data from Williams et al. when the interpreter was a GP with a special interest in cardiology (not shown) were similar. However, the study showed a decrease in specificity when the GP interpreted the lead‑I ECG; 76% (95% confidence interval [CI] 64% to 85%) compared with 86% (95% CI 76% to 94%) when the cardiologist interpreted them.

Diagnostic accuracy results: ECG trace interpreted by the device's algorithm

4.11 Four studies that reported sensitivity and specificity of the lead‑I ECG device when the trace was interpreted by the device's algorithm alone were included in meta-analyses. Two studies reported data for MyDiagnostick alone (Tieleman et al. 2014; Vaes et al. 2014), 1 study for Kardia Mobile alone (Lau et al. 2013) and 1 study for both MyDiagnostick and Kardia Mobile (Desteghe et al. 2017). Pooled sensitivity and specificity estimates from meta‑analyses are presented in table 4.

Comparisons between lead-I ECG devices

4.13 The EAG commented that the available data were not sufficient to formally assess differences between the different lead-I ECG devices. Desteghe et al. (2017) assessed the concordance between Kardia Mobile and MyDiagnostick. There was no statistically significant difference in agreement between the devices (based on kappa values) when assessing all patients (p=0.677) or after excluding those with an implanted device (for example, a pacemaker or implantable cardiac defibrillator; p=0.411).

4.14 The EAG commented that the pooled sensitivity and specificity values were similar across all the meta-analyses done, irrespective of how the lead‑I ECG trace was interpreted (algorithm or healthcare professional) or which lead‑I ECG devices were used (pooled estimates produced by the EAG used Kardia Mobile, MyDiagnostick and Zenicor-ECG).

Diagnostic accuracy results: further studies excluded from the EAG's main report

4.15 The EAG identified further studies that reported sensitivity and specificity estimates of the lead‑I ECG devices. However, it did not include them in its main report because they did not meet 1 of the eligibility criteria for inclusion, that is, that the reference standard in the studies was not a 12‑lead ECG interpreted by a trained healthcare professional. Results were presented in appendix 6 of the diagnostics assessment report. They included 1 unpublished study which assessed imPulse (no other studies were identified for this device). Ranges were reported for sensitivity (67% to 100%) and specificity (83% to 100%). These data were used in the economic model.

Diagnostic yield

4.17 There were 13 studies that reported diagnostic yield of atrial fibrillation detection by lead‑I ECG devices (various devices), which ranged from 0.38% to 5.84%. However, the location of testing varied between studies; primary care (6 studies), secondary care (2 studies), tertiary care (1 study) and in the community (4 studies). In the primary care studies, the range was 0.49% to 5.84%. None of the studies assessed people with signs and symptoms of atrial fibrillation. The enrolled populations varied from the general population or people who were attending primary care for a reason unrelated to atrial fibrillation (for example, for flu vaccination) to people admitted to a cardiology ward and people with known atrial fibrillation. The prevalence of atrial fibrillation in these populations is likely to vary and may not be applicable to the population that is the focus of this assessment. No data were found on any benefit of lead‑I ECGs in identifying people with paroxysmal atrial fibrillation, compared with later ECG testing.

Test failure rate

4.18 Test failure rate (which included both the device failing to produce a result and producing a poor-quality ECG trace) varied between 0.1% and 9% (various devices). Reasons suggested for uninterpretable lead‑I ECGs were sinus tachycardia or bradycardia, that patients had a tremor or that hospitalised patients were unable to hold the devices firmly enough.

Time to diagnosis of atrial fibrillation

4.19 A study done in Australia (Lowres et al. 2014) reported a time to diagnosis of atrial fibrillation of 16.6 days (standard deviation of 14.3 days) from detection by an initial lead‑I ECG diagnostic test at a pharmacy to confirmed diagnosis with a 12‑lead ECG.

Ease of use of devices

4.20 Tieleman et al. (2014) reported that people were able to use MyDiagnostick with minimal instructions. Chan et al. (2017) reported that Kardia Mobile was easy to use. Orchard et al. (2016) commented that it may be difficult for older people to hold the Kardia Mobile device still enough to take a reading. In Desteghe et al. (2017), 7% of people were excluded from the study because they could not hold the devices as intended (the study used both MyDiagnostick and Kardia Mobile).

Effect on clinical decision making

4.21 In Hussain and Thakrar (2016), 5 out of 6 people had a change in the clinical management of their condition after atrial fibrillation was detected by Kardia Mobile (1 person died as an inpatient after referral to hospital). In Lowres et al. (2014), oral anticoagulants were prescribed for 6 out of 10 new patients with atrial fibrillation detected by a lead‑I ECG followed by a 12‑lead ECG interpreted by a cardiologist.

Evidence on patient- and healthcare professional-reported outcomes

4.22 In Orchard et al. (2016), which used Kardia Mobile, patients and GPs commented that they liked using the device. Chan et al. (2017) reported that all patients asked were willing to have further testing with Kardia Mobile at future GP visits, and 86% of GPs surveyed considered that the device was useful for atrial fibrillation screening and they would use it in their daily practice. Gibson et al. (2017) reported generally positive responses to using MyDiagnostick, although some issues with implementing use of the device were raised. A further study reported that Kardia Mobile was easily administered and that no one declined testing with the device (Hussain and Thakrar 2016). In Chan et al. (2017), interviewed patients commented that having access to the lead-I ECG device in the surgery was more convenient than having to attend another healthcare facility for a 12‑lead ECG.

'Real world' data

4.23 The EAG also looked at unpublished evidence from a quality control audit on the use of Kardia Mobile across Eastbourne, Hailsham and Seaford clinical commissioning group and Hastings and Rother clinical commissioning group. This was provided by a specialist committee member as an example of an ongoing audit. Over a 2-year period the device was used in primary care or for home visits if people had an irregular pulse or signs of atrial fibrillation. There were 183 ECG traces reported, identifying 128 cases of atrial fibrillation from the lead‑I ECG trace alone. The proportion of people newly diagnosed with atrial fibrillation (69.9%) was considerably higher than the diagnostic yield in studies identified by the EAG (0.38% to 5.84%), although the audit was designed for quality control, and not to assess atrial fibrillation yield.

Model structure

4.26 The model compared the effect of using a lead‑I ECG device in primary care for people with signs and symptoms of atrial fibrillation who have an irregular pulse (detected by manual pulse palpation) with standard diagnostic testing (that is, without the use of a lead‑I ECG device). The model was in 2 phases: a diagnostic phase followed by a post-diagnostic phase.

Diagnostic phase

4.27 This phase covered the initial assessment of people presenting in primary care with signs and symptoms of atrial fibrillation, and who have had manual pulse palpation that shows an irregular pulse. The model compared 2 strategies: referral for a subsequent 12‑lead ECG to check for atrial fibrillation (standard diagnostic pathway) or having a lead‑I ECG in primary care at the same primary care appointment to check for atrial fibrillation (lead‑I ECG pathway) followed by a 12‑lead ECG if the clinician thought this was appropriate.

4.28 The diagnostic phase model covered the first 3 months after the initial primary care appointment. By the end of the diagnostic phase, people have either been diagnosed as having atrial fibrillation, or no atrial fibrillation has been detected (either correctly or incorrectly). People diagnosed with atrial fibrillation can have anticoagulants and rate control treatment (beta blockers).

4.29 People can have up to 2 cerebrovascular events (transient ischaemic attack, ischaemic or haemorrhagic stroke), a non-major bleeding event, or die. This was modelled using a Markov model. The probability of having a cerebrovascular event for people with atrial fibrillation is reduced if they are taking anticoagulants. However, anyone taking anticoagulants has an associated higher risk of having a bleeding event.

Post-diagnostic phase

4.30 After the 3-month diagnostic phase model, people entered a second Markov model. This had the same structure as the Markov model in the diagnostic phase after a diagnosis has been made, but ran over a 30‑year timespan (with 3‑month cycles). People entered based on their history of cerebrovascular events (none, 1 or 2) and they could have further cerebrovascular events, non‑major bleeding events, or die.

Diagnostic accuracy of lead-I ECG devices

4.32 Estimates of the diagnostic accuracy of the 4 lead‑I ECG devices were obtained from the EAG's systematic review and meta-analyses. The EAG used estimates of accuracy based on healthcare professionals interpreting the ECG traces, because it assumed that atrial fibrillation would not be diagnosed based on a device's algorithm alone.

Treatment effects: mortality and cerebrovascular events

4.33 For people with atrial fibrillation, the rate of mortality and cerebrovascular events (transient ischaemic attack, ischaemic or haemorrhagic stroke) in people who did not have anticoagulants was taken from Sterne et al. (2017). The effect of anticoagulants on the incidence of these events in people with atrial fibrillation was also taken from this study. For people without atrial fibrillation the rate of mortality and cerebrovascular events was taken from various sources (for example, Public Health England report, Office for National Statistics report, Rothwell et al. 2005). The risk of cerebrovascular events and mortality for people with untreated atrial fibrillation does not vary by type of atrial fibrillation. That is, risk is the same for paroxysmal, permanent and persistent atrial fibrillation. After people have a cerebrovascular event, their risk of mortality increases. The EAG assumed that this risk was 2.6 times greater based on a study of stroke survivors in Norway (Mathisen et al. 2016). The risk of having a further cerebrovascular event was based on a meta-analysis of stroke survivors (Mohan et al. 2011) with increased risk in the first year, then a lower risk from year 2 onwards.

Treatment effect: clinically relevant bleeding

4.34 The risk of clinically relevant bleeding is increased for people who have anticoagulants, based on Sterne et al. (2017). This is the case for people with or without atrial fibrillation.

Lead-I ECG device costs

4.35 Annual costs of the devices used in the base-case model are shown in table 6. Because the lead‑I ECG could be used outside the scope of this assessment, the EAG also did a scenario analysis that excluded the costs of the devices. No extra cost was included for administering and interpreting the lead‑I ECG because it was assumed that this could be done during a standard GP consultation.

Costs of 12-lead ECGs and Holter monitoring

4.36 The EAG devised base cases that differed depending on where 12‑lead ECGs were done. If a 12‑lead ECG was done in primary care, the cost of administering it was assumed to be £12.34. This was based on the costs of the device, disposables and staff time to do and interpret the ECG. The cost of administering a 12‑lead ECG in secondary care was assumed to be £52 (from NHS reference costs). The cost of Holter monitoring (for 7 days) was assumed to be £120.23.

Treatment and event costs

4.37 Costs for anticoagulant (apixaban) and rate control (beta blockers) treatment were obtained from the British national formulary and NHS drug tariff. Costs of bleeding events and transient ischaemic attack were taken from NHS reference costs. Age and sex-adjusted 1- and 5‑year costs for strokes were from the Sentinel Stroke National Audit Programme's cost and cost-effectiveness report (2016).

Health-related quality of life and QALY decrements

4.38 Berg et al. (2010) was used to provide utility values for people with atrial fibrillation (see table 7). Beta blockers were assumed to improve symptoms for people with atrial fibrillation.

Deterministic sensitivity analysis

4.45 The model was most sensitive to the proportion of patients whose atrial fibrillation was paroxysmal (assumed to be 50% in the base case) in one-way analyses for all of the lead-I ECG devices. Cost effectiveness improved as the proportion of paroxysmal atrial fibrillation increased. Conversely, lower estimates of the proportion of paroxysmal atrial fibrillation made the devices less cost effective (increased incremental costs and decreased incremental QALYs).

Probabilistic sensitivity analysis

4.46 In a probabilistic sensitivity analysis (done in base case 1) all other lead‑I ECG devices were dominated by Kardia Mobile in a fully incremental analysis. In pairwise comparisons with the standard pathway, ICERs were similar to the deterministic results, and all were less than £17,000 per QALY gained.