Lead-I ECG devices for detecting symptomatic atrial fibrillation using single time point testing in primary care

Clinical effectiveness

5.3 The committee considered the studies included in the diagnostic accuracy review. It noted that the external assessment group (EAG) had concerns over the applicability of several of the studies because lead‑I ECG traces were interpreted by the device's algorithms alone, rather than by a trained healthcare professional. It noted that the companies stated that the algorithms alone should not be used to diagnose atrial fibrillation. Clinical experts highlighted the importance of having trained healthcare professionals review ECG traces generated by the lead‑I ECG devices. This is to confirm or exclude atrial fibrillation and to check any algorithm outputs, and therefore inform treatment decisions. The committee noted that the trained healthcare professionals interpreting the ECGs in the identified studies were generally cardiologists or electrophysiologists, who may be more experienced in interpreting ECG traces than GPs. In 1 study (Williams et al. 2015), in which the interpreter was a GP with a special interest in cardiology, specificity estimates were lower than those obtained when a cardiologist interpreted the trace. Also, accuracy estimates of the devices varied between the 2 electrophysiologists in Desteghe et al. (2017), suggesting that interpretation of the lead‑I ECG traces is likely to be subject to inter-observer variability. The committee concluded that it was important that decisions about treatment based on lead‑I ECG traces are made only after review by a trained healthcare professional, because this may have a substantial effect on false results.

5.4 The committee noted that the populations varied in the studies included in the EAG's diagnostic accuracy review. Most of the studies were done in people who did not report symptoms of atrial fibrillation, but who were attending cardiology services because of an underlying cardiac problem. It recalled that the EAG had highlighted this as a generalisability issue. The clinical experts explained that because the populations in the included studies tended to be older, the burden of atrial fibrillation would be expected to be greater than in a truly asymptomatic population. The committee considered that the absence of studies that were directly applicable to the population in this assessment was not ideal. But it concluded that the available studies provided a reasonable estimate of the ability of the devices to correctly identify atrial fibrillation.

5.5 The committee considered the diagnostic accuracy data that were available for each of the devices. It noted that 5 studies were available for Kardia Mobile, 3 for MyDiagnostick and 1 for Zenicor-ECG. The committee also noted that there was uncertainty about whether current versions of the algorithms had been used in the diagnostic accuracy studies for the lead‑I ECG devices. Most of the studies compared each of the devices with a 12‑lead ECG and did not include formal comparisons of the devices. There was 1 study (Desteghe et al. 2017) that assessed concordance between MyDiagnostick and Kardia Mobile and reported no statistically significant difference. The committee concluded that the available accuracy data were limited and were not sufficient to assess differences in accuracy between the lead-I ECG devices.

5.6 The committee considered the reference standard used in the identified diagnostic accuracy studies: a 12‑lead ECG done within about 6 hours of the lead‑I ECGs. It noted that the comparator for this assessment was a 12‑lead ECG done several days after the initial GP appointment where the irregular pulse was detected. The EAG identified no studies showing that lead‑I ECGs increased detection of atrial fibrillation when compared with 12‑lead ECGs done later after an irregular pulse was detected. It noted that studies identified by the EAG that reported diagnostic yield of atrial fibrillation were not done in a population who had symptoms, which is the focus of this assessment. The committee recalled that the potential value of the devices in this context was increased detection of atrial fibrillation, particularly paroxysmal, compared with a 12‑lead ECG done later (see section 5.2). It concluded that the identified data did not allow the committee to assess the likely clinical effect of the lead‑I ECG devices in increasing detection of atrial fibrillation compared with current practice (that is, a 12‑lead ECG done later).

Cost effectiveness

5.7 The committee considered the cost per use of the lead‑I ECG devices assumed in the model. It heard that the lifespan of MyDiagnostick was incorrect in the original report, but noted that the EAG had corrected this. The committee questioned the expected average number of people seen by a full-time GP per year that the EAG had used to estimate the cost per use of the devices, noting evidence from NHS Digital that the average number of people per GP is potentially higher. The EAG commented that its estimate was conservative and that if the average number of people per GP was higher this would reduce the cost per use of the devices and improve the cost-effectiveness estimates. The committee also questioned whether the model included the costs of training to use the device. The EAG explained that this was not explicitly included, but it had looked at the effect of increasing the costs of using the lead‑I ECG devices and the cost-effectiveness estimates were robust to increases in the costs per use. The committee concluded that, although there were uncertainties in the costs per use assumed in the model, they were not a key driver of the results.

5.8 The committee discussed the costs associated with interpreting the lead-I ECG traces in practice and considered whether these had been adequately captured in the model. It noted its conclusion that the ECG traces from the devices need to be interpreted by a trained healthcare professional to diagnose atrial fibrillation and make decisions about treatment (see section 5.3). The clinical experts explained that there is likely to be wide variation in the ability of GPs to interpret ECGs, and that some practices may use centralised services for this. The committee concluded that there was uncertainty about how lead‑I ECGs generated in primary care would be interpreted in practice, and therefore the effect on staff time and costs associated with introducing lead‑I ECGs into primary care. Further research was recommended to assess this (see section 6.2).

5.9 The committee considered the risk of bleeding associated with anticoagulant treatment, and noted that the model assumed that all patients have direct oral anticoagulants. It noted that people incorrectly identified as having atrial fibrillation by the lead‑I ECG devices in the model (false positive results) were assumed to have anticoagulants, and so were at risk of bleeding. The clinical experts explained that false positive results were likely to be caused by atrial ectopy, a benign condition that is not associated with an increased risk of stroke. They also commented that this group of people was likely to continue anticoagulants over the longer term, unless they chose to stop treatment. The committee questioned whether the risk of bleeding had been adequately captured in the analyses. The EAG explained that the model did allow for people to have bleeding events, and that a scenario analysis in the addendum including a quality-adjusted life year (QALY) decrement for minor bleeds had very similar results to the base-case analysis. The committee noted that the EAG's model did not account for any excess mortality in people who had a haemorrhagic stroke because of anticoagulants. The EAG commented that the increase in the number of bleeds in the model caused by adopting lead‑I ECGs was very small. The clinical experts commented that lead‑I ECG traces are reviewed by trained healthcare professionals, which helps to minimise the risk of false positive diagnoses. The committee concluded that there was some uncertainty about whether the model had captured all the adverse effects caused by anticoagulants.

5.10 The committee noted that the model was sensitive to an assumption about the proportion of cases of atrial fibrillation that are paroxysmal. The EAG explained that because of a lack of evidence this had been assumed to be 50% in the base case. The clinical experts commented that about 25% of atrial fibrillation is likely to be paroxysmal, and that the proportion in the modelled population is unlikely to be less than this. If the proportion of paroxysmal atrial fibrillation was set to 25% in the model, the incremental cost-effectiveness ratio (ICER) for Kardia Mobile compared with the standard pathway was about £7,500 per QALY gained, an increase from £1,060 per QALY gained in base case 1, in which it dominated the other lead‑I ECG devices. As the proportion of paroxysmal atrial fibrillation was decreased the ICER increased, to around £250,000 per QALY gained when the prevalence was set to 0. The committee concluded that because there were no data on the proportion of people with symptomatic atrial fibrillation that is paroxysmal the cost-effectiveness estimates were highly uncertain.

5.11 The EAG commented that most of the patient benefits in the model (from the use of the lead‑I ECG devices compared with the standard pathway) came from an estimated increase in detection of people with paroxysmal atrial fibrillation. However, the committee recalled that no clinical evidence had been identified that showed that lead‑I ECG devices increased the detection of people with atrial fibrillation compared with a later 12‑lead ECG in practice (see section 5.6). The EAG had made assumptions in the model to estimate the effect of the likely increase in detection of paroxysmal atrial fibrillation associated with the lead‑I ECG devices. However, because of a lack of data, it was unclear whether this increase would occur in clinical practice. The committee concluded that although there is plausible potential for the lead‑I ECG devices to be cost effective when used for single time point testing in primary care (for people with signs and symptoms of atrial fibrillation with an irregular pulse), there was insufficient evidence at present to determine if the predicted benefits of using the devices would be realised in practice. The committee considered that further research would help to address this (see section 6.1).

5.12 The committee considered the usability of the devices and noted that the EAG identified several studies reporting that the devices were easy to use and were liked by patients and healthcare professionals. However, it noted that 1 study (Desteghe et al.) reported that up to 7% of people were not able to use the devices because they were unable to hold them as recommended by the companies. A patient expert submitted comments that some people may need help in holding the devices while a recording is taken, for example people who have had a stroke or people with arthritis. The committee concluded that healthcare professionals should bear this in mind when using the devices and encouraged the companies to improve the usability of their devices for these groups of people.

5.13 The committee considered the results of the fully incremental economic analyses (see sections 4.43 and 4.46). It noted that all lead-I ECG devices were dominated by Kardia Mobile (that is, using the Kardia Mobile cost less but produced more QALYs). However, the committee recalled its earlier conclusion that the available accuracy data for the lead‑I ECG devices were limited and were not sufficient to assess differences in accuracy between the lead-I ECG devices (see section 5.5). It also noted that the Kardia Mobile did not dominate in all simulations in the probabilistic sensitivity analysis. The committee concluded that there was considerable uncertainty about the relative cost effectiveness of the different lead‑I ECG devices, and that a conclusion about which device was most cost effective could not be made from the available data.

Research considerations

5.14 The clinical experts explained that lead‑I ECG devices were increasingly being used in primary care settings. The committee noted that Academic Health Science Networks (AHSNs) are assessing the effect of introducing lead‑I ECG devices into primary and community care, although their project is broader than the scope of this assessment. The committee considered consultation responses on the AHSN project. It noted that data collected as part of the project may be relevant to the population covered by this guidance and could help answer some of the uncertainties identified on the system impact of adopting the devices (see section 6.2). Clinical experts explained their processes to ensure appropriate governance of patient information when using these devices to detect atrial fibrillation. The committee noted the importance of this and concluded that centres should ensure appropriate information governance is in place for these devices.

5.15 The committee heard that the focus of the AHSN project is to evaluate the extent of spread and adoption of the mobile ECG technology and to describe the optimum environment for implementing a national procured innovation. It is not an evaluation of the technology itself. The committee concluded that data collected as part of the AHSN project were unlikely to resolve uncertainty about the extent of any increased detection of atrial fibrillation by the devices compared with current practice (see section 6.1) and that further research would be needed to address this.