3 Approach to evidence generation
3.1 Ongoing studies
There are 4 ongoing studies assessing KardiaMobile 6L in cardiology settings. The generalisability of these to a psychiatric setting, in people having or about to have antipsychotic medication, is limited. No further ongoing studies that could address the evidence gaps for this evaluation have been identified.
3.2 Real-world data collections
Local or regional data collections, such as the sub-national secure data environments, which measure outcomes specified in the evidence generation plan, could be used to collect data that addresses the evidence gaps. NICE's real-world evidence framework provides detailed guidance on assessing the suitability of a real-world data source to answer a specific research question.
The NHS Mental Health Services Data Set (MHSDS) is a mandated national data collection that could potentially collect the necessary data. But it may not routinely collect all the outcome measures identified in the early value assessment. Also, there are potential issues with data quality and whether data on all people who are eligible are included. NHS England has suggested that modification of MHSDS may take up to 2 years, so it is unlikely that it could be modified in time to support data collection.
The evidence base for this evaluation included information from 2 Academic Health Science Network pilots that implemented this technology in 2 mental health trusts. Real-world data from local electronic health records of trusts implementing the technology, collected retrospectively, could provide information to supplement the clinical and economic evidence generated.
The quality and coverage of real-world data collections are of key importance when used in generating evidence. Active monitoring and follow up through a central coordinating point is an effective and viable approach of ensuring good-quality data with broad coverage.
3.3 Evidence generation plan
Diagnostic accuracy study
This could be done as a diagnostic cross-sectional study in people having or about to have antipsychotic medication in a psychiatric setting. The study should attempt to enrol a representative population, that is, people who would be expected to be offered the test in the real world.
The study would compare agreement between KardiaMobile 6L and 12-lead electrocardiogram (ECG) device results (the reference standard). When both tests are successfully done, it would be possible to report measures of accuracy for diagnosis of prolonged QT (including sensitivity, specificity, negative predictive values and positive predictive values) and concordance for QT length and QTc measurements. Test results should be interpreted without knowledge of the results of the other test (for both KardiaMobile 6L and 12-lead ECG device) and reflect use and interpretation by specialities or healthcare professionals in a real-world setting. Information should be provided on whether results were calculated manually or generated by the device.
A user preference survey embedded in the study could assess the preferences and acceptability of KardiaMobile 6L.
Real-world implementation study
This study could compare impact on services before and after implementing KardiaMobile 6L. Prospective data collection in the period before and after implementation of KardiaMobile 6L may be required to ensure sufficient data quality. Baseline characteristics should be reported for all people included in this study.
To adequately assess impact, it is important that the context and clinical pathway within which the technology is evaluated in the study reflects how it will be used in routine practice.
3.4 Data to be collected
The following information should be collected in the suggested studies:
-
time to complete the test and get the QT interval result (including set up, ECG recording, QT measurement and correction calculation, reporting time)
-
grade of NHS professional and different services involved in doing and interpreting the ECG when using KardiaMobile 6L and 12-lead ECG device
-
accuracy of the technology for diagnosing prolonged QT interval
-
concordance for QT length and QTc measurements between KardiaMobile 6L and 12-lead ECG device
-
number and proportion of technical failures (how often results were insufficient for decision making or needed to be repeated for both KardiaMobile 6L and 12-lead ECG device)
-
number and proportion of people who refused either test or for whom it was not possible to do the tests
-
number and proportion of people eligible for an ECG test (for example, having antipsychotic medication) who have one
-
number and proportion of people who have KardiaMobile 6L compared with 12-lead ECG device in the first instance
-
number of times the ECG was repeated using a 12-lead ECG device after using KardiaMobile 6L
-
among those who have not yet started treatment, time from initial referral for either test to antipsychotic prescription
-
treatment decision (to start medication, stop medication, reduce dose, switch antipsychotic medication or continue medication) after ECG result
-
training and implementation costs
-
information on baseline factors such as age, sex, medications, conditions needing antipsychotic medication, and comorbidities including any previously known cardiac abnormalities or heart disease; it should be clear whether the person has already started antipsychotic medication prior to ECG testing
-
specific antipsychotic medications should be reported; this could allow for subgroup analysis, for example, for medications that have a greater risk of QT prolongation
-
overall prevalence of prolonged QT in people having antipsychotic medication.