Digital technologies for assessing attention deficit hyperactivity disorder

The committee concluded that when used with standard clinical assessment by a healthcare professional, QbTest was likely to allow diagnostic decisions to be made quicker. The AQUA trial was the only randomised controlled trial identified in the EAG's review. It showed that a diagnostic decision had been made for a larger proportion of people within 6 months of baseline when QbTest results were available, compared with when they were not. The AQUA trial findings were supported by data from 5 before-and-after studies which found that using QbTest resulted in fewer consultations being needed to reach a diagnostic decision. Unfortunately, the largest implementation study (FOCUS) was severely affected by the COVID-19 pandemic and so was judged to be at serious risk of bias. The EAG also said that AQUA time-to-event outcomes were judged as being at a high risk of bias. This was because a large proportion of participants were censored from the analysis because they did not have a diagnosis at 6 months. This assumed that censored participants would have the same diagnosis rate as those for whom a diagnosis was reached at 6 months. But the EAG explained that this was not an issue for outcomes for people who had received a diagnosis in the study period. The committee noted that there could be variation in clinical practice for ADHD assessment across the NHS and was unsure whether the impact of QbTest would be the same everywhere. But it noted that the AQUA trial was done across 10 non-academic sites across the UK in both child and adolescent mental health services and community paediatric clinics, which provided some reassurance. Despite some limitations of the AQUA trial, the committee considered it was suitable for decision making. The committee recalled that the process of diagnosing ADHD is complex, and so valued having direct data on the impact of test use on outcomes related to the diagnostic process.

The committee also concluded that when used with standard clinical assessment, QbTest was likely to provide healthcare professionals with higher confidence in their decisions. The clinical experts noted that diagnosis could be particularly difficult when information was missing (for example, observer reports from school or family members) and the test could provide further information in these scenarios. The committee considered qualitative and quantitative evidence from the AQUA trial that reported an increase in confidence in diagnostic decisions when QbTest results were available. In addition, the AQUA trial reported that clinicians were able to rule out ADHD in more cases when using QbTest than when using standard assessment alone.

Most evidence identified by the EAG investigated the diagnostic accuracy of the technologies as a standalone tool, compared with standard clinical assessment. The EAG highlighted that estimates of accuracy evaluated in isolation were generally lower than when evaluated in combination with clinical judgement. The EAG stated that the AQUA trial was the only study to combine QbTest information with clinical assessment in the same way that it would be used in practice. The clinical experts were concerned that the tests could be used inappropriately in practice. That is, to make decisions about whether further assessment was needed, or even to make diagnostic decisions based on triage assessments, without appropriate healthcare professional input. The indications for use for the tests state that they should be used to supplement healthcare professional decision making, not to replace it. The committee emphasised that the standalone use of any of the technologies was not appropriate and not in line with their intended use.

The committee raised that the technologies may not be suitable for use in people with existing learning disabilities, visual impairments or physical disabilities. Technologies with wearable components such as a headband or headset may not be suitable for all people, such as those with anxiety and sensory difficulties associated with autism spectrum disorder.

The committee noted that although the AQUA trial showed low specificity when incorporating QbTest into clinical assessment, the standard assessment arm of the trial also showed low specificity. The EAG stated that this may have been because of the limited information available for the reference standard, which may have resulted in the diagnosis being too stringent, leading to an underestimate of specificity. The committee noted that the sensitivity to detect ADHD was lower when the QbTest result was available in the AQUA trial. Although the EAG commented that this difference was not statistically significant, this could be because the size of the population tested was not large enough. The EAG highlighted concerns about the accuracy data from the AQUA trial. This included that the diagnostic accuracy data was only from people who received a diagnosis, but that a large proportion of people did not receive any diagnostic decision in either arm of the study by 6 months. The committee concluded that there was uncertainty about the impact on accuracy of using the tests to detect ADHD. But it recalled that the tests should only be used to supplement healthcare professional judgement, not to replace it (see section 3.7). The EAG had also run scenario analyses in its economic model which reduced the diagnostic accuracy of the test compared with standard assessment alone. So, any possible negative impact of adding the test to clinical practice had been explored in the cost-effectiveness analyses.

The committee concluded that there was not enough evidence to support using the tests in adults. No studies were identified in the EAG's review that used any of the tests with standard clinical assessment in the diagnosis of ADHD in adults. The clinical experts stated that the data obtained from studies in children and young people was not appropriate to show how the tests would work when used for adults. This is because of differences in the clinical presentation and information available to make a diagnosis in adults compared with children and young people. For example, observer reports of behaviour are less commonly available for adults and there are more differential diagnoses. But the clinical experts did highlight the considerable potential benefits of the tests if they can help diagnose ADHD in adults. So, the committee decided that although there were potential benefits for the tests in adults, more research was needed.

The committee considered the targeted use of tests in complex cases. That is, for people for whom standard clinical assessment by a healthcare professional does not typically lead to a diagnosis. This may be because of co-existing conditions, some of which have a large overlap in traits with ADHD, such as autism spectrum disorder, trauma, mental health conditions, learning disorders or behavioural conditions. It may also be complex because there is substantial missing or conflicting information, or because of medication or substance use. It was noted that some centres already use QbTest in this way. The committee concluded that although there may be potential benefits in the targeted use of tests only in complex cases in which standard assessment could not reach a diagnosis, more data is needed.

The technologies under consideration are also indicated for evaluating treatment effectiveness for people diagnosed with ADHD who are having treatment. The EAG identified 1 accuracy study and 1 randomised controlled trial feasibility study evaluating QbTest in this population. The EAG considered both to be at high risk of bias. The committee concluded that there was insufficient evidence to recommend any technology for evaluating treatment effectiveness.

The committee concluded that there was insufficient evidence to assess the cost effectiveness of any technology other than QbTest to aid in the diagnosis of ADHD:

• No studies were identified for EFSim Test Web Version or Nesplora Attention Adults Aquarium.

• Studies were available for EFSim Test, Nesplora Aula and QbCheck, but the EAG highlighted that none of them used the test with standard clinical assessment in line with their intended use (see section 3.7).

• No studies for any of the technologies reported on the impact of the test on diagnostic process outcomes.
  
  The manufacturers of the EFSim and Nesplora technologies emphasised that their tests were considerably different in their mechanisms of action and outputs to QbTest. QbCheck, although it has the same outputs as QbTest, uses a different motor activity sensor, and is designed for remote use without the presence of a healthcare professional. The committee concluded that the data generated using QbTest, such as from the AQUA trial, was not generalisable to any other technologies in this evaluation and that there was limited data for the other technologies considered. So, the committee did not consider the potential cost effectiveness of the other technologies for detecting ADHD. The outcomes used by the EAG in modelling to assess the cost effectiveness of QbTest, such as the impact on waiting time and time to diagnosis, gave an indication of potentially relevant outcomes. Any assessment of test accuracy should be done as the tests are intended to be used; that is, alongside healthcare professional judgement.

The committee concluded that using QbTest alongside standard clinical assessment was likely to be a cost-effective use of NHS resources for assessing ADHD in children and young people. In the EAG's base case analysis, the QbTest strategy incremental cost-effectiveness ratio was £6,184 per quality-adjusted life year gained. QbTest was cost effective in almost all scenario analyses run by the EAG. The impact of QbTest was largely modelled using data from the AQUA trial, which the committee considered was suitable for decision making (see section 3.5). The committee concluded that QbTest was likely to be cost effective when used alongside standard clinical assessment by a healthcare professional to help diagnose ADHD for children and young people.

The committee also noted the importance of suitable training for healthcare professionals in interpreting the results of the test alongside other clinical information. Manufacturer submissions confirmed that initial training to administer and interpret QbTest reports is provided. This is supplemented by a 3-month update training, and annual training is also offered.

Because of the limited available data, the EAG did not provide cost-effectiveness estimates for using the technologies to help diagnose ADHD in adults. The committee recalled its conclusion that it was not appropriate to use data from studies of children and young people to show performance in adults (see section 3.10) and agreed that further evidence was needed for this population.

The committee was uncertain if targeted use of the tests in complex cases only (when standard assessment does not lead to a diagnosis) was likely to be cost effective compared with using the test in all ADHD assessments. Because of limited data for this population, the EAG was only able to explore the cost effectiveness of QbTest when used for complex cases by making the strong assumption that QbTest would only be used if a diagnosis was not made in 2 appointments (including the initial appointment). This analysis also made the strong assumption that the test impact on the diagnostic process and diagnostic accuracy would be the same in complex cases. The results of the exploratory analysis suggested that this strategy could be cost effective compared with testing everyone. The committee recalled that diagnosing ADHD is a complex process, and exactly how the tests affect this process could be variable (see section 3.5). The committee noted that further evidence on exactly how the tests are used in clinical decision making could be beneficial to help identify if there may be particular benefit for more complex cases.

The EAG did not provide cost-effectiveness estimates for using the tests to help evaluate treatment effectiveness for people with ADHD. It considered there was insufficient data to populate an economic model for dose titration or long-term monitoring. The committee agreed that further evidence is needed to show the impact of the tests on people with ADHD and the healthcare system when used by a healthcare professional to help make decisions about treatment.