Evidence generation plan

Comment on section: 3.1 Evidence gaps and ongoing studies3.1 Evidence gaps and ongoing studies

Table 1 summarises the evidence gaps and ongoing studies that might address them. Information about evidence status is derived from the external assessment group's (EAG) report; evidence not meeting the scope and inclusion criteria is not included. The table shows the evidence available to the committee when the guidance was published. The EAG did not identify any ongoing studies that may address the evidence gaps.

Table 1 Evidence gaps and ongoing studies

Comment on section: 3.2 Data sources3.2 Data sources

Data could be collected using a combination of primary data collection, suitable real-world data sources, and data collected through the technology itself (for example, engagement data).

NICE's real-world evidence framework provides detailed guidance on assessing the suitability of a real-world data source to answer a specific research question.

The NHS England Secure Data Environment (SDE) service could potentially support this research. This platform provides access to high-standard NHS health and social care data that can be used for research and analysis. SDEs are data storage and access platforms that bring together many sources of data, such as from primary and secondary care, to enable research and analysis. They could be used to collect data to address the evidence gaps. The sub-national SDEs are designed to be agile and can be modified to suit the needs of new projects. Within an SDE, the data may be linked to other useful data such as that from primary care and could provide information on important confounders (for example, comorbidities).

The NHS Digital's Improving Access to Psychological Therapies data set (IAPT) and Mental Health Services Data Set (MHSD) are real-world data sets that could also be used to collect information about the impact that disorders have on mental health.

The quality and coverage of real-world data collections are of key importance when used in generating evidence. Active monitoring and follow up through a central coordinating point is an effective and viable approach to ensuring good-quality data with broad coverage.

Comment on section: 3.3 Evidence collection plan3.3 Evidence collection plan

The suggested approach to addressing the evidence gaps for Neupulse is a longitudinal, parallel cohort study over 12 months. The study will follow 2 groups of people (intervention arm and a control arm) over 3 months (ideally 6 months), and compare their outcomes.

The studies should enrol a representative population, that is, people who would be offered standard care, including behavioural therapy, without digital technologies. This may include face-to-face appointments and monitoring. The studies should compare people with tic disorders or Tourette syndrome using digital technologies for self-management with a similar group having standard care. Eligibility for inclusion, and the point of starting follow up, should be clearly defined and consistent across comparison groups to avoid selection bias.

Data should be collected in all groups from the point at which a person would become eligible for standard care. The data from both the intervention and comparison groups should be collected at appropriate time intervals and up to a minimum of 3 months. Data from people in different centres, with comparable standard care and patient population, but no access to digital technologies for self-management, should form the comparison group. Ideally, the studies should be run across multiple centres, aiming to recruit centres that represent the variety of care pathways in the NHS.

Despite consistent eligibility criteria, non-random assignment to interventions can lead to confounding bias, complicating interpretation of the treatment effect. So, approaches should be used that balance confounding factors across comparison groups, for example, using propensity score methods. To achieve this robustly, data collection will need to include prognostic factors related both to the intervention delivered and patient outcomes. These should be defined with input from clinical specialists. Also, analysis should be stratified according to the severity of the tic disorder. Incomplete records and demographically imbalanced groups can lead to bias if unaccounted for. Data collection should follow a predefined protocol and quality assurance processes should be put in place to ensure the integrity and consistency of data collection. See NICE's real-world evidence framework, which provides guidance on the planning, conduct, and reporting of real-world evidence studies. This document also provides best practice principles for robust design of real-world evidence when assessing comparative treatment effects using a prospective cohort study design.

Comment on section: 3.4 Data to be collected3.4 Data to be collected

Comment on section: 3.5 Evidence generation period3.5 Evidence generation period

The evidence generation period should be 2 years (during which a minimum of 1 year of follow-up data will be collected). This will be enough time to implement the evidence generation study, collect the necessary information and analyse the collected data.