How are you taking part in this consultation?

You will not be able to change how you comment later.

You must be signed in to answer questions

  • Question on Consultation

    Has all of the relevant evidence been taken into account?

  • Question on Consultation

    Are the summaries of clinical and and cost effectiveness reasonable interpretations of the evidence?

  • Question on Consultation

    Are the recommendations sound and a suitable basis for guidance to the NHS?

  • Question on Consultation

    Are there any equality issues that need special consideration and are not covered in the medical technology consultation document?

  • Question on Document

    Are there any other relevant ongoing studies that address the evidence gaps?
The content on this page is not current guidance and is only for the purposes of the consultation process.

Evidence generation plan

Closed for comments This consultation ended on at   Request commenting lead permission

3 Approach to evidence generation

3.1 Evidence gaps and ongoing studies

The external assessment group (EAG) did not identify any ongoing studies that may address the evidence gaps. The committee was made aware of an ongoing Online Remote Behavioural Intervention for Tics (ORBIT) study.

Table 1 summarises the evidence gaps. Information about evidence status is derived from the EAG's report; evidence that does not meet the scope and inclusion criteria is not included. The table shows the evidence available to the committee when the guidance was published.

Table 1 Evidence gaps and ongoing studies

Evidence gap

ORBIT

Impact of ORBIT on people's symptoms and health-related quality of life

Limited evidence

Ongoing study

Longer-term data on the clinical impact of ORBIT

Limited evidence

Ongoing study

Resource use

Limited evidence

Ongoing study

Clinical and cost effectiveness in different subgroups

Limited evidence

Ongoing study

3.2 Data sources

Data could be collected using a combination of primary data collection, suitable real-world data sources, and data collected through the technology itself (for example, engagement data).

NICE's real-world evidence framework provides detailed guidance on assessing the suitability of a real-world data source to answer a specific research question.

The NHS England Secure Data Environment (SDE) service could potentially support this research. This platform provides access to high-standard NHS health and social care data that can be used for research and analysis. There are also sub-national SDEs that are designed to be agile and potentially modified to suit the needs of new projects. SDEs are data storage and access platforms that bring together many sources of data, such as from primary and secondary care, to enable research and analysis. They could be used to collect data to address the evidence gaps.

The NHS Digital's Improving Access to Psychological Therapies data set (IAPT) and Mental Health Services Data Set (MHSD) are real-world data sets that could potentially provide information about the impact that tic disorders have on mental health.

Some unreported data may be obtained from the UK ORBIT randomised controlled trial, such as further data on tic severity.

The quality and coverage of real-world data collections are of key importance when used in generating evidence. Active monitoring and follow up through a central coordinating point is an effective and viable approach to ensuring good-quality data with broad coverage.

3.3 Evidence collection plan

The suggested approach to addressing the evidence gaps for ORBIT is using mixed-methods. Quantitative data can be collected through a self-controlled study and qualitative information through interviews or surveys of people who have used the technology.

In a self-controlled study, each participant is compared with themselves at different time points. In this study those time points should be at baseline, 3 and 6 months. Further follow up to 24 months would be ideal.

The study should enrol a representative population; that is, people who would be offered standard care, including behavioural therapy, without digital technologies. This may include face-to-face appointments and monitoring. Eligibility for inclusion, and the point of starting follow up should be clearly defined and consistent during the study, to minimise selection bias.

Data should be collected from the point at which a person would become eligible for standard care. The study should also capture information on people who were eligible but chose not to use the technology or could not access it. Ideally, the studies should be run across multiple centres, aiming to recruit centres that represent the variety of care pathways in the NHS.

Incomplete records and potentially demographically imbalanced selection can lead to bias if unaccounted for. Data collection should follow a predefined protocol and quality assurance processes should be put in place to ensure the integrity and consistency of data collection. See NICE's real-world evidence framework, which provides guidance on the planning, conduct, and reporting of real-world evidence studies.

3.4 Data to be collected

Study criteria

  • At recruitment, eligibility criteria for suitability of using the digital technology and inclusion in the real-world study should be reported, and include:

    • a clinical diagnosis of tic disorder or Tourette's syndrome

    • position of the technology in the clinical pathway

    • the point that follow up starts.

Baseline information and patient outcomes

  • Information about individual characteristics at baseline, for example, sex, age, ethnicity, socioeconomic status, clinical diagnosis (and date of diagnosis), details of any comorbidities and treatments. Other important covariates should be chosen with input from clinical specialists to support subgroup analysis.

  • Changes in tic severity using the Yale Global Tic Severity Rating Scale total score at baseline and over follow up (for a minimum of 3 and 6 months).

  • Changes in patient quality of life using the Gilles de la Tourette Syndrome-Quality of Life Scale at baseline and over follow up (for a minimum of 3 months).

  • Qualitative information about the impact of ORBIT on daily life, ideally including self-esteem, social interactions and school or work attendance and performance.

  • Information on healthcare resource use and exacerbation-related hospitalisation costs related to tic disorders and Tourette syndrome. This should include emergency department visits, hospital admissions, length of stay, and GP visits. Changes in a person's medication and any referrals should be captured within ORBIT's escalation processes.

Implementation

  • Costs of digital technologies for supporting treatment of tic disorders and Tourette syndrome, including licence fees, healthcare professional staff time and training costs to support the service and costs of integration with NHS systems

  • Access and uptake, including the number and proportion of eligible people who were able to, or accepted an offer to, access the technology, and reasons for refusal

  • Engagement (including number of chapters completed and time to complete them, and ideally any revisits to the platform after the initial intervention) and drop-out information, including reasons for stopping.

Safety monitoring outcomes

  • Any adverse events arising from using digital technologies to support treatment of tic disorders and Tourette syndrome.

Data collection should follow a predefined protocol and quality assurance processes should be put in place to ensure the integrity and consistency of data collection. See NICE's real-world evidence framework, which provides guidance on the planning, conduct, and reporting of real-world evidence studies.

3.5 Evidence generation period

The evidence generation period should be 2 years. This will be enough time to implement the evidence generation study, collect the necessary information and analyse the collected data.