Evidence generation plan
Closed for comments This consultation ended on at Request commenting lead permission
3 Approach to evidence generation
3.1 Evidence gaps and ongoing studies
Most technologies do not have ongoing studies that will address the evidence gaps. However, Wellmind Health and getUBetter are doing comparative studies, which may address some of the evidence gaps and will complete within the evidence generation period.
Table 1 summarises the evidence gaps and the existing evidence for each technology. Information about evidence status is derived from the external assessment group's report; evidence not meeting the scope and inclusion criteria is not included.
Technologies and evidence gaps | Musculoskeletal disability and quality of life | Adherence | Healthcare resource use | Placement of technology in the clinical pathway |
|---|---|---|---|---|
Kaia (Kaia Health) | Limited information available | Limited information available | Limited information available | No relevant evidence identified |
Hinge (Hinge Health) | Limited information available | Limited information available | No relevant evidence identified | No relevant evidence identified |
SelfBACK (SelfBack Consortium) | Limited information available | Limited information available | No relevant evidence identified | No relevant evidence identified |
SupportBack (University of Southampton) | Limited information available | Limited information available | No relevant evidence identified | No relevant evidence identified |
ACT for PAIN (Pain Medicine Specialist Ltd) | No relevant evidence identified | No relevant evidence identified | No relevant evidence identified | No relevant evidence identified |
getUBetter (getUbetter Ltd) | No relevant evidence identified Ongoing study | No relevant evidence identified Ongoing study | No relevant evidence identified Ongoing study | No relevant evidence identified |
Pathway through Pain (Wellmind Health) | No relevant evidence identified | No relevant evidence identified Ongoing study | No relevant evidence identified Ongoing study | No relevant evidence identified |
3.2 Data sources
This topic is likely to need primary data collection for certain outcomes, information is also needed from primary and secondary care services. Data collection can be supported using routinely collected data.
Local or regional data collections such as the subnational secure data environments (SDEs) could be used to collect data to address the evidence gaps. SDEs are data storage and access platforms that bring together many sources of data, such as from primary and secondary care, to enable research and analysis. Subnational SDEs are designed to be agile and can be modified to suit the needs of new projects, for example, helping to collect pain scores and quality-of-life data. The data environment of an SDE supports linkage and may allow researchers a more comprehensive view of medical history, diagnoses, treatments, and outcomes related to low back pain as well as resource use. The West Midlands subnational SDE specialises in collecting data about musculoskeletal health and may be particularly well suited.
It is possible that some data may be generated through the technologies themselves, such as starting treatment and engagement outcomes. This data can be integrated with other data collected.
The quality and coverage of real-world data collections are of key importance when used in generating evidence. NICE's real-world evidence framework provides detailed guidance on assessing the suitability of a real-world data source to answer research questions. Active monitoring and follow up through a central coordinating point is an effective and viable approach of ensuring good-quality data with high coverage.
3.3 Evidence collection plan
A mixed methods approach is suggested to address the evidence gaps, for example, a prospective cohort or a before and after study in combination with a qualitative survey. This study should include people with low back pain who would be expected to be offered the technology in the real world and should compare the use of the technologies alongside standard care with standard care alone, with an embedded qualitative study.
The technologies may be applied at different points in the care pathway, for example, primary care or alongside specialist services such as physiotherapy. Depending on setting, different experimental designs may offer greater robustness.
Where there is limited variation and a well-defined standard of care across services then a prospective comparative cohort study is suggested
Where there is considerable variation between services a before and after study is suggested.
Data collection should follow a predefined protocol. Quality assurance processes should be put in place to ensure the integrity and consistency of data collection. See NICE's real-world evidence framework, which provides guidance on the planning, conduct, and reporting of real-world evidence studies. This document also provides best practice principles for robust design of real-world evidence when assessing comparative treatment effects.
Prospective comparative cohort study
In this study, data should be collected from healthcare services where the technology is offered for low back pain. It should be compared with other similar services where the technology is not offered. It is important that people in both services are included and followed up from the point at which they would be offered the technology, this should be in line with the intended use of the technology in the clinical pathway.
High-quality data on patient characteristics is needed to correct for any important differences between comparison groups. Important confounding factors should be identified with input from clinical experts during protocol development.
Before and after study
In a before and after study, data is collected and compared before and after implementing the intervention in the same setting. This ensures that service performance and configuration are accounted for in the final outcomes and so, reduces the risk of bias.
In this study design, after an enrolment period, data should be collected for people in the period prior to the implementation of the technology. The data collection period should be long enough to ensure there is sufficient follow up data for the standard care group. The digital technology should then be implemented in the service and data should then be collected from people receiving care through the technologies.
Developers should provide clear descriptions of the services and settings in which the study is done. This study could be done at a single centre or ideally, replicated across multiple centres to show how the technology can be implemented across a range of services, representative of the variety in the NHS. Outcomes may reflect other changes that occur over time in the population, unrelated to the interventions. Additional robustness could be achieved by collecting data in a centre which has not implemented the technology but is as similar as possible (in terms of clinical practice and patient characteristics) to a site in which the technology is being used. This could control for changes over time that might have occurred anyway.
Qualitative survey
Feedback should be collected through a survey or structured interviews with people using the technologies.
Robustness of survey results depend on comprehensive distribution across people who are eligible and on the sample of respondents being representative of the population of potential users.
3.4 Data to be collected
Prospective comparative cohort / before and after study
The following information should be collected:
Eligibility criteria (e.g., the indication for referral) and point of starting follow up. These should be consistent between comparison groups.
Type of low back pain at baseline and how this was defined (that is, acute or chronic – for example, chronic back pain may be defined as symptoms for 3 months or longer)
Type of healthcare professional referring patient to the technology
Musculoskeletal Health Questionnaire (MSK‑HQ) at baseline, 30 days, and 6 and 12 months
Ideally, EQ‑5D‑5L at baseline, 30 days, and 6 and 12 months
use of the technology including:
number of people offered the technology
number and proportion referred
number and proportion who started using the technology
engagement at 30 days and over time
number and proportion who stopped by 30 days
reasons why people stop using the technologies (for example, whether engagement stopped because of improvements in symptoms, lack of improvement or other reasons)
The following information on healthcare resource use should be collected at 30 days, and 6 and 12 months:
number of GP appointments per patient
number of physiotherapist appointments per patient
number of visits to the emergency department per patient
Additional prespecified resource use outcomes could focus on areas in which the technologies are expected to have the most benefit, for example:
number of occupational therapist appointments
number of cognitive behavioural therapy (CBT) or acceptance and commitment therapy (ACT) sessions
number of secondary care appointments
number of people starting or stopping pharmacological treatment
number of people with imaging referrals.
patient characteristics at baseline, including important confounders such as comorbidities and concomitant interventions, or other characteristics that may be related to likelihood of choosing to access the technology, for example, socioeconomic status, language or ethnicity
Qualitative survey study
Outcomes to be collected from people with the condition:
Perceptions from patients about whether the technologies help alleviate pain and return to normal activity
Satisfaction, engagement, and accessibility of the technology including barriers encountered by people using and continuing to use the technologies
Information about the technologies
Information about how the technologies were developed
Information about any updates to the technologies.
See the NICE Evidence Standards Framework for guidance.
3.5 Evidence generation period
This will be about 3 years to allow for setting up, implementation, data collection, analysis, and reporting.
How are you taking part in this consultation?
You will not be able to change how you comment later.
You must be signed in to answer questions
Question on Document