3 Approach to evidence generation
3.1 Evidence gaps and ongoing studies
Table 1 summarises the evidence gaps and ongoing studies that might address them. Information about evidence status is derived from the external assessment group's report. More information on the studies in the table can be found in the supporting documents.
| Evidence gap | myCOPD |
|---|---|
|
Effect on exercise capacity using a validated measure |
Limited evidence |
|
Adverse events |
Good evidence Ongoing study |
|
Comparison with current practice |
No evidence |
|
Resource use |
Limited evidence Ongoing study |
|
Position in the care pathway |
No evidence |
|
Engagement and adherence |
Good evidence |
|
Health-related quality of life |
Good evidence Ongoing study |
|
Effectiveness in different subgroups |
No evidence |
3.2 Data sources
There are several data collections that could potentially support evidence generation. NICE's real-world evidence framework provides detailed guidance on assessing the suitability of a real-world data source to answer a specific research question.
The National Respiratory Audit Programme (NRAP) is a clinical audit dataset for people with respiratory disease (including COPD). It collects information about people referred from primary care. It includes much of the data needed to address the evidence gaps, such as face-to-face pulmonary rehabilitation use, exercise capacity outcome measures and EQ‑5D data. NRAP can be linked to other datasets such as the Hospital Episode Statistics dataset, and this combined dataset used to estimate resource use. Some people with COPD who need pulmonary rehabilitation may have treatment solely in primary care, so data about these people is not recorded in NRAP. The dataset can be quickly and easily amended to support additional data collection where necessary.
The Clinical Practice Research Datalink (CPRD) and The Health Improvement Network (THIN) systems are primary care databases of anonymised medical records from general medical practitioners. They can be linked to secondary care data and may be able to provide data that will help address the evidence gaps. They are not national, and modification to add new data fields is unlikely to be possible.
The quality and coverage of real-world data collections are of key importance when used in generating evidence. Active monitoring and follow up through a central coordinating point is an effective and viable approach of ensuring good-quality data with broad coverage.
3.3 Evidence collection plan
Use-case survey across services
Clinical leads would be contacted to understand how the technology is currently used across the NHS and in what settings. The survey should also ask how the technology could be optimised or used in different positions in the clinical pathway. This should include, when and where assessments are done to determine if people are eligible for the technology, eligibility criteria for using the technology and who refers people to use the technology. This will help to inform future evidence generation and study design.
Real-world prospective cohort studies
Prospective controlled cohort studies are the proposed approach to addressing the evidence gaps. The studies should enrol a representative population, that is, people who would be expected to be offered face-to-face pulmonary rehabilitation in the real world but cannot have it or do not want it. The studies should compare people with COPD having digital pulmonary rehabilitation with a similar group having an appropriate comparator, such as face-to-face pulmonary rehabilitation, wait list or no treatment. Eligibility for inclusion should be clearly defined and consistent across comparison groups.
Data should be collected in all groups from the point at which a person would become eligible for pulmonary rehabilitation. The data from both the intervention and comparison groups should be collected at appropriate time intervals and up to 12 months. Data from people in different centres, with comparable standard care and patient population, but no access to digital pulmonary rehabilitation, could form the comparison group. Ideally, the studies should be run across multiple centres, aiming to recruit centres that represent the variety of care in the NHS.
Non-random assignment to interventions can lead to confounding bias, complicating interpretation of the treatment effect. So, approaches should be used that avoid selection bias and balance confounding factors across comparison groups. For example, using matching or adjustment approaches such as propensity score methods. To achieve this robustly, data collection will need to include prognostic factors related both to the intervention delivered and patient outcomes. These should be defined with input from clinical specialists.
Data could be collected using a combination of primary data collection, suitable real-world data sources, and data collected through the technology itself (for example, engagement data).
Data collection should follow a predefined protocol, and quality assurance processes should be put in place to ensure the integrity and consistency of data collection. See NICE's real-world evidence framework, which provides guidance on the planning, conduct, and reporting of real-world cohort studies to assess comparative treatment effects.
3.4 Data to be collected
Study criteria
-
At recruitment, eligibility criteria for suitability of using the digital technology and inclusion in the real-world study should be reported, and should include:
-
a clinical diagnosis of COPD
-
position of the technology in the clinical pathway, and
-
the point that follow-up starts.
-
-
Description of the standard care offered.
Baseline information and outcomes
-
Exercise capacity measurement using either the 6‑minute walk test or the incremental shuttle walking test at baseline and over follow up (up to 12 months).
-
Changes in COPD symptoms, including exacerbation rates, at baseline and over follow up (up to 12 months).
-
Information on healthcare resource use and exacerbation-related hospitalisation costs, including emergency department visits, hospital admissions, length of stay, and GP visits.
-
Costs of digital pulmonary rehabilitation, including:
-
licence fees
-
use and implementation of the technology
-
healthcare professional staff and training costs
-
integration with NHS systems.
-
-
EQ‑5D at baseline and over follow up (up to 12 months).
-
Any adverse events arising from using digital pulmonary rehabilitation.
-
Access and uptake, including the number and proportion of people who were able to access digital pulmonary rehabilitation, from the broader population needing pulmonary rehabilitation.
-
Engagement with and information about stopping digital technology for pulmonary rehabilitation, including reasons for stopping. This should include:
-
the number of people starting digital pulmonary rehabilitation
-
engagement
-
the number of people who finish the digital pulmonary rehabilitation course, and
-
reasons for stopping (for example, because of improvements in symptoms, adverse effects, or other reasons).
-
-
Information about individual characteristics at baseline, for example, sex, age, ethnicity, clinical diagnosis, medicines (including supplemental oxygen use), comorbidities and past medical history (including time since diagnosis and recent hospitalisations), urban or rural location. Other important covariates should be chosen with input from clinical specialists.
3.5 Evidence generation period
The evidence generation period should be 3 years. This will be enough time to implement the evidence generation study, collect the necessary information and analyse the collected data.