Evidence generation plan

3.1 Evidence gaps and ongoing studies

Table 1 summarises the evidence gaps and ongoing studies that might address them. Information about evidence status is derived from the external assessment group's reports; evidence not meeting the scope and inclusion criteria is not included.

There are ongoing studies for all technologies except Roczen and Weight Loss Clinic. But these studies may be limited in addressing the evidence gaps for the relevant population within specialist weight-management services in the UK.

Information on relevant ongoing studies shared by Liva and Oviva included reported outcomes in people having weight-management medicine and formed part of the evidence base (see table 1). Mature data from these studies could help the evaluation of clinical benefit in terms of weight loss and medicine adherence for these technologies.

3.2 Data sources

There are data collections with different strengths and weaknesses that could potentially support evidence generation. NICE's real-world evidence framework provides detailed guidance on assessing the suitability of a real-world data source to answer a specific research question.

NHS Digital's Community Services Data Set (CSDS) is a patient-level secondary uses dataset with clinical and operational data collected from publicly funded community services. It includes data from tier 2 and tier 3 weight-management service providers in England and could support evidence generation.

Patient‑level data from the CSDS can be linked to other datasets such as NHS Digital's Hospital Episode Statistics (HES) to support the evaluation of outcomes such as adverse events, further hospital appointments and referral to bariatric surgery. But all the outcomes needed to address the evidence gaps may not be available from the CSDS (for example, adherence data collected by the technology) and linking to other sources may not be possible. This may mean that additional data collection is needed.

The CSDS also informs the NHS National Obesity Audit. The audit data could provide supporting contextual information for evaluating cost effectiveness, such as current national uptake and accessibility of specialist weight-management services and assessment of changes over time.

The quality and coverage of real-world data collections are of key importance when used in generating evidence. Active monitoring and follow up through a central coordinating point is an effective and viable approach of ensuring good-quality data with broad coverage.

3.3 Evidence collection plan

To address the evidence gaps, a before and after implementation study is suggested. A before and after design allows for comparisons when there is considerable variation across services in the standards and mode of delivery of specialist weight-management programmes. The study should include people who have had a clinical assessment and are referred to specialist weight-management services for support. After an enrolment period, data should be collected separately for people having:

The data collection period should be long enough to ensure there is 1‑year follow‑up data for the standard care groups. The digital technology should then be implemented in the service and data collected from new people having support provided through the technologies, for the same groups defined above. In the second observation period, follow up for 2 years is preferable to show if any weight loss is sustained. The study should compare outcomes between those whose weight management is delivered through the digital technologies and those who had treatment with standard care before implementation of the technologies, for each group.

This study could be done at a single centre or ideally, replicated across multiple centres. This could show how the technology can be implemented across a range of services, representative of the variety in the NHS. Outcomes may reflect other changes that occur over time in the population, unrelated to the interventions. Additional robustness can be achieved by collecting data in a centre that has not implemented the technology but is as similar as possible (in terms of clinical practice and patient characteristics) to a site where the technology is being used. This could help control for changes over time that might have occurred anyway and could also enable 2‑year follow up in a standard care group.

High-quality data on patient characteristics is needed to correct for any important differences between comparison groups and to assess who the technologies would not be suitable for. Important confounding factors should be identified with input from clinical experts during protocol development. Loss to follow up should be reported, with reasons, over the data collection period. Differences between self-reported and clinically measured weight loss outcomes are also a potential source of bias, particularly if these vary between comparison groups, so a consistent measure should be used. It is essential that appropriate safeguarding and risk management processes are in place when generating evidence. The pathway must allow for clinical review before referral to the technology programme. Also, any safety issues and related adjustments to medicine during the intervention must be flagged to the appropriate teams for onward referral and investigation. Composition of the multidisciplinary team must be reported and should include psychological support.

Data collection should follow a predefined protocol and quality assurance processes should be put in place to ensure the integrity and consistency of data collection. See NICE's real-world evidence framework, which provides guidance on the planning, conduct, and reporting of real-world evidence studies. This document also provides best practice principles for robust design of real-world evidence when assessing comparative treatment effects.

3.4 Data to be collected

To address the evidence gaps, data should be collected on the following outcomes. For people who are taking weight-management medicine, baseline is when the medicine is prescribed. For people who are not taking weight-management medicine, baseline is when they are referred for specialist weight-management support:

3.5 Evidence generation period

This will be 4 years to allow for setting up, implementing the technologies, supplying and accessing medicine, data collection, analysis and reporting.