3 Approach to evidence generation

3.1 Evidence gaps and ongoing studies

Table 1 summarises the evidence gaps and ongoing studies that might address them. Information about evidence status is derived from the external assessment report. More information on the studies in the table can be found in the supporting documents.

The REINFORCE trial, MASTERY study and the MAYFLY study are collecting data that may address many of the evidence gaps (see section 3.3).

**Table 1 Evidence gaps and ongoing studies**
Evidence gap	Da Vinci SP	Da Vinci X and Xi	Hugo RAS system	Senhance Surgical System	Versius
Understanding the learning curve associated with RAS introduction	Limited evidence Ongoing study	No evidence Ongoing study	No evidence	Limited evidence	Limited evidence
Resource use	Limited evidence Ongoing study	No evidence Ongoing study	Limited evidence	No evidence	Limited evidence Ongoing study
Clinical impact of RAS	Limited evidence Ongoing study	Limited evidence Ongoing study	Limited evidence	Limited evidence	Limited evidence Ongoing study
Surgeon experience of RAS	Limited evidence	No evidence	No evidence	No evidence	Limited evidence

Abbreviations: RAS, robot-assisted surgery.

3.2 Data sources

There are several data collections that could potentially support evidence generation. NICE's real-world evidence framework provides detailed guidance on assessing the suitability of a real-world data source to answer a specific research question.

For uses of robot-assisted surgery (RAS) in cancer-related surgical procedures, the National Disease Registration Service (NDRS) provides the Cancer Outcomes and Services Dataset (COSD) as part of the National Cancer Registration and Analysis Service. COSD is a compiled dataset that includes all people diagnosed with or having cancer treatment in or funded by the NHS in England. It includes much of the data needed to address the evidence gaps, such as individual patient outcome data items, length of hospital stay, and the surgery type done for each surgical procedure (for example, conventional minimally invasive or open surgery).

COSD can be linked to other datasets such as NHS Digital's Hospital Episode Statistics (HES) dataset which contains details about admissions, outpatient appointments and historical A&E attendances at NHS hospitals in England. This combined dataset can be used to estimate resource use.

The quality and coverage of real-world data collections are of key importance when used in generating evidence. Active monitoring and follow up through a central coordinating point is an effective and viable approach of ensuring good-quality data with broad coverage.

3.3 Evidence collection plan

The REINFORCE, MASTERY and MAYFLY studies may deliver comparative evidence addressing the evidence gaps for resource use, clinical impact of RAS technologies and the learning curve associated with implementation of RAS technologies.

Technologies that are not involved in these studies may need to generate equivalent comparative evidence. Approaches to delivering this are described in the NICE's real-world evidence framework.

REINFORCE trial

The REINFORCE trial is investigating the impact of RAS as it is introduced to or scaled up across NHS hospitals. The primary outcome measures include outcomes at:

patient level (quality of life and complications)
surgeon or team level (covering precision or accuracy, and surgery-specific workload)
organisation level (equipment failure, standardisation of operative quality, overall economic or cost effectiveness)
population level (equity of access).

The trial has recruited patients across colorectal, gynaecology, upper gastrointestinal and urology soft tissue specialties from 16 NHS trusts in England and Wales. This study recruited sites that were procuring new RAS equipment and sites that were already using RAS but were considering a change in specialty or procedure using RAS methods. The study aims to recruit 2,560 participants and has an estimated completion date of December 2025.

MASTERY study

The MASTERY study is using robotic systems to measure progress in surgical training and quality of surgeries. The primary outcomes measure is surgical complication rate at day 30. Other outcome measures are:

total operating time per surgery
time taken by surgeon to complete each surgery 'cardinal step'
number of robotic-assisted surgeries carried out by surgeon before enrolment in study
number of patients with complete or incomplete surgical resection
number of patients with readmission at day 30
number of patients with adverse events reported at day 30.

The study has recruited patients who have agreed to RAS for colorectal tumours, ear, nose and throat tumours, gynaecological tumours, hepatobiliary tumours, lung tumours and prostate tumours. The study has recruited 500 participants across 15 NHS trusts in England and Scotland.

MAYFLY study

The MAYFLY study is evaluating the clinical, economic, and efficiency outcomes of using robotic systems for outpatient procedures in England. The study has invited all patients treated at Queen Alexandra Hospital robotic outpatient surgery unit to participate in the study. The study has a completion date of 30 September 2026.

Real-world observational study

Additional evidence generation is necessary to supplement the studies and collect information that shows the evidence from them is generalisable to other implementations of the technologies.

An approach to delivering this is through a real-world observational study in which applicable outcomes are collected for patients having RAS. For example, post-surgical complications and length of stay. This data could be used to strengthen evidence for the learning curve, resource use and clinical impact evidence gaps. This study could be done both in centres that are already using RAS and those that are making new implementations.

Where RAS is being used in people with cancer, the COSD could be a suitable data source to support this study. Retrospective data from the COSD could also be used to deliver long-term outcome information.

Qualitative survey

A qualitative survey may provide qualitative evidence on healthcare professional preferences. This study should include questions about healthcare professional experience using the technologies, their acceptability, and ideally, the ergonomic impact of these technologies to the surgeons.

Data collection should follow a predefined protocol and quality assurance processes should be put in place to ensure the integrity and consistency of data collection. See NICE's real-world evidence framework, which provides guidance on the planning, conduct and reporting of real-world cohort studies to assess comparative treatment effects.

3.4 Data to be collected

Study criteria

At recruitment, eligibility criteria for suitability of using RAS technologies and inclusion in the real-world study should be reported, along with a detailed description of the RAS technologies, their specific versions and the type of procedures in which the RAS technologies are used. The evidence should be technology specific. Companies can work with other stakeholders in this process to generate such evidence.

Patient information and outcomes

Conversion rates to open or standard minimally invasive surgery from RAS.
Length of hospital stay.
Clavien–Dindo score as a measure of surgical complications.
Peri- and postoperative complication frequency (ideally up to 12 months).
Ideally, EQ-5D-3L or EORTC QLQ-C30 at baseline and over follow up (ideally up to 12 months).
Return to normal activities.
Applicable condition-specific outcomes.
Revision surgery (ideally for a minimum of 12 months follow up).

Surgeon information and outcomes

Number and type of RAS surgeries performed.
Any prior experience of using any RAS technology before recruitment into the study.

Organisational information and outcomes

Rate of minimally invasive surgery compared with open surgery after robot-assisted surgery is introduced.
Number and type of procedures done at the hospital.
Hospital capacity and surgical waiting lists.
Patient readmission rates, ideally up to 12 months.
Number and type of RAS surgeries performed at the hospital.
Information on healthcare resource use and hospitalisation costs related to RAS, including:
- technology acquisition cost and how it has been paid for
- technology set-up costs including staff training and time costs
- technology maintenance and consumable (for example, surgical blades) costs.

Safety monitoring outcomes

Any adverse events arising from using RAS technologies.

3.5 Evidence generation period

The evidence generation period should be 3 years (during which a minimum of 12 months of follow-up data will be collected). This will be enough time to implement the evidence generation study, collect the necessary information and analyse the collected data.

What we do

Into practice

Who we are

Robot-assisted surgery for soft tissue procedures: early value assessment

Evidence generation plan for robot-assisted surgery for soft tissue procedures