Robot-assisted surgery for orthopaedic procedures: early value assessment

  • Health technology evaluation
  • HTE22
  • Published: 

Register as a stakeholder

Download (PDF)

Evidence generation plan for robot-assisted surgery for orthopaedic procedures

3 Approach to evidence generation

3.1 Evidence gaps and ongoing studies

Table 1 summarises the evidence gaps and the evidence available to the committee when the guidance was published. Information about evidence status is derived from the external assessment report. Evidence that did not meet the scope and inclusion criteria is not included.

REINFORCE trial

The REINFORCE trial is investigating the impact of robot-assisted surgery (RAS) as it is introduced and scaled up across NHS hospitals. The primary outcome measures include outcomes at:

  • patient level, such as:

    • quality of life and

    • complications

  • surgeon or team level, such as:

    • precision or accuracy and

    • surgery-specific workload

  • organisation level, such as:

    • equipment failure

    • standardisation of operative quality and

    • overall economic or cost effectiveness

  • population level, such as equity of access.

The study aims to recruit 2,560 participants and has an estimated completion date of April 2025.

RACER-Knee and RACER-Hip

The RACER-Knee and the RACER-Hip trials are investigating the clinical and cost effectiveness of knee and hip replacement surgery (respectively) of RAS using the Mako SmartRobotics platform, compared with conventional surgery. The studies are likely to collect data on many of the evidence gaps, but they include a 10‑year follow up and are anticipated to end in 2032 (RACER-Knee) and 2033 (RACER-Hip). Interim data may be available before then (12‑month follow up completes in 2024).

Table 1 Evidence gaps and ongoing studies
Technology (procedure) Impact on people's quality of life Resource use Clinical impact in different subgroups

ApolloKnee System (total knee arthroplasty)

No evidence

Limited evidence

No evidence

CORI Surgical System (total knee arthroplasty)

Limited evidence

Limited evidence

No evidence

Mako SmartRobotics (total knee arthroplasty)

Good evidence

Ongoing study

Limited evidence

Ongoing study

No evidence

Ongoing study

ROSA Knee Solution (total knee arthroplasty)

Limited evidence

Limited evidence

No evidence

SkyWalker Robotic-assisted technology (total knee arthroplasty)

Limited evidence

Limited evidence

No evidence

VELYS Robotic-Assisted Solution (total knee arthroplasty)

Limited evidence

Limited evidence

No evidence

Mako SmartRobotics (partial knee arthroplasty)

Good evidence

Limited evidence

No evidence

CORI Surgical System (partial knee arthroplasty)

No evidence

Limited evidence

No evidence

Mako SmartRobotics (total hip arthroplasty)

Limited evidence

Limited evidence

Ongoing

CORI SmartRobotics (total hip arthroplasty)

No evidence

No evidence

No evidence

ROSAKnee, ApolloKnee, SkyWalker Robotic-assisted technology and VELYS are not indicated for use in partial knee arthroplasty or total hip arthroplasty.

3.2 Data sources

Data could be collected using a combination of suitable real-world data sources and primary data collection. NICE's real-world evidence framework provides detailed guidance on assessing the suitability of a real-world data source to answer a specific research question.

The National Joint Registry (NJR) is the data source that is most likely to be able to collect the real-world data necessary to address the essential evidence gaps. The registry includes everyone having joint replacement surgery (conventional or robot-assisted) across private healthcare settings and in the NHS. The registry also records the specific robotic systems used, and links to the NHS Personal Demographics Service to get data for revision surgery and mortality outcomes.

NHS England's national patient-reported outcome measures (PROMs) programme records PROMs before and 6 months after surgery. The relevant PROMs measured for joint replacement include the EuroQol 5D (EQ‑5D) 3L index score, Oxford Hip Score and Oxford Knee Score. Patient-level data from the NJR can be linked to other datasets such as NHS Digital's Hospital Episode Statistics. This could support the evaluation of outcomes such as adverse events, further hospital appointments and referral for physiotherapy.

Combining these real-world evidence data sources will address most of the evidence gaps around resource use and the impact on people's quality of life. The high-quality data and broad coverage within the NJR should enable relevant subgroup analyses to assess who the technologies might benefit.

The addition of outcomes to the registry is also unlikely within the timeframe of an early value assessment. Outcomes not already collected will need to be collected separately, for example, in a prospective audit.

The quality and coverage of real-world data collections are of key importance when used in generating evidence. Active monitoring and follow up through a central coordinating point is an effective and viable approach for ensuring good-quality data with broad coverage.

3.3 Evidence collection plan

Most of the evidence gaps can be addressed through a real-world historical control study. For evidence gaps not addressed by the real-world evidence datasets, a prospective audit is proposed to collect data on the impact of RAS on surgical capacity.

Real-world historical control study with propensity score methods

A historical control study could compare outcomes before and after the implementation of RAS. This could assess the clinical impact of RAS as well as resource use associated with RAS, such as:

  • volume of procedures and RAS uptake

  • hospital stays

  • readmission

  • revision rates and

  • use of other associated services.

The NJR has data on RAS from March 2020 onwards, but this could enable collection of longer-term data such as revision rates (ideally up to 5 years). Hospital location data could inform evidence gaps around geographical access to RAS. Data on the location of RAS systems may be available through the National Equipment Tracking and Inventory System, which aims to provide visibility around equipment assets. Collection of other baseline patient characteristics such as sex, age, gender, body mass index (BMI) and ethnicity will enable relevant subgroup analyses. These baseline cohort differences may affect clinical outcomes and should be corrected for in future analyses.

Despite consistent eligibility criteria, non-random assignment to interventions can lead to confounding bias, complicating interpretation of the intervention effect. To minimise bias and identify a suitable control group, appropriate statistical approaches that balance confounding factors across comparison groups should be used, for example, using propensity score matching. The comparator group of primary interest is conventional surgery using manual techniques. NICE's real-world evidence framework provides further detailed guidance on the planning, conduct and reporting of real-world evidence studies assessing comparative effects.

Prospective audit

Some of the evidence gaps around resource use will not be captured by the historical control study. For example, surgical time and total theatre time, or volume and cost of surgical consumables. An audit to collect data on the impact of RAS on surgical capacity is proposed to address these gaps. Technical failure rates should also be reported.

3.4 Data to be collected

Technologies

  • A detailed description of the RAS technologies and the specific versions.

Patient characteristics and outcomes

  • Information about individual characteristics at baseline, for example, age, sex, gender, BMI, ethnicity, surgery indication, and where in the country the operation was done. Characteristics should include those needed for adjustment to address confounding, and for subgroup analysis.

  • Patient pain, mobility and functioning PROMs at baseline and post-surgery. Currently PROMs linked to the NJR are collected before surgery and 6 months after. Ideally, this information would also be collected at 12 and 18 months. PROMs should include Oxford Knee scores, Oxford Hip scores and EuroQol 5D (EQ‑5D) 3L index scores (outcomes already collected and linked to the NJR).

Resource use

  • Immediate consumables and resourcing associated with surgery, including:

    • pre-operative CT imaging requirements

    • training time and costs

    • surgical and theatre accessories

    • staffing (number and NHS band)

    • total theatre time and total surgical time

    • volume of procedures per day and

    • implant costs.

  • Post-surgery treatment and service use, including:

    • length of hospital stay

    • readmission rates

    • number of physiotherapy sessions and

    • revision rates (stratified by implant type).

  • All costs associated with the immediate consumables and resourcing and with post-surgery treatment and services.

Subgroup analyses

Data could be stratified by:

  • patient characteristics including age, sex, gender, BMI and ethnicity

  • where in the country the procedure was done

  • volume of procedures

  • people from Southeast Asian backgrounds

  • pre-existing medical conditions

  • people having more complex surgeries, such as alternative alignment approaches

  • physical status as defined by American Society of Anaesthesiologists risk scores.

Other important covariates should be chosen with input from clinical specialists to support subgroup analysis.

Safety

  • Adverse events, including conversion to manual surgery and dislocation.

  • Consequences of additional radiation exposure if more imaging is needed.

Data collection should follow a predefined protocol and quality assurance processes should be put in place to ensure the integrity and consistency of data collection. See NICE's real-world evidence framework, which provides guidance on the planning, conduct, and reporting of real-world evidence studies.

3.5 Evidence generation period

This will be 3 years to allow for setting up, implementation, data collection, analysis and reporting.

This page was last updated: