Guidance
3 Evidence
NICE commissioned an external assessment centre (EAC) to review the evidence submitted by the company. This section summarises that review. Full details of all the evidence are in the project documents on the NICE website.
Clinical evidence
The clinical evidence comprises 31 randomised controlled trials
3.1 The evidence assessed by the EAC included 31 randomised controlled trials including over 14,000 people. For full details of the clinical evidence, see section 3 of the assessment report in the supporting documentation on the NICE website.
The evidence for reducing surgical site infection incidence is of good quality
3.2 The evidence base for Plus Sutures is extensive, of relatively high quality and is generalisable to the UK NHS. The EAC used the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) methodology for appraising the quality of evidence for each outcome and said that the quality of evidence for surgical site infection incidence was high. This was considered the most important outcome and was reported by nearly all the included studies, with most of them using the same definition. None of the other outcomes listed in the scope had sufficiently robust empirical evidence to show Plus Sutures was statistically superior to standard sutures. However, some other outcomes can be inferred or extrapolated from the established reduction in incidence of surgical site infection, such as a shorter hospital stay, and lower readmission rates and healthcare costs. The EAC concluded that Plus Sutures use is associated with a causative reduction in the incidence of surgical site infection.
Results of company meta-analyses show that Plus Sutures is associated with a 30% reduction in the risk of surgical site infection
3.4 The company did 6 de novo meta-analyses to establish the overall pooled effect size associated with Plus Sutures on the incidence of surgical site infections. The primary outcome was the relative risk of developing a surgical site infection between Plus Sutures and control groups. The 6 separate meta-analyses were done using:
-
all studies that provided enough data (base case, 28 studies)
-
a subset of studies in adults (25 studies)
-
a subset of studies in children (2 studies)
-
a subset of studies in patients with clean wounds (15 studies)
-
a subset of studies in patients with non-clean wounds (12 studies)
-
all studies of Plus Sutures including STRATAFIX Plus that provided enough data, as a sensitivity analysis (31 studies).
The results of the meta-analyses showed that Plus Sutures is associated with a nearly 30% reduction in the risk of surgical site infection in the base case and all results were considered statistically significant (with a 95% confidence interval of 0.59 to 0.85). The EAC noted that the company meta-analyses are of a high quality and at a low risk of bias. The methodology and results are transparent and clearly reported.
The EAC did additional meta-analyses
3.5 The EAC validated the company's meta-analyses by replicating the analysis, and did 3 additional analyses. The EAC noted that because of heterogeneity the studies were not similar enough for fixed effects analysis, and the analysis should primarily be reported using a random effects model. However, this variation had minimal effects on the results. The additional analyses included stratifying the evidence by study quality, sample size and location. The results of the additional analyses indicated that Plus Sutures reduced the risk of surgical site infection, but the size of the effect appeared to be related to study quality and sample size. When only high-quality studies were included in the analysis the difference was not statistically significant. However, the EAC advised that this should be interpreted with caution because the smaller sample sizes and varied event rates will affect the precision and impact of the analysis.
The company submitted additional analyses suggesting sustainability benefits
3.6 Based on the Sustainable Care Pathways Guidance, the company provided an analysis of the environmental impact of surgical site infections to NHS England. Environmental impact is presented in the guidance document in terms of 3 main environmental metrics: greenhouse gas emissions, fresh water use and waste generation. The report indicates that by preventing surgical site infections, using Plus Sutures results in potential environmental benefits to the NHS in England.
Cost evidence
The company identified 8 economic studies
3.7 The company identified 8 studies that were relevant to the economic submission. The EAC concluded that the literature search was satisfactory and agreed that the 8 studies were relevant to the evaluation. The company said that all of the studies reporting on introduction of Plus Sutures resulted in cost savings but that none of the parameters in the company's de novo model were informed by the economic literature. For full details of the cost evidence, see section 9.1.2 of the assessment report in the supporting documentation on the NICE website.
The company's model structure and time horizon are appropriate
3.8 The company submitted a simple decision tree which models a population of adults and children who need wound closure after a surgical procedure. The model assesses the cost of wound closure plus the cost of treatment for people who develop a surgical site infection. An additional branch of the decision tree modelled the mortality of people with a surgical site infection and was used by the company to calculate a cost per death avoided using cost-effectiveness methodology. The EAC considered the model structure to be appropriate, except for the mortality branch of the decision tree which complicates the model for the purposes of a cost-consequence modelling approach. The time horizon modelled was 1 year. The EAC noted that this aligns with published economic evaluations of Plus Sutures.
The EAC accepted all assumptions in the company model
3.9 The company model made the following assumptions:
-
Risk of surgical site infection relates only to those detected and treated during the initial inpatient episode or on readmission.
-
The average surgical site infection episode cost does not include the cost of treating surgical site infections in the community.
-
The risk of infection with Plus Sutures is calculated by applying the relative risk of surgical site infection associated with using Plus Sutures reported in the meta-analyses to a baseline risk of surgical site infection. The baseline risk of surgical site infection is based on UK data.
-
Adverse events were not included in the model.
The EAC concluded that the model assumptions were appropriate, conservative and supported by the evidence.
The EAC made some minor changes to the costs of the technology
3.10 The company provided an estimate of the cost of Plus Sutures based on a weighted average of sales, including knotless, barbed sutures, and STRATAFIX Plus. The EAC reported that the company's estimation of the cost was not sufficiently transparent or reproducible, and included STRATAFIX Plus, which the EAC did not include in their analysis. The EAC amended the cost of the technology by calculating a mean cost of £3.63 to £4.94 depending on Plus Suture type.
The EAC's changes to the model have a minimal effect on results
3.11 Because there were so few changes to the model parameters the EAC and the company's results were similar. In the EAC's base-case analysis Plus Sutures was found to be cost saving by a mean of £13.62 per person compared with the company's £13.88 per person.
The company's extensive sensitivity analyses suggest that using Plus Sutures is cost saving
3.12 The company reported results of a 1‑way deterministic sensitivity analysis that showed that the model was most sensitive to changes in the incidence of surgical site infection. However, the model was still cost saving even when the lowest plausible surgical site infection incidence was used (0.5%). Two‑way deterministic sensitivity analyses were used to explore the combined effect of surgical site infection incidence and relative risk, and surgical site infection incidence and cost of surgical site infection. The results were cost saving in all cases. This was further supported by threshold analyses that reported the following break-even points (deemed by the company and the EAC to be unlikely or implausible):
-
a cost of surgical site infection of less than £1,410
-
incidence of surgical site infection of less than 0.24%
-
a relative risk of 0.93
-
at least 21 sutures.
The results of the probabilistic sensitivity analysis, reported for the base case only, showed that Plus Sutures was cost saving in 99.8% of iterations (out of 1,000 iterations). The summary result was that Plus Sutures was associated with cost savings of £13.96 (95% credible intervals £4.97 to £22.22) per person.
Plus Sutures remains cost saving in the EAC's additional sensitivity analyses
3.13 The EAC did additional sensitivity analyses that explored the uncertainty in the cost savings associated with each subgroup (adults, children, clean wounds and non-clean wounds) and the effect of different relative risk values reported in the EAC's meta-analyses of the clinical evidence (study quality, sample size and location). Plus Sutures was cost saving in all subgroups investigated. The most uncertainty was in the clean wound subgroup (£9.30; 95% credible intervals -£2.24 to £19.26; 94.6% probability of cost saving). The meta-analysis showed that the size of the effect of using Plus Sutures (lowering the risk of surgical site infection) diminished when only studies of a high quality, or large sample size, were included in the analysis (see section 3.5). The sensitivity analyses showed that using Plus Sutures remains cost saving when the relative risk from the higher quality studies and studies with larger samples sizes was adopted, but there was more uncertainty in the results.