SepsiTest assay for rapidly identifying bloodstream bacteria and fungi

The Committee considered the evidence for the diagnostic accuracy of each of the rapid molecular tests compared with blood culture. It noted that 54 studies reported data for the LightCycler SeptiFast Test MGRADE, 6 of which included children or neonates, 4 reported data for SepsiTest and 4 reported data for the IRIDICA BAC BSI assay. The Committee noted that most of the included studies were considered to have unclear risks of bias, particularly about details of the reference standard and the populations included in the studies. The Committee considered that the unclear risk of bias was attributable to poor reporting in the studies, and concluded that it was not possible to adequately assess the quality of the studies included in the diagnostic accuracy meta‑analyses.

The Committee noted that 2 studies compared either the LightCycler SeptiFast Test MGRADE or SepsiTest with MALDI‑TOF mass spectrometry (MS). It considered that both of these studies had relatively small sample sizes and it is likely that the results are not applicable to the UK because of differences in clinical practice in Europe, where the studies were done. The Committee concluded that there was insufficient evidence to establish either the diagnostic accuracy or the clinical utility of the rapid molecular tests against this comparator. Also, because of insufficient clinical data, the Committee concluded that there was too much uncertainty in the analyses for it to be confident that the rapid molecular tests would be cost effective compared with MALDI‑TOF MS.

The Committee questioned the assumption in the diagnostic accuracy meta‑analyses that blood culture is 100% accurate and noted that clinical specialists consider it to be an imperfect reference standard. It heard from the External Assessment Group that on this basis it was possible that the pooled estimates of sensitivity and specificity had been underestimated in the analysis, and so the rate of false positive and false negative results may also have been overestimated. The Committee discussed the reasons for false positive results with the rapid molecular tests. It noted that false positives may be real false positives in situations in which the rapid molecular test detects DNA from contaminant organisms in the blood sample which may result from the testing process. But it also heard from clinical specialists that it is possible that the rapid molecular tests may give more accurate results in some scenarios, such as the detection of fastidious organisms that may not grow in culture. The Committee also heard from clinical specialists that the rapid molecular tests may detect transient bacteraemia in some people, but that the clinical implications of this are not fully understood. It is possible that people may have extended courses of antibiotics and stay in hospital for longer if transient bacteraemia is detected. The Committee concluded that although the sensitivity and specificity may have been underestimated in the meta‑analyses, the absence of data on the clinical significance of discordant results means that the size of any underestimation cannot be determined.

The Committee discussed the number of positive blood cultures in the included diagnostic accuracy studies and their prevalence in clinical practice. It heard from clinical specialists that blood culture is often negative in practice, with only around 10% of blood cultures being positive. The Committee considered that the low prevalence of positive blood cultures was likely to mean that there would be a relatively low number of false negative rapid molecular test results in routine practice. Also, the absolute rate of false‑positive rapid molecular test results is likely to be high because of the greater prevalence of negative blood cultures. The Committee concluded that although the absolute number of false‑negative rapid molecular test results was likely to be low in practice, the consequences of changing antimicrobial treatment in this group could be severe.

The Committee discussed the studies included in the clinical‑outcomes systematic review. It noted that fewer studies reported clinical‑outcome data compared with diagnostic‑accuracy data, and that studies typically reported data for the LightCycler SeptiFast Test MGRADE only. The Committee noted that most of the studies were done in Europe or the USA and questioned the applicability of the clinical‑outcome studies to the UK. It heard from clinical specialists that although the treatment of sepsis is based on international guidelines, clinical outcomes such as duration of intensive care unit stay and duration of antimicrobial therapy cannot usually be applied to the UK from international studies because of differences in antibiotic prescribing practices. The Committee concluded that although the included studies provide some indication of the likely effect of the rapid molecular tests on clinical outcomes, additional UK‑based studies are needed to show the clinical utility of the tests in practice.

The Committee considered the test turnaround times reported in the studies and heard from clinical specialists that the shorter times seen in research studies are unlikely to be seen in routine clinical practice, unless a molecular service is available 24 hours a day. It noted that 24‑hour services may become available if microbiology laboratories are joined into networks or centralised, but that this was unlikely to happen in the very near future. The Committee also noted that in some studies, the reported test‑failure rates for the LightCycler SeptiFast Test MGRADE were high (up to 24.2%) and considered that this could further affect its potential to rapidly deliver information for clinical decision‑making. The Committee questioned why the reported test‑failure rates were high in some studies but heard from the External Assessment Group that the reasons for the failed tests were not reported. The Committee concluded that faster reporting of results is highly dependent on laboratory infrastructure and that the turnaround times needed to gain benefits from the rapid molecular tests are unlikely to be achieved in routine practice.

The Committee considered the data for mortality and duration of intensive care unit or hospital stay and noted that the studies were unlikely to have had sufficient power to detect statistically significant differences for these clinical endpoints. The Committee also noted that most of the studies did not report statistically significant differences between the rapid molecular tests and standard practice. Also, it heard from clinical specialists that both mortality and duration of stay among people with suspected bloodstream infection are likely to be influenced by multiple factors, and that any differences are unlikely to be solely because of the use of a rapid molecular test. The Committee concluded that mortality and duration of stay may not be appropriate primary clinical outcomes for studies, and suggested that future studies should consider using change in antimicrobial prescribing as a surrogate clinical outcome.

The Committee discussed the plausibility of the rapid molecular tests having an effect on antimicrobial prescribing. It noted that the results of the clinical‑effectiveness analysis suggested that only small numbers of people, if any, would have changes made to their antimicrobial treatment plan. The Committee heard from clinical specialists that there may be some situations in which the rapid molecular tests could affect patient management. Also, it heard that these situations would be restricted to instances in which the rapid test was positive, because the current accuracy of the tests was not sufficient to convince clinicians to withdraw antibiotic therapy on the basis of a negative test result. The Committee concluded that although the rapid molecular tests might give results more quickly, it was unlikely that the information they give would have an effect on patients' treatment plans and antimicrobial prescribing at present.

The Committee discussed the results of the economic analyses and questioned whether the use of an imperfect reference standard to calculate the estimates of diagnostic accuracy for the rapid molecular tests could have introduced bias. The Committee heard from the External Assessment Group that negative results were assumed not to have an effect on outcomes and that false‑positive results were associated with benefits in the model. The Committee concluded that any underestimate of pooled diagnostic accuracy in the clinical‑effectiveness analysis is unlikely to have a substantial effect on the results of the economic model.

The Committee questioned the assumptions made about the number of tests processed per day. It heard from clinical experts that the estimates based on 68 tests per day were unrealistic and that a large service laboratory would be unlikely to get more than 40 blood cultures per day. The Committee noted that the External Assessment Group had also produced estimates based on 2.4 and 17 tests per day. The Committee heard from the External Assessment Group that an assumption of 68 tests per day was included as an extreme scenario to show the effect on the results of the economic analyses. The Committee concluded that the most representative scenarios in the economic analyses were those that assumed either 2.4 or 17 tests per day.

The Committee discussed the differences in the results produced in the 2 different base cases of the economic analyses. It noted that the main difference between the 2 base cases came from the difference in data source for clinical outcomes: base case 1 used data taken from the systematic review, and base case 2 used data based on expert opinion. The Committee noted that the systematic review suggested that the rapid molecular tests had no effect on clinical outcomes, but some of the clinical experts thought that the tests may be beneficial, although their estimates of the size of the benefit varied widely. The Committee concluded that the tests may offer clinical benefit, but there is too much uncertainty in the size of the benefit to determine the effect of introducing the tests into clinical practice. The Committee also noted that the incremental cost‑effectiveness ratios (ICERs) in base case 2 ranged from the rapid molecular tests being more costly and equally effective (dominated) than blood culture, to being less costly and more effective (dominant) than blood culture alone, when using estimates from individual clinicians. The Committee considered that the wide range of ICERs resulted from the high level of variation between the clinicians' estimates. The Committee concluded that the effect of introducing the rapid molecular tests on NHS resources was highly uncertain and that the results of the economic analyses were subject to substantial uncertainty.

The Committee considered the likely effect of the costs and outcomes that had been excluded from the economic analyses and noted that these included laboratory overhead and additional staff costs, and clinical benefits that may be accrued through improved antimicrobial stewardship. The Committee noted that because the results of the clinical‑effectiveness analysis suggested that the effect of the rapid molecular tests on antimicrobial prescribing was highly uncertain, it would have been inappropriate to extrapolate the clinical outcomes to estimate an effect on antimicrobial stewardship. It also noted that the rapid molecular tests would most likely increase laboratory overhead costs, and possibly staff costs, and concluded that because of the clinical uncertainties their absence from the economic analyses was unlikely to have a substantial effect.

The Committee considered the results of the threshold analyses and noted the reductions in antimicrobial costs that would be needed for the tests to be considered cost effective. The Committee noted that this ranged from £823.34 to £1482.28 per 100 positive tests, depending on whether the rapid molecular tests were compared with blood culture or blood culture plus MALDI‑TOF MS. The Committee concluded that because of the prevalence of positive tests in clinical practice, the costs of the rapid molecular tests were unlikely to be offset by reduced antimicrobial costs alone.

The Committee noted that the economic analyses did not include neonates and children, and that the model was based on an adult population with a mean age of 58 years. The Committee considered that the estimated quality‑adjusted life year (QALY) gain by avoided 30‑day mortalities would be greater for children and neonates because of their greater number of life years remaining, but accepted that there were insufficient clinical‑utility data for this population for an economic analysis.

The Committee considered the potential benefits of the interventions in practice. It heard from clinical experts that because the tests can be used directly on whole blood samples, they may be able to give information on a pathogen's identity earlier in the care pathway than tests that need incubated blood samples or samples from culture plates, which could be beneficial for antimicrobial stewardship. Also, it heard that the information from the rapid molecular tests may be used to modify a person's antimicrobial therapy, particularly when empirical antimicrobial therapy (antibiotics which are prescribed based on clinical presentation) has been prescribed. The Committee concluded that one of the key claimed benefits of the rapid molecular tests is their potential to contribute towards antimicrobial stewardship.

The Committee considered that because the rapid molecular tests need to be used in addition to blood culture for antimicrobial susceptibility testing, they may be less suitable for use in neonates and children. The Committee heard from clinical experts that this is a particular issue for tests that need a large volume of whole blood. The Committee also heard from clinical specialists that using a lower volume of blood from these patients for the molecular tests may have an adverse effect on the test's sensitivity and concluded that further exploration of these analytical issues should be encouraged.

The Committee discussed the value of developing research recommendations for the rapid molecular tests. The Committee considered that for the tests to have clinical utility in both research settings and routine practice, clinicians would need to be certain that the tests are sufficiently accurate, and be confident that basing antimicrobial prescribing decisions on the results of the tests would not lead to adverse outcomes for people. The Committee noted that the reported accuracy data from the systematic review were unlikely to be sufficient for clinical decision‑making at present. The Committee concluded that further research in the UK is needed to determine the clinical scenarios in which the tests may offer most benefit in clinical decision‑making and to quantify their clinical utility. The Committee also considered that future studies should investigate using the rapid molecular tests in conjunction with other biomarkers, such as procalcitonin, and diagnostic tests that may be used to assess people with suspected sepsis.

The Committee considered that, conceptually, the molecular tests show promise for the early identification of fungal pathogens in people who are thought to be at increased risk of developing invasive fungal infections. The Committee concluded that if the accuracy of the tests was sufficient to guide clinical decision‑making in this population, they could offer substantial value and address a clinically unmet need. The Committee encouraged future studies in this population and highlighted that the studies should aim to quantify the clinical utility of the rapid molecular tests, including their effect on antifungal prescribing. The Committee noted that studies planned by the National Institute for Health Research Health Technology Assessment Programme may investigate the use of rapid tests for identifying fungal pathogens.

The Committee considered the utility of further research to quantify the levels of certainty about the results of rapid molecular tests, which clinicians need to have before they decide to change treatment and level of care for patients. The Committee noted that the results of an elicitation exercise could be used to guide the development of future diagnostic tests that are designed to be used to change treatment plans for patients who are acutely unwell, and wished to encourage this research.

The Committee considered that because an increasing number of microbiology laboratories are adopting MALDI‑TOF MS for rapidly identifying bloodstream bacteria and fungi, future studies aiming to establish the clinical utility of rapid molecular tests should include this technology as a comparator when possible.

The Committee noted that there was insufficient evidence to determine whether the tests were clinically effective in children and neonates. It wished to encourage the inclusion of these populations in future research studies, and noted that particular consideration should be given to establishing whether the blood volumes needed for the tests in this assessment are suitable for these populations.