3 Committee discussion

The diagnostics advisory committee considered evidence on Genedrive MT‑RNR1 ID Kit for detecting a genomic variant to guide antibiotic use and prevent hearing loss in babies from several sources, including an early value assessment report and an overview of that report. Full details are in the project documents for this guidance.

Patient and carer considerations

3.1 A patient expert explained the impact that aminoglycoside-induced hearing loss has on children and their families. Parents and families are already under considerable stress when a baby is admitted to a neonatal unit because of suspected infection or sepsis. When hearing loss does occur it is a shock and can cause a mix of emotions including confusion, anger, sadness, disbelief and guilt. The patient expert noted that these emotions could be made worse if they knew a test existed that could have prevented this. It can be difficult for parents to explain to other family members and friends. Parents have several concerns and unanswered questions around how severe the hearing loss may be, how it will affect their child's ability to communicate and develop language skills, and what support or treatment they may need. They highlighted that the severity of hearing loss and its effects can vary between children. Children with hearing loss have reduced access to spoken language, which affects their ability to communicate, as well as their social and emotional development. It can also affect their education and employment opportunities in the future. The committee concluded that aminoglycoside-induced hearing loss has a major impact on the quality of life of children and their families.

Equality considerations

3.2 The committee discussed how the prevalence of the m.1555A>G variant differs between ethnic backgrounds. It said that further evidence should be generated in centres with different patient demographics, for example with different proportions of people from diverse ethnicities. The committee also considered equity of access and noted that it would be necessary to ensure that the test was implemented in a wide range of geographical regions, to include patients from various socioeconomic groups. It concluded that these implementation considerations would be essential to avoid inequalities.

Clinical effectiveness

Time to antibiotic treatment

3.3 Evidence on time to antibiotic treatment from the PALOH study showed no statistically significant difference between standard care and when using the Genedrive test. The PALOH study was a prospective observational implementation trial based in 2 large specialist neonatal intensive care units. But most of the data (94% of babies recruited) was from 1 of these units. A clinical expert said that it was uncertain if the result from the PALOH study was generalisable to other settings such as smaller non-specialist centres, or outside of a neonatal intensive care unit (on a labour ward, for example). There may be differences in how wards are set up, the experience of staff in implementing new technologies, or there may be more demands on staff time. The clinical expert explained that time to antibiotic treatment is very important for the efficacy of the antibiotics, but it is often a challenge in standard care to ensure that antibiotics are administered within 1 hour. However, another clinical expert said that, of those babies that were prescribed antibiotics on the basis of clinical suspicion of sepsis (10% to 12% of babies), only a small proportion would benefit from this treatment being delivered within the hour (around 1%). This is because in a lot of cases, with hindsight, the antibiotics were not needed because there was no infection. However, they are given because the consequences of missing the small proportion who benefit would be severe. The committee agreed that further evidence in alternative settings was needed but the risk of introducing the test while further evidence was generated on time to antibiotics was low. It concluded that the benefits of using the test and preventing aminoglycoside-induced hearing loss in susceptible babies outweighed this risk.

Antibiotic prescribing

3.4 A clinical expert explained that the reason gentamicin is the first-choice antibiotic is because it is active against a wide range of bacteria (Gram-negative bacteria and Staphylococcus aureus), and has a low risk of increasing antibiotic resistance. They said that other antibiotics such as cefotaxime (a cephalosporin) are equally effective and have a better safety profile but are not the first choice because using them can lead to an increase in antibiotic resistance. In the PALOH study, babies that needed antibiotics and were found to have the m.1555A>G variant were treated with cefotaxime. A clinical expert said that if the test was implemented, failed tests and false positive results could lead to more widespread use of alternative antibiotics that may promote antibiotic resistance. The committee said that this would not be a concern if the failure rate and false positive rates remained low but noted that there was some uncertainty in this data (see sections 3.5 and 3.6). The committee said that this should be monitored closely and if the test was implemented, alternative antibiotics should only be used when the test is positive or when the test fails and there is no time for a second test to be done. The committee concluded that, although alternative antibiotics are associated with increased antibiotic resistance, a relatively small number of babies would be treated with them if the real-world test failure rate and false positive rate were low. So if the Genedrive test was implemented it would not lead to increased antibiotic resistance in neonatal units. The committee also concluded that the alternative antibiotics were equally effective and so there were no concerns around treating babies with an alternative if they have the m.1555A>G variant.

Test failure rate

3.5 The PALOH study reported an initial test failure rate for the Genedrive test of 17.1%. This was reduced to 5.7% after modifications to the test buffer. The committee said that it had some concerns about the test being modified during the study and whether the reported improvement in the failure rate would be replicated in real-world use. A clinical expert said that if the test failed, clinicians may decide to use an alternative antibiotic. In many cases this would be unnecessary and could promote the spread of antibiotic resistance (see section 3.4). They said that failed tests could also lead to delays to antibiotic treatment (see section 3.3). However the company said the PALOH study was an implementation study and the results were used to update the test. The updated test is currently in use at Saint Mary's hospital in Manchester and is the commercially available Genedrive MT‑RNR1 ID Kit. Comments from the Manchester Centre for Genomic Medicine said that the current failure rate in real-world use since PALOH is 1.81% (95% confidence interval [CI] 0.6% to 5.18%). The committee noted that this was promising but that this was based on data from a large specialist neonatal intensive care unit and may not generalise to all settings where the test may be used. It said that further data should be collected on the real-world failure rate of the Genedrive test in different settings. The committee concluded that, although there was some uncertainty around the failure rate of the test, this did not present a risk to the babies being tested and the test could reduce the risk of aminoglycoside-induced hearing loss. It also concluded that data on the failure rate could be collected as part of real-world evidence generation.

Diagnostic accuracy of the test

3.6 No false negative results were reported in the PALOH study but only 3 babies were identified with the m.1555A>G variant (true positives). So, although the reported sensitivity estimate was high at 100% there is still considerable uncertainty about the value (95% CI 29.2% to 100%). The committee noted that it would be difficult to reduce this uncertainty without a very large study because of the low prevalence of the m.1555A>G variant in the population (around 0.19%). A clinical expert explained that hearing loss can occur soon after exposure to aminoglycosides, but in some babies it might be after they have passed the newborn hearing test. For babies that develop hearing loss later, it can be more difficult to link it to their exposure to aminoglycosides. So, it may be difficult to follow up and identify babies that had a false negative result in a real-world setting. But the clinical experts said that they were not concerned about false negative results because at the moment there is no testing at all. So even with a risk of false negatives, any testing improves the chance of babies with the m.1555A>G variant being identified and reduces the risk of them having aminoglycosides. The PALOH study also reported 5 false positive results. The authors said that this was corrected with an updated test cartridge design. The committee said that false positive results did not pose any particular risk to babies, because they would be treated with an alternative, equally effective, antibiotic. But it reiterated its concerns about the test being updated during the study and noted that it was uncertain whether changes made to reduce the false positive rate could affect the sensitivity. The committee concluded that there was some uncertainty in the estimated sensitivity of the Genedrive test. It said that post-market surveillance could be used to follow up babies that test negative on the Genedrive test but later go on to develop hearing loss. Retrospective laboratory testing of these babies, in addition to routine confirmatory testing of babies with a positive result, could help improve the precision of the sensitivity estimate and confirm the real-world false positive rate.

Risk of aminoglycoside-induced hearing loss

3.7 The external assessment group (EAG) said that there was clear evidence that the m.1555A>G variant is a risk factor for aminoglycoside-induced hearing loss. However, the evidence is from case-control studies that may overestimate this risk, and so the precise level of risk is uncertain. The committee said that it would be difficult to use an alternative study design because of ethical concerns, so it would likely always be uncertain. Other variants in the MT‑RNR1 gene are also associated with a risk of aminoglycoside-induced hearing loss. Therefore, it considered that babies that test negative for the m.1555A>G variant but still go on to develop hearing loss should be followed up with laboratory testing to determine if they have had a false negative result, or have a different MT‑RNR1 variant. The committee concluded that, although there was some uncertainty around the risk and severity of hearing loss, further evidence generation would be unlikely to address this.

Cost effectiveness

Upfront costs

3.8 Implementing the Genedrive test with real-world data collection would incur upfront costs that would be lost to the NHS (that is, sunk costs) if it is later shown not to be value for money. The committee considered that these upfront costs included the Genedrive MT‑RNR1 ID System (£4,995) and Bluetooth printer (£400). A clinical expert said that centres may need more than 1 Genedrive system to successfully implement the test and the company noted that Saint Mary's hospital in Manchester has 2 systems in use. The EAG noted that this would not substantially affect the cost per test because of the high volume of testing and expected lifespan of the equipment. However, the committee concluded that the upfront costs of implementing the Genedrive test should be carefully considered by commissioners. It noted that purchase options not associated with large capital investment costs should be explored for any conditional recommendation and real-world data collection.

Model assumptions

3.9 The EAG's early economic model made a number of key assumptions. It assumed no effect on time to antibiotic treatment when using the Genedrive test so the consequences of any delay were not included in the economic analysis (see section 3.3). The EAG explained that it made a pragmatic decision to assume no effect on the time to antibiotics (as reported in the PALOH study). It also said that the data on time to antibiotics and outcomes was uncertain. The committee said that any future model for a full diagnostics evaluation and guidance should include time to antibiotic treatment and explore how any delays to antibiotics affect clinical outcomes. The model used the high diagnostic accuracy estimates reported in the PALOH study, which the committee recalled had some uncertainty around them (see section 3.6). The committee noted that the early model also assumed that all babies with the m.1555A>G variant treated with aminoglycosides had severe or profound hearing loss. It said that the true risk and severity of deafness was difficult to estimate and would likely always be uncertain because it would be difficult to study (see section 3.7). The committee concluded that overall, the model assumptions were reasonable, but that any future model to be used for full guidance recommendations should include time to antibiotic treatment and its associated outcomes.

Cost effectiveness

3.10 The committee noted the high cost of identifying 1 baby with the m.1555A>G variant because it is relatively uncommon in the population. But the EAG's early economic model showed that the Genedrive test has the potential to be cost effective because over the lifetime of the baby it is cheaper and more effective than standard care. The committee said that this was based on a number of assumptions (see section 3.9). But it understood that hearing loss is associated with substantial healthcare costs and can have a major impact on quality of life. It also noted that the sensitivity analysis showed that changing these assumptions was unlikely to change the model conclusions. A patient expert highlighted the wider societal costs and educational impact of hearing loss. The committee noted that these costs were not captured in the model. It said that, although these are important considerations, they are outside of the NICE reference case. The committee concluded that, based on the early economic model results, the Genedrive test had the potential to be cost effective over a lifetime.

Evidence generation considerations

Patient demographics and ethnicity

3.11 The prevalence of the m.1555A>G variant varies between different ethnicities. So the committee considered that any further evidence generation should ensure that the Genedrive test is implemented in centres with babies from different patient demographics, for example with different proportions of patients from diverse ethnicities (see section 3.2).

Data should be collected in smaller non-specialist centres

3.12 The committee said that further evidence generation should include smaller non-specialist centres and other settings outside of neonatal intensive care units where the test may be used. This is to ensure that the evidence generated can help assess if the time to antibiotic treatment, test failure rate and diagnostic accuracy estimates reported in the PALOH study are generalisable to other settings (see section 3.3, and sections 3.5 and 3.6). This will also reveal what effect test implementation has on antibiotic prescribing decisions in different centres and settings (see section 3.4).

Test implementation and equity of access

3.13 The committee said that if the test was implemented for further evidence generation this should be done in a wide range of geographical regions, to ensure equal access to patients from different socioeconomic groups.

How are you taking part in this consultation?

Genedrive MT‑RNR1 ID Kit for detecting a genetic variant to guide antibiotic use and prevent hearing loss in babies