Advice

Expert comments

Expert comments

Comments on this technology were invited from clinical experts working in the field and relevant patient organisations. The comments received are individual opinions and do not represent NICE's view.

All 4 experts were familiar with the technologies, but none of them had used them. Two of the experts were aware of their use in screening programmes in Scotland and Singapore.

Level of innovation

Two experts were not aware of any other relevant technologies. The other 2 experts reported that there are other similar companies with AI technologies for diabetic retinopathy, but they have not published their research or have not received CE marking yet.

Potential patient impact

All experts agreed that the technologies could be applied to all people with diabetes aged 12 and over as per current protocol.

All 4 experts agreed using these technologies has potential benefits for the patients. One explained that early detection of sight-threatening diabetic retinopathy by these technologies would result in earlier treatment and potentially saving sight. People with diabetic retinopathy can be at a higher risk of cardiovascular disease and stroke, so early intervention could prevent them becoming a problem. Potentially, other non‑diabetes related serious illnesses could be diagnosed from the retinal images. One expert said that because these technologies can be used for any eye and for patients from different ethnicities, genders, ages and duration of diabetes, they serve a wide range of patients. Another expert mentioned reducing hospital visits as a benefit if people are able to be screened at home, adding that these systems will help people unable to get to hospital because of reduced mobility.

Potential system impact

Three out of 4 experts said that these technologies are useful and could be used to save time and resources. They also said they could improve reproducibility and consistency across practices and provide quality assurance for screening programmes. Two of the experts suggested that these technologies should be used in screening programmes. All 4 experts agreed that these technologies may be used alongside human graders and standard care, and may replace at least primary level human grading. Two experts said that AI might gradually replace secondary and arbitration level human grading as well.

Each expert reported different benefits of using AI systems, including real-time reporting of results to diagnosis and reduction in waiting time for patients, less delay in follow-ups, reduction in human contacts with the possibility of home monitoring, and increased efficiency of screening programmes, and cost savings for the NHS. One of the experts noted that implementing these technologies might at first lead to increased costs, but in the long term, they can be cost saving. However, they said there should be an ongoing economic evaluation of these technologies as further developments are anticipated. One expert said using AI technologies will lead to less dependency on scarce staff.

One of the experts believed that these systems could reduce the workload and the need for staff. However, 2 experts highlighted that, because the systems can grade constantly, the possible higher detection rate of diabetic retinopathy could lead to increased referrals to hospital eye service. Another expert said that the current pathway could be shortened by using AI systems. They said that experienced human graders could instead better manage sight-threatening retinopathy, potentially leading to better clinical outcomes. One expert said using these systems is unlikely to change the current pathway or clinical outcomes. The grading protocol determines the outcome, so changes to protocols in the future may lead to fine-tuning of the algorithms used by these systems. Two experts also said that camera hardware changes may affect the performance of these technologies and shift the setting from clinic to home. Therefore, the benefits to the healthcare system depend on where the technologies are implemented in the clinical pathway. They could add an extra layer of quality assurance to the graders. Almost 90% of negative cases received only 1 human grading. Adding these systems would mean all images were graded by the AI system and at least 1 human grader.

Three experts said better and secure cloud storage and data transferring facilities would be needed, which may need investment. One of the experts said that none of the systems had been integrated into existing patient management systems. Two experts said staff needed training in using web-based and data-based technologies. One expert mentioned a lack of trust in the systems, while another brought up the potential to miss some pathologies if human graders are replaced. One expert said ownership of errors was a potential barrier, and one said governance issues was another.

General comments

Potential usability issues mentioned by the experts were:

  • the use of a different grading classification

  • appropriate integration of the AI system into existing systems

  • smooth data transfer

  • staff training and testing how the technologies perform when screening high volumes of images, similar to what would be expected in routine practice

  • camera type and cost

  • the number of images that need to be taken (and so time)

  • the need for dilated pupils.

The experts said more research was needed on existing real-world datasets (retrospectively), studying false positive and false negative cases, and tests in screening programmes (prospectively). They also recommended continuous performance audits and potential upgrades and assessment of systems' ability to identify all major eye diseases, not just diabetic retinopathy. One of the experts said that none of the companies reported results the same way so pre‑proliferative and proliferative features could be different across platforms.