There are several potential benefits to using AI methods in health technology assessment (HTA). These must be balanced against potential risks, for example: algorithmic bias, cybersecurity, and reduced human oversight, transparency and accessibility to non-experts (Gervasi et al. 2022; Zemplényi et al. 2023). In light of the potential risks and the rapidly evolving nature of AI methods, they should only be used when there is demonstrable value from doing so.
The use of AI methods may introduce added complexity. It is important that submitting organisations (including manufacturers and external assessment groups) considering using them for evidence generation and reporting ensure the rationale for doing so is clear. If more explainable and common methods are potentially robust, those should be presented in the first instance, with supplementary use of less transparent approaches. Submitting organisations should clearly justify the use of these methods and outline assumptions (using, for example, the PALISADE checklist) and consider the plausibility of their results.
Submitting organisations considering using AI methods should engage with NICE to discuss their plans. When appropriate, early engagement could be sought through NICE Advice. At later stages of evidence development, organisations should discuss their plans with appropriate NICE technical teams.
Requests to use NICE content for AI purposes are subject to an approval process, licensing arrangement and a fee (for international use). If you would like to use our content for AI purposes, you can find more information on NICE's webpage on reusing our content.
All use of AI should align to the UK Government framework for regulating AI and the key principles should be referenced when considering the value of AI use cases. In the UK, the public sector has embraced various AI ethical frameworks to guide the development and deployment of AI systems (Cabinet Office 2018; DHSC 2021; DSIT 2019; DSIT et al. 2019; Leslie 2019). It is the submitting organisation's responsibility to determine which legislation applies, including data protection laws and ethical standards. When relevant these should be clearly documented.
In alignment with the Medicines and Healthcare products Regulatory Agency (MHRA) and European Parliament guidance for AI use in a medicinal product lifecycle, it is the submitting organisation's responsibility to ensure that all algorithms, models, datasets, and data processing pipelines used are fit for purpose and are consistent with ethical, technical, scientific, and regulatory standards (MHRA 2024; EU 2024; European Medicines Agency 2023).
There remains a need to build trust in the application and use of AI in decision making (Zemplényi et al. 2023). Therefore, any use of AI methods should be based on the principle of augmentation, not replacement, of human involvement (that is, having a capable and informed human in the loop; Fleurence et al. 2024). For example, submitting organisations should conduct careful technical and external validation when AI methods are used, and present the results.
When AI is used, the submitting organisation and authors should clearly declare its use, explain the choice of method and report how it was used, including human input (see paragraph 1.7). The submitting organisation remains accountable for the content included in any submission.
It is the submitting organisation's responsibility to ensure that it is compliant with any licensing agreements. This includes, but is not limited to:
copyright considerations, such as whether the organisation is authorised to use copyrighted or licensed materials in the AI tool, how the AI tool handles copyrighted or licensed materials, and its compliance with copyright law or user licences
whether a business licence is required for any third-party AI tools that are used
who owns the intellectual property produced by the AI tool, and is the organisation authorised to share it with NICE.
The use of AI methods, particularly 'black box' models, can introduce challenges for transparent reporting of evidence (Fleurence et al. 2024). When their use is justified, submitting organisations should consider how these methods can be accessibly presented, including appropriate referencing (for example, of AI tools used and suitability assessment) and the use of lay language. When available, consider using tools to support the explainability of AI methods and increase transparency of their application (Amann et al. 2020).
The use of AI can introduce new risks. These risks should be mitigated by adhering to established guidance and checklists (such as Cochrane, PALISADE, TRIPOD+AI and the Algorithmic Transparency Reporting Standard) during the development, application and reporting of AI, and using AI only in the context of following other relevant best practice guidance recommended by NICE (such as NICE's real-world evidence framework).
When using AI methods, submitting organisations should report the risks they identified with doing so (for example, regarding concerns about transparency and bias) and steps they took to address those risks (Fleurence et al. 2024).
The use of novel AI methods presents cybersecurity risks, such as manipulation of data (data poisoning) or injecting malicious content into prompts (prompt injection attacks; Branch et al. 2022; National Cyber Security Centre 2024). These risks should be considered alongside other risks posed by AI systems. When using AI methods, submitting organisations should provide evidence of the steps taken to ensure robust security measures are in place to prevent such unauthorised access and manipulation.
The use of AI methods in the context of estimating comparative treatment effects (causal inference), represents a potentially very influential and therefore higher-risk application of AI. Their use should be accompanied by considered sensitivity analysis, checked against other suitable methods, and results presented in the context of available clinical evidence ('triangulation').
Ideally, the use of machine-learning methods should be accompanied by pre-specified outcome-blind simulations, conducted independently, to demonstrate their statistical properties in similar settings (for example, different data types or populations) and the correctness of their implementation.
AI methods used for real-world data extraction and curation must be reported, in detail, as part of the data suitability assessment outlined in NICE's real-world evidence framework, making use of reporting tools when possible.