Advice
Clinical and technical evidence
Clinical and technical evidence
A literature search was carried out for this briefing in accordance with the interim process and methods statement. This briefing includes the most relevant or best available published evidence relating to the clinical effectiveness of the technology. Further information about how the evidence for this briefing was selected is available on request by contacting mibs@nice.org.uk.
Published evidence
In total, 5 observational studies are summarised in this briefing including a total of 2,844 slides for analysis. These include 3 full text articles and 2 abstracts. One abstract (Kannan et al. 2020) reports on work later published in full text (Da Silva et al. 2021). The clinical evidence and its strengths and limitations is summarised in the overall assessment of the evidence.
Overall assessment of the evidence
The evidence base for the technology is of low to moderate methodological quality. None of the studies are based in the UK and performance may vary across different populations. The studies reported on suggest that the device may increase diagnostic performance and productivity to current standard care. However further evidence would benefit from sufficiently powered sample sizes across multiple pathology lab assessments within the UK system to show statistically significant clinical benefit compared with standard care.
Da Silva et al. (2021)
Key outcomes
Aimed to assess the true diagnostic performance of the AI system in whole slide images and TRUS prostate biopsies, assessing sensitivity, specificity, positive predictive values and negative predictive values (NPV) Paige Prostate displayed a favourable sensitivity (0.99; confidence interval (CI) 0.96 to 1.0) and NPV (1.0; CI 0.98 to 1.0) and specificity (0.93; CI 0.90 to 0.96). at the part-specimen level. At the patient level, it produced optimal sensitivity of (1.0; CI 0.93 to 1.0) and NPV (1.0; CI 0.91 to 1.0) at a specificity of 0.78 (CI 0.64 to 0.89). Paige Prostate results were generated for 661 whole slide images from 579 prostate needle core biopsy parts. Of the 682 slides initially assessed, 5 were excluded because they could not be retrieved, and 41 discordant part-specimen results were seen which upon re review with IHC were reduced to 29 discordant results. Using Paige Prostate resulted in identifying 4 additional patients whose diagnoses were changed from benign to malignant. The study collected the median time spent per glass slide and calculated that where only whole slide images from parts classified as suspicious were assessed by the pathologists (200/579), Paige Prostate could result in a 65.5% reduction in the diagnostic time taken for the full 579 whole slide images assessment. Findings conclude that Paige Prostate could accurately identify the parts containing cancer as suspicious, without flagging a disproportionately high number of parts as suspicious.
Perincheri et al. (2021)
Key outcomes
This study investigated 2 uses for Paige Prostate in both its utility as a pre‑screening tool to identify negative cores not needing manual review by a pathologist and its utility as a second read tool to identify cancer foci not identified by the pathologist. In this study, the performance of Paige Prostate was reported to be similar to that of pathologists in a highly specialised setting in which prostatic biopsies are typically reviewed by a genitourinary pathologist more than once. Paige Prostate categorised at least 1 core as suspicious of malignancy in 84 of the 86 patients with adenocarcinoma while no cores were identified as suspicious of malignancy in 26 of the 32 without carcinoma or glandular atypia. There was an apparent discrepancy between final diagnosis and Paige Prostate diagnosis in 80 cores, which upon blinded re review reduced to only 21 discordant cores. Issues were reported in 37 slides because of absent tissue or bad scans. From these results the study suggested that using Paige Prostate as a pre-screening tool would reduce the number needed for review by the pathologist to 589 of 1,876 core biopsies, increasing productivity. In the absence of any additional quality review this would also mean that 14 cores with adenocarcinoma would be missed, as well as 6 cores with glandular atypia.
Raciti et al. (2020)
Study size, design and location
Retrospective analysis of 304 prostate needle core biopsy whole slide images in the US.
Intervention and comparator(s)
Paige Prostate Alpha
Paige Prostate Alpha has some operational improvements from Paige Prostate 1.0 to computational efficiency, memory management and improved handling of slides in the iSyntax format and slides of abnormal scan resolutions.
Key outcomes
Pathologists assessed 304 anonymised prostate needle core biopsies and repeated the review 4 weeks later using Paige Prostate Alpha (phase 2). In the analysis, the dataset consisted of 232 anonymised whole slide images of hematoxylin and eosin-stained prostate needle core biopsies. With Paige Prostate Alpha, the average sensitivity for pathologists significantly increased to 90% (from 74%) with no statistically significant change in specificity. The aggregate number of whole slide images classified incorrectly (false negative or false positive) by pathologists without Paige Prostate Alpha was 87. In phase 2, 61 of those slides were correctly classified, while 26 remained incorrect. Pathologists were reported to classify smaller, lower grade tumours more often correctly with Paige Prostate Alpha. Pathologists were also reported to be significantly faster (21%) with Paige Prostate Alpha (p<0.001) taking an average of 63 (+/-39) seconds per slide, compared with 55 (+/-43) seconds with Paige Prostate Alpha. A survey given to the participating pathologists reported that they would consider digitally reviewing whole slide images for primary diagnosis if such a system included Paige Prostate Alpha.
Strengths and limitations
The study was sufficiently powered which allowed appropriate statistical analysis to be done. The authors reported that the pathologists had similar background experience and further studies with users across the pathology community would inform the general useability. The study did not consider using additional inputs that also inform pathologists, including immunohistochemical stains or consultation. Seven authors are employees and equity holders at Paige.
Dogdas et al. (2020)
Key outcomes
Evaluating the performance of Paige Prostate 1.0 at identifying treated prostatic tumour on 64 hematoxylin and eosin-stained slides. These cases were of neoadjuvant treated prostate tissue from needle core biopsies and radical prostatectomies (post‑treatment), as an investigative exercise and is not what the Paige Prostate was designed for. Analysis of the receiver operating characteristic curve showed an area under the curve of 0.96. Using the Paige Prostate 1.0 operating point, it achieved a sensitivity of 91% and a specificity of 94%, corresponding to correctly identifying challenging treated morphology in 59 of 64 slides using expert pathologists as the reference. False negative cases were typically represented by atypical small acinar proliferation that needed expert pathological consensus confirmation. This showed tumour identification despite treatment effects and proposes it as a complementary pathological assessment.
Key outcomes
Evaluating the impact of Paige Prostate on biopsy review, 2 pathologists reviewed the 600 hematoxylin and eosin-stained slides. Paige Prostate's slide‑level sensitivity was 98.9% and its specificity was 93.3% (100% and 78.0% at patient level). The pathologists' average slide-level sensitivity and specificity without Paige Prostate was 90.9% and 98.6%, respectively. The sensitivity with their consensus read and Paige Prostate increased by 5.7% to 96.6% with 0.8% decrease in specificity. Three new prostate cancer cases were discovered with Paige Prostate that were initially missed.
Recent and ongoing studies
Oxford University and regional NHS partners have won the Phase 4 Artificial Intelligence in Health and Care award from the Accelerated Access Collaborative to study Paige Prostate prospectively in a real‑world cancer laboratory setting.