4 Evidence
The diagnostics advisory committee (section 8) considered evidence on the DYSIS colposcope with DYSISmap (hereafter referred to as DYSIS) and the ZedScan I for detecting cervical intraepithelial neoplasia (CIN) from several sources. Full details are in the project documents for this guidance.
4.1
For the diagnostic accuracy review, studies were included if a prospective cohort had the index test or their prototypes (DYSIS or ZedScan I done in addition to colposcopy) and reference standard (histopathology) done independently, and contained enough data to allow diagnostic accuracy estimates to be calculated. For the effectiveness and implementation reviews, observational or experimental studies were included if DYSIS or ZedScan I, or their prototypes, were used in addition to colposcopy. All studies included in the diagnostic accuracy review were appraised using the QUADAS-2 tool. Studies in the implementation review were appraised using guidance from Burns et al. (2008) and the Centre for Evidence Based Management (2014).
4.2
In total, 12 studies were included: 11 in the diagnostic accuracy review, 3 in the review of clinical outcomes, and 5 in the implementation review. Some studies included outcomes that were relevant to more than 1 review. Most studies were reported in more than 1 paper or abstract.
Diagnostic accuracy
4.3
Of the 11 studies included in the diagnostic accuracy review, 9 included data for DYSIS and 2 included data for ZedScan (1 for ZedScan I and 1 for a prototype). All studies were done in hospital-based colposcopy clinics, and 6 were multicentre studies. Five studies included at least 1 centre in England (both ZedScan studies and 3 DYSIS studies). Most of the people in the studies were referred for colposcopy because of an abnormal screening result.
4.4
Of the 9 DYSIS studies, 1 was considered to be at low risk of bias and the other 8 at high risk of bias. Both ZedScan studies were considered to be at a high risk of bias. The main source of bias in the studies was verification bias. This was because biopsies were not taken to confirm the absence of disease when the colposcopist did not identify any abnormalities because this is not generally considered to be good clinical practice. Concerns about the generalisability of the results of the ZedScan studies were highlighted because most of the people in the studies were examined at a single centre.
4.5
Meta-analyses were done for the diagnostic accuracy of DYSIS, which included 6 studies. Two studies were excluded because they only reported data for subgroups and 1 was included in a narrative analysis only. The analyses assume that DYSIS video colposcopy (without the DYSISmap), the comparator in the DYSIS studies, is equivalent in diagnostic accuracy to binocular colposcopy (used in the ZedScan studies and in routine NHS practice). The threshold used to determine a positive result was CIN 2 or worse (CIN 2+). No meta-analysis was done for the ZedScan studies.
DYSIS
4.6
The pooled results from the meta-analyses are summarised in table 1. The pooled positive predictive value of colposcopy was 55.78% (95% confidence interval [CI] 47.54% to 64.03%) and of DYSISmap with colposcopy was 43.60% (95% CI 33.12% to 54.07%). The corresponding negative predictive value of colposcopy was 86.70% (95% CI 80.17% to 93.22%) and of DYSISmap with colposcopy was 92.20% (95% CI 88.06% to 96.34%). A sensitivity analysis was done with a logistic regression model. Roensbo et al. (2015) was excluded because this study did not assess DYSIS with colposcopy directly but recorded whether a colposcopist agreed or disagreed with the DYSISmap. To examine the effect of verification bias, results were stratified by the number of biopsies taken in the studies when both DYSIS and colposcopy did not identify any areas of abnormality.
4.7
The results of the meta-analyses suggest that compared with colposcopy alone, DYSIS with colposcopy improves sensitivity for detecting CIN 2+, although this is associated with a reduction in specificity. However, the results of the logistic regression model show a statistically significant difference in specificity between DYSIS and colposcopy (difference in log odds 1.33, p<0.0001), but no significant difference in diagnostic odds ratio (difference in log odds 0.04; p=0.84). This suggests that DYSIS increases the number of people suspected of having CIN 2+ and may therefore increase the number of biopsies taken. But it may not improve the ability to discriminate between lesions with and without CIN 2+ when compared with colposcopy. The results of the sensitivity analyses designed to explore verification bias in people with negative DYSIS and colposcopy examinations suggested that sensitivity and specificity estimates decline as the number of random biopsies taken increases.
4.8
An additional 5 studies were included in a separate narrative analysis. This confirmed the results of the meta-analyses; DYSIS improves sensitivity but reduces specificity when compared with colposcopy.
Abbreviations: 95% CI, 95% confidence interval; NPV, negative predictive value; PPV, positive predictive value.
References:
a Budithi et al. (in press), Coronado et al. (2016), Louwers et al. (2011), Roensbo et al. (2015), Salter et al. (2016) and Soutter et al. (2009).
b Coronado et al. (2016), Louwers et al. (2011) and Roensbo et al. (2015).
c Budithi et al. (in press), Coronado et al. (2016), Louwers et al. (2011), Salter et al. (2016) and Soutter et al. (2009).
d Budithi et al. (in press), Coronado et al. (2016) and Salter et al. (2016).
ZedScan I
4.9
Two studies were included in a narrative analysis; 1 included the current version (ZedScan I) and the other a third-generation prototype. The results are shown in table 2. Tidy et al. (in press) includes results for the current version of the device in a human papilloma virus (HPV) primary screening setting, but none for colposcopy alone. The results of the studies suggest that using ZedScan with colposcopy may have better sensitivity or specificity than colposcopy alone depending on the threshold used (which is set by the manufacturer). But when a regression model was fitted to the results from Tidy et al. (2013), the improvement in diagnostic accuracy was not quite statistically significant (difference in log diagnostic accuracy 0.488, p=0.078). However, only 1 study was available for analysis and the EAG commented that this is a conservative approach which should be considered as exploratory only.
Note: Disease present, colposcopy was considered positive if at least 1 measurement point was suggested for biopsy; colposcopic impression, colposcopy was considered positive if it was judged that high-grade CIN was present.
Abbreviations: CI, confidence interval; CIN, cervical intraepithelial neoplasia.
4.10
Further data on ZedScan I were available in 2 substudies of Tidy et al. (in press). In a conference abstract Tidy et al. (2016) reported that the performance of the technology varied across colposcopy clinics in England, Ireland and Germany, with sensitivity ranging from 73.1% to 100% and specificity from 25.7% to 58.1%. McDonald et al. (2017) evaluated the accuracy of ZedScan I in people with known high-risk HPV genotypes and compared its performance among those with HPV 16 and those with other high-risk genotypes. The sensitivity of ZedScan I was high (100%) regardless of genotype but the sensitivity of standard colposcopy was higher in the HPV 16 group (86.9%) than in the other high-risk genotypes group (79.7%).
4.11
A study including 91 people (Muszynski et al. 2017) was submitted during consultation. In 1 French hospital, using ZedScan I with colposcopy increased detection of people with high-grade lesions by 47.3%. The rate at which biopsies were taken also increased when making decisions using results from both ZedScan I and colposcopy, compared with using colposcopy alone. The reported sensitivity of ZedScan I with colposcopy was 93.3% compared with 61.3% for colposcopy alone. The reported specificity of ZedScan I with colposcopy was 34.4% compared with 80.0% for colposcopy alone.
Test positive rates
4.12
Test positive rates ranged from 21.22% to 55.51% for DYSIS and from 13.77% to 42.68% for colposcopy alone in 6 DYSIS studies (Budithi et al. in press, Coronado et al. 2016, Louwers et al. 2011, Roensbo et al. 2015, Salter et al. 2016 and Soutter et al. 2009). In each study the test positive rate was always higher for DYSIS than for colposcopy alone.
4.13
Test positive rates ranged from 30.20% to 77.04% for ZedScan, depending on the cut-off used in the 2 studies (Tidy et al. 2013, Tidy et al. in press). Test positive rates for colposcopy were 41.84% when colposcopic impression was used as a cut-off and 73.47% when disease present was used as a cut-off (Tidy et al. 2013).
Test failure rates
4.14
Test failure rates (including failures not related to the technology) with DYSIS were reported in 6 studies and ranged from 2.9% to 31.4%. The highest failure rate was reported by Soutter et al. (2009), which included a prototype version of the system that had problems with unsatisfactory view and faulty acetic acid applicators. Failure rates for ZedScan (including failures not related to the technology) were reported in 2 studies. They were 5.6% (Zedscan I) and 13.6% (prototype; Tidy et al. in press and Tidy et al. 2013).
Biopsy rates
4.15
All diagnostic accuracy studies included in the external assessment group's (EAG) analysis included some data on the number of diagnostic and treatment biopsies taken, but there were not enough details to assess whether the adjunctive technologies had a substantial effect on this.
4.16
Two prepublication manuscripts by Cholkeri-Singh et al. (2018) and DeNardis et al. (2017), which included additional data from the IMPROVE-COLPO trial, were submitted during consultation. Diagnostic accuracy data from this study had been included in the EAG's analysis. IMPROVE-COLPO was an observational study done in 39 colposcopy clinics in the US.
4.17
Cholkeri-Singh et al. (2018) reported results of a 2-arm observational study in which people who were prospectively assessed using DYSIS were compared with historical controls (people assessed with standard colposcopy by the same colposcopists). The yield of CIN 2+ (defined as the proportion of people with at least 1 biopsy showing CIN 2+) was higher in the DYSIS group (9.48% compared with 7.21%; p=0.014). The yield of CIN 3+ was also higher in this group (3.23% compared with 2.07%; p=0.031). The number of people having biopsies between the groups was similar (71.6% compared with 71.5%), but the average number of biopsies per person was higher for the DYSIS group (1.26 compared with 1.03).
4.18
DeNardis et al. (2017) reported results of a cross-sectional observational study in which DYSISmap was used after an initial assessment with DYSIS video colposcopy to identify further sites for biopsy. DYSIS video colposcopy-directed biopsies identified 78 people with CIN 2+; DYSISmap-assisted biopsies identified a further 34 people with CIN 2+. Also, DYSIS video colposcopy-directed biopsies identified 30 people with CIN 3+ and DYSISmap-assisted biopsies identified a further 15 people with CIN 3+. The positive predictive value of DYSIS video colposcopy-directed biopsies was 13.24% compared with 16.16% for DYSISmap-assisted biopsies.
Subgroup analyses
4.19
Data on referrals for low-grade and high-grade cytology suggested that colposcopy was less sensitive for detecting CIN 2+ in low-grade cytology referrals. No differences in sensitivity were seen for DYSIS and ZedScan I.
4.20
There were not enough data to determine whether the accuracy of any of the technologies differed between people with and without high-risk HPV.
4.21
Founta et al. (unpublished) reported data from a test of cure population for whom the EAG calculated 95% confidence intervals. This showed a sensitivity of 0% (95% CI 0% to 53%) and a specificity of 94.0% (95% CI 89.35% to 98.65%) for colposcopy, and a sensitivity of 80.0% (95% CI 44.94% to 100%) and a specificity of 64.0% (95% CI 54.59% to 73.41%) for DYSIS in a test of cure population. The accuracy of colposcopy was substantially different in this study compared with the summary estimates provided in the meta-analyses for all colposcopy referrals.
Clinical effectiveness
4.22
Data on adverse events were reported in 5 studies. In a ZedScan prototype study, 1 person felt unwell after the examination and 2 people had issues with bleeding after biopsies were taken. It is uncertain whether these events were related to using the ZedScan. No adverse events were reported in 4 DYSIS studies.
4.23
No data were found for morbidity and mortality associated with treatment and biopsy during colposcopy, or for health-related quality of life. There were insufficient data to determine whether the increased detection of CIN 2+ was associated with a reduction in cervical cancer.
4.24
Two systematic reviews of adverse outcomes of CIN treatment were found. Kyrgiou et al. (2015) focused on fertility and early pregnancy outcomes (less than 24 weeks' gestation). People who had treatment for CIN were at increased risk of miscarriage in the second trimester of pregnancy (relative risk 2.60, 95% CI 1.45 to 4.67). Kyrgiou et al. (2016) focused on obstetric (more than 24 weeks' gestation) and neonatal outcomes. People who had large-loop excision of the transformation zone (LLETZ) were at increased risk of giving birth prematurely (relative risk 1.56, 95% CI 1.36 to 1.79). The risk increased as the depth of the excision increased.
Implementation
4.25
Five studies were included in the implementation review. Of these, 3 were done in the UK (Lowe et al. 2016, Palmer et al. 2016 and Budithi et al. in press), 1 in Spain (Coronado et al. 2014) and 1 in the Netherlands (Louwers et al. 2015). None of the studies used validated questionnaires.
Patient and clinician satisfaction
4.26
Lowe et al. (2016) surveyed 763 patients in 4 NHS hospitals that were using DYSIS. Two questionnaires were used: 1 for people having their first colposcopy and 1 for people who had previously had a colposcopy. The number of respondents per questionnaire was not reported in the conference abstract available to the EAG. Participants reported that DYSIS did not take longer than their previous smear test or colposcopy and that anxiety was reduced during and after examinations compared with previous examinations.
4.27
Louwers et al. (2015) gave a patient satisfaction questionnaire to 239 people who had a DYSIS examination. Results showed that 93.9% of people agreed or strongly agreed to have colposcopy with DYSIS if it helped locate CIN; 29.5% agreed or strongly agreed that DYSIS was less comfortable than a cervical smear; 16.5% reported that DYSIS made them feel nervous during the examination, and 6.5% thought that an examination with DYSIS took too long.
4.28
Budithi et al. (2017) gave questionnaires to both patients and colposcopists in 5 colposcopy clinics in Wales; 68 patients responded and 45 colposcopist responses were received (the number of colposcopists was not reported in the abstract). Results from patients showed that 86% agreed or strongly agreed that the DYSIS images helped their understanding and were reassuring; 52% believed DYSIS to be more accurate than their previous colposcopy; 4% thought that DYSIS lasted too long compared with previous colposcopies and 13% found it less comfortable. Of the responses received from colposcopists, 96% agreed or strongly agreed that they were confident about colposcopy and their decision-making in selecting biopsy sites. But only 48% went on to agree that DYSISmap affected their selection of biopsy sites; 58% said they were able to identify additional sites with DYSISmap and 55% agreed or strongly agreed that DYSISmap improved their colposcopic examination.
Colposcopist experience
4.29
Coronado et al. (2014) surveyed 63 colposcopists with different levels of experience. A retrospective review of 50 colposcopy and DYSISmap images was also done. The study found that the correct diagnosis (either normal, low-grade lesion, high-grade lesion or cancer) was made more frequently with DYSIS than with standard colposcopy for colposcopists with low and medium levels of experience. There was no difference for highly experienced colposcopists. All groups agreed that DYSIS is better at directing diagnosis and provides more information than standard colposcopy. The survey also reported that using DYSISmap improved detection of CIN 2+ by colposcopists of all experience levels. However, the EAG noted that this was based on a small subgroup analysis of the retrospective review of stored images.
Cost effectiveness
Review of economic evidence
4.30
Two relevant economic evaluations were identified; 1 for DYSIS compared with colposcopy over a lifetime time horizon (Wade et al. 2013) and another for a ZedScan prototype compared with colposcopy over a 3‑year time horizon (Whyte et al. 2013). Wade et al. was produced for NICE's diagnostics guidance 4 on adjunctive colposcopy technologies and found that DYSIS dominated colposcopy (that is, DYSIS cost less and was more effective than colposcopy). Whyte et al. reported lower costs associated with the use of a prototype ZedScan device per person with CIN 2 or 3 treated, because it reduced both rates of overtreatment and the number of follow-up appointments needed for people with CIN 1. However, this was associated with a reduction in the number of CIN 2 or 3 lesions treated and a consequent reduction in the number of cancers detected. Neither study fully addressed the decision problem.
Modelling approach
4.31
The EAG developed a de novo economic model designed to assess the cost effectiveness of DYSIS and ZedScan I, used with colposcopy, in both an HPV triage and an HPV primary screening setting. The analyses took the perspective of the NHS and personal social services and had a 60‑year (lifetime) time horizon. All costs and effects were discounted at 3.5%.
Model structure
4.32
A patient-level state-transition model with a 6‑month cycle time was constructed using TreeAge Pro (2016) software. The model included 500,000 simulations to ensure that first-order uncertainty was adequately captured, that is, variability in the simulated experiences between identical patients. The model incorporated both screening and treatment pathways: 1 submodel simulated the natural history of CIN and cervical cancer, and another submodel simulated adverse obstetric outcomes for people who had treatment for CIN. The adverse obstetric outcome model captured the costs and quality-adjusted life year (QALY) decrements associated with initial management and the increased probability of neonatal mortality and QALY decrements associated with higher risks of disability among infants born preterm. The natural history model was adapted from Kulasingam et al. (2013) with invasive cancer parameters taken from Campos et al. (2014).
4.33
At the beginning of the first cycle each person is referred for colposcopy and has treatment if needed, before entering the natural history model. In subsequent cycles, the person can follow 1 of 4 screening and treatment pathways: no screening, colposcopy referral, routine screening, or a follow-up pathway for those who have had previous treatment, unless they died in the previous cycle. Every pathway ends with the person entering the natural history model.
4.34
The model was implemented using a random walk and for each person it simulated the following uncertain events occurring: disease progression, diagnostic results or treatment outcomes. The characteristics that determined the associated events and transitions for each person in the model were:
-
age
-
health state (clear, HPV, CIN 1, CIN 2 or 3, cancer)
-
reason for referral for colposcopy (high-grade or low-grade cytology)
-
next scheduled screening (routine call, 6‑month cytology, 6‑month colposcopy, test of cure, CIN 1 follow-up)
-
time elapsed since last screening
-
type of clinic visited ('see and treat' or 'watchful waiting').
Identical patients were run through each treatment strategy and random numbers were maintained across all runs of the model.
4.35
Two base cases were modelled: HPV triage and HPV primary screening. The modelled pathways for HPV triage were based on those outlined in the NHSCSP's colposcopy and programme management guidelines (2016). For HPV primary screening the modelled pathways were based on the testing algorithms used in the NHSCSP's pilot sites.
Base-case results
4.44
The following assumptions were applied in the base-case analysis:
-
Diagnostic accuracy estimates for both colposcopy and the adjunctive technologies were based on a cut-off of CIN 2+.
-
The probability of a positive colposcopy result was:
-
identical for people with clear, HPV or CIN 1 results
-
identical for people with CIN 2 or 3 or invasive cancer.
-
The choice between a 'see and treat' clinic and a 'watchful waiting' clinic was independent of diagnostic accuracy.
-
Biopsy and histopathology (the reference standard) were 100% accurate.
-
Excision at the first colposcopy appointment was only possible for referrals for high-grade lesions with a positive colposcopy result.
-
For low-grade lesion referrals, CIN 2+ was confirmed by diagnostic biopsy before treatment.
-
CIN 1 lesions were not treated and people had a 12‑month follow-up screening in the community.
-
People whose lesions were treated for CIN remained at risk of preterm birth (before 37 weeks' gestation) for each year after treatment up to the age of 45.
-
When cancer was detected, treatment was offered appropriate to the stage. An excess risk of mortality was applied for 5 years and decreased according to time since diagnosis.
-
DYSIS or ZedScan I examinations were the same length as a standard colposcopy examination.
-
ZedScan I was used for diagnostic colposcopies only.
4.45
There were 2 base cases: 1 for HPV triage and 1 for HPV primary screening. In a 'see and treat' clinic, treatment was done at the first visit for people who had a referral for a high-grade lesion according to cytology and a colposcopy examination graded as CIN 2+. In a 'watchful waiting' clinic, treatment was done at the second visit when the results of any diagnostic biopsies showed CIN 2+.
4.46
The results of the HPV triage base case showed that both technologies dominated standard colposcopy in 'see and treat' clinics (that is, they cost less and are more effective). In 'watchful waiting' clinics, DYSIS dominated standard colposcopy for low-grade lesion referrals and for all referrals combined, but had an incremental cost-effectiveness ratio (ICER) of £675 per QALY gained for high-grade lesion referrals compared with standard colposcopy. ZedScan I had an ICER of £272 per QALY gained for low-grade lesion referrals and £4,070 per QALY gained for high-grade lesion referrals. For all referrals, it had an ICER of £418 per QALY gained. Indirect comparisons suggest that ZedScan I always costs more but is more effective than DYSIS in both 'see and treat' and 'watchful waiting' clinics. The results of the HPV primary screening base case were similar to the HPV triage base case. The EAG highlighted that because the diagnostic accuracy of DYSIS and ZedScan I have not been compared directly, these results should be considered exploratory.
4.47
The number of treatments, biopsies and missed disease in each base case is shown in table 6. This table shows the cumulative occurrence of events over the lifetime of the modelled cohort, therefore an event can occur more than once per person. Because of their increased sensitivity, the adjunctive technologies are associated with less missed disease and so less cancers. However, they also have reduced specificity and result in more unnecessary diagnostic biopsies and treatments (except in 'watchful waiting' clinics).
Note: * Missed CIN 2+ refers to the number of CIN 2+ cases not detected by the technologies (colposcopy, DYSIS, ZedScan I) rather than cases not detected following referral for colposcopy. In the model people with high-grade cytology referrals have a diagnostic biopsy and are identified as CIN 2+ even if a colposcopic examination is incorrectly negative.
Abbreviations: CIN 2+, cervical intraepithelial neoplasia grade 2 or worse; HPV, human papilloma virus; LLETZ, large-loop excision of the transformation zone.
Scenario analyses
4.48
The following scenario analyses were done to explore the effect of alternative structural assumptions:
-
time horizon restricted to 1 screening interval (3 years)
-
adverse obstetric outcomes excluded
-
ZedScan I used in both diagnostic and treatment colposcopies.
4.49
When the time horizon was restricted to 3 years, colposcopy dominated (that is, it cost less and was more effective) both DYSIS and ZedScan I in most scenarios except for high-grade lesion referrals in HPV triage 'see and treat' clinics. In this scenario, DYSIS had an ICER of £236,692 saved per QALY lost and ZedScan I had an ICER of £84,045 saved per QALY lost. For HPV primary screening, the respective ICERs were £250,587 saved per QALY lost for DYSIS and £110,371 saved per QALY lost for ZedScan I. Colposcopy generally dominated because its higher specificity resulted in fewer treatments, and because people with untreated CIN (false negatives) did not go on to develop cancer within the 3‑year time horizon. The results of the model did not change substantially in the other scenario analyses.
Sensitivity analyses
4.50
The following inputs were changed in sensitivity analyses to explore the effect of parameter uncertainty:
-
diagnostic accuracy
-
costs of the technologies
-
costs of treatment and biopsies
-
characteristics of the population referred for colposcopy in HPV primary screening.
4.51
When the accuracy of colposcopy relative to ZedScan I was taken from Tidy et al. (2013), the incremental costs associated with ZedScan I compared with colposcopy increased, whereas the QALYs decreased. Under these assumptions ZedScan I became less cost effective than in the base case and it no longer dominated colposcopy in 'see and treat' clinics. Its highest ICER was £24,686 per QALY gained for high-grade lesion referrals in HPV primary screening 'watchful waiting' clinics.
4.52
The DYSIS results were sensitive to assumptions around reduced throughput and a consequent increase in cost per test because of its higher purchase price. When it was assumed that only 614 people per year were seen, it no longer dominated colposcopy in HPV primary screening 'watchful waiting' clinics and had an ICER of £270 per QALY gained for all referrals. None of the other sensitivity analyses changed the results substantially.
4.53
The ZedScan I results were sensitive to changes in the cost of diagnostic and treatment biopsies because of its increased sensitivity and lower specificity than colposcopy. When the cost of a diagnostic biopsy was increased to £102.72 and a treatment biopsy (LLETZ) to £490.89, ZedScan I no longer dominated colposcopy for low-grade lesion referrals and all referrals combined. Under these assumptions, its highest ICER was £6,709 for high-grade referrals to an HPV primary screening 'watchful waiting' clinic. None of the other sensitivity analyses changed the results substantially.