1 Provisional recommendations

1.1 Whole lymph node analysis using the RD-100i OSNA system is recommended as an option for detecting sentinel lymph node metastases in people with early invasive breast cancer who have a sentinel lymph node biopsy and in whom axillary lymph node dissection will be considered.

1.2 The Metasin test is not recommended for detecting sentinel lymph node metastases in people with early invasive breast cancer in routine clinical NHS practice. The Metasin test shows promise and published peer-reviewed research is recommended to demonstrate its clinical effectiveness.

2 The technologies

2.1 The RD-100i OSNA system (Sysmex UK) and the Metasin test (TIB MOLBIOL) are intraoperative molecular tests that are designed to indicate if cancer has spread to the lymph nodes in people diagnosed with breast cancer.

2.2 The Metasin test was developed within the NHS at the Princess Alexandra Hospital in Harlow, Essex. It was CE marked by TIB MOLBIOL in December 2012.

3 Clinical need and practice

The problem addressed

3.1 The intraoperative molecular tests (RD-100i OSNA system and Metasin test) are used during breast cancer surgery to detect the presence of 1 or 2 biological markers that are associated with metastatic spread in sentinel lymph node samples. The intention is that the test results are available during surgery and may be used to determine if other axillary lymph nodes should be removed at the same time as the initial tumour. This could avoid the need for a second operation and allow subsequent treatments such as chemotherapy to begin earlier.

3.2 The aim of this evaluation is to determine the clinical and cost effectiveness of using the RD-100i OSNA system and the Metasin test to detect metastases in the sentinel lymph nodes of patients having breast cancer surgery.

The condition

3.3 Breast cancer is one of the most common cancers in women in England and Wales; there are about 46,000 new cases diagnosed and 10,900 deaths recorded each year. Around 1 in 9 women develop breast cancer at some stage in their life. Most breast cancers develop in women over 50 years, but they can also occur in younger women and, in rare cases, in men. There are around 260 cases of breast cancer diagnosed and 68 deaths recorded in men in England and Wales each year. Around 11,000 women with newly diagnosed breast cancer need additional surgery to manage the spread of breast cancer to the lymph nodes every year. In a few people, the tumour has spread significantly within the breast or to other organs of the body at initial diagnosis. Also, some people who have been treated with curative intent subsequently develop either a local recurrence or metastases.

3.4 Breast cancer mainly spreads by local spread to nearby tissues, or by regional or distant spread through the circulatory or lymphatic system. Spread through the lymphatic system is of relevance for this evaluation. It occurs when cancer cells become detached from the main breast tumour and are then usually carried in the lymph to the axillary (armpit) lymph nodes, most likely the sentinel lymph nodes. The cancer cells can grow in the lymph nodes and cause swelling, although not all metastatic lymph nodes are morphologically abnormal. Lymph nodes are often used to stage cancer (measure the extent of the disease) because their function is to monitor lymph and trigger an immune response if a foreign substance is detected, so they are one of the earliest sites at which the spread of cancer can be detected.

3.5 The treatment of breast cancer can cause many side effects including pain, fatigue, reduced fertility and osteoporosis. A diagnosis of breast cancer and subsequent treatment can cause long-term anxiety, depression and isolation in both the patient and their relatives. Hair loss from cancer chemotherapy and radiotherapy, and changes to the body because of, for example, a mastectomy, are associated with social stigma and can significantly impact on quality of life and reduce self-esteem.

3.6 One side effect of lymph node surgery is lymphoedema, which is more likely after axillary lymph node dissection than sentinel lymph node biopsy. The most common symptom is swelling of the arm, hands and fingers on the side of the body that has been operated on, which can persist for months or years. Swelling can also affect the breast, chest and shoulder. Lymphoedema does not affect all people who have lymph node surgery but, in some people, it can develop soon after treatment or years later because of inflammation, infection and scarring.

3.7 Axillary lymph node dissections result in minor and major complications for 80% of women. Major complications include a 22% incidence of seromas (pockets of fluid under the skin), a 21% incidence of arm lymphoedema (general swelling) and a 14% infection rate. Other complications include pain, limited mobility, numbness and sensory loss. Sentinel lymph node biopsy is associated with a 7% incidence of lymphoedema, a 7% incidence of seroma and a 2% infection rate.

The diagnostic and care pathways

3.8 The current breast cancer care pathway is outlined in ‘Early and locally advanced breast cancer: diagnosis and treatment’ (NICE clinical guideline 80). This guideline recommends that ultrasound evaluation of the axilla (armpit) is done in all patients being investigated for early invasive breast cancer. If morphologically abnormal lymph nodes are identified, ultrasound-guided needle sampling is offered preoperatively.

3.9 For patients who have no evidence of abnormal lymph nodes on ultrasound images or aspiration cytodiagnosis, minimal surgery is performed to stage the axilla during breast surgery to confirm that the cancer has not spread. Sentinel lymph node biopsy, in which the first lymph nodes are removed to see if the cancer has spread from the original site, is the preferred technique. A radioactive solution and a blue dye are injected into the breast before surgery to help identify the sentinel lymph nodes during surgery. However, identifying the nodes during surgery can be difficult and there is a widely recognised learning curve for performing sentinel lymph node biopsy. The Royal College of Surgeons of England, Cardiff University and the Department of Health established a surgical training programme (NEW START) for performing the biopsy and set standards for surgeons to achieve a greater than 90% localisation rate (ability to locate sentinel nodes) and a less than 10% false negative rate. One study reported that the localisation rate achieved for sentinel lymph node biopsy was around 98% (Mansel et al. [2006]).

3.10 The fresh biopsy tissue from sentinel lymph node biopsy is currently analysed using postoperative histopathology, which involves slicing the lymph node into very thin sections. These tissue sections are then stained and viewed by a consultant histopathologist to identify any abnormalities in the tissue. There is a small risk that histopathological analysis misses a metastasis because only a few sections of the lymph node are examined, and metastatic foci are not evenly distributed through a lymph node so may not be present in sections that are examined. It is not clear how many sections from a lymph node are currently analysed in routine NHS practice. Results from histopathology usually take between 5 and 15 working days in the NHS. If the results are positive, the patient will have a second operation to remove the remaining lymph nodes (axillary dissection). This may be more technically challenging than performing the axillary dissection as part of the initial surgery.

3.11 Two pathological methods that can be used intraoperatively are frozen section and touch imprint cytology. Frozen section involves a section of the lymph node being snap-frozen, stained and sliced before being viewed by a consultant histopathologist. Touch imprint cytology involves the lymph node being sliced and the cut surface of the node imprinted on to a slide, which is then stained and viewed by a consultant histopathologist or cytopathologist. Both intraoperative pathological methods can be used to help determine if axillary lymph node dissection should be done at the same time as the first operation. Postoperative histopathology analysis is usually carried out on the remaining tissue to reduce the risk of a false negative result. However, in practice these intraoperative methods are rarely used because they have low accuracy and pathology resources are very limited within the NHS.

3.12 The RD-100i OSNA system and the Metasin test can be used to analyse either the whole lymph node or half of the lymph node with a follow-up histopathology on the remaining half to confirm the results. The decision on whether to analyse the whole lymph node is based on clinical judgement.

3.13 People who have macrometastases or micrometastases detected in their sentinel lymph node are regarded as lymph node-positive, and usually receive axillary lymph node dissection. People who have isolated tumour cells in their sentinel lymph node are regarded as lymph node-negative and will not receive axillary lymph node dissection.

3.14 All information on the sentinel nodes, axillary nodes and primary breast tumour is typically discussed at a multidisciplinary team meeting to determine the appropriate systemic adjuvant therapy. ‘Early and locally advanced breast cancer: diagnosis and treatment’ (NICE clinical guideline 80) recommends that adjuvant chemotherapy and radiotherapy should be started as soon as clinically possible within 31 days of completion of surgery in patients with early breast cancer having these treatments. The use of intraoperative molecular tests and a potential consequent reduction in the number of second operations performed may result in patients starting adjuvant therapy earlier.

4 The diagnostic tests

The interventions

RD-100i OSNA system

4.1 The RD-100i OSNA system analyses and amplifies mRNA from solubilised biopsy samples of sentinel lymph node tissue. It detects the level of expression of the cytokeratin-19 (CK19) gene, an epithelial marker associated with breast cancer. CK19 is normally not present in healthy lymph node tissue. The OSNA technology involves the homogenisation of sentinel lymph node tissue followed by analysis of the CK19 mRNA using reverse transcription loop mediated isothermal amplification (RT-LAMP) on the automated analyser within the RD-100i OSNA system. The system does not need the mRNA to be extracted from the tissue and purified before analysis. The expression level of CK19 mRNA correlates with the size of the metastatic foci. Since the metastatic foci may not be evenly distributed throughout the node, the system provides more accurate results if more of the node is analysed because there is less risk of tissue allocation bias (sample bias). Tissue allocation bias can occur when a half of the lymph node is analysed using an intraoperative test and the other half using histopathology, in situations when the metastasis is only contained in the tissue slices used for one method but not the other. The RD-100i OSNA system can be used with half of the lymph node (1 piece or alternate slices), allowing for the possibility of follow-up histopathology but potentially decreasing the accuracy of the results because of the increased risk of tissue allocation bias. The time to results depends on the number of lymph nodes analysed, but the test takes approximately 30 to 45 minutes. The RD-100i OSNA system result is expressed both quantitatively and qualitatively: − for lymph node negative test results; + for lymph nodes with a micro-metastatic tumour burden (that is, greater than 250 copies of CK19 mRNA/microlitre); and ++ for lymph nodes with a macro-metastatic tumour burden (that is, greater than 5000 copies of CK19 mRNA/microlitre). The analyser amplifies and detects the CK19 mRNA by using 6 different primers that have been specifically designed to avoid the amplification of CK19 pseudogenes or their transcripts because amplification of these would lead to false positive results.

4.2 The manufacturer estimates that 1% of breast tumours do not express CK19 mRNA and so, if cancer spreads to the lymph nodes from these tumours, CK19 mRNA will not be detected even though the lymph nodes are metastatic. Pre-screening of tumour biopsies for CK19 expression could be carried out before using the RD-100i OSNA system to reduce the small risk of false negative results for metastatic sentinel lymph nodes.

Metasin test

4.3 The Metasin test is an intraoperative molecular test developed within the NHS at the Princess Alexandra Hospital in Harlow, Essex. The test has similarities to a discontinued commercial test (Veridex GeneSearch breast lymph node [BLN]assay) and uses quantitative reverse transcription polymerase chain reaction (qRT-PCR) to detect 2 predictive markers of metastases, CK19 and mammaglobin. Mammaglobin is expressed mainly by breast epithelial cells and high levels of mammaglobin are associated with breast cancer. A reference gene (porphobilinogen deaminase; PBGD) is used to confirm the validity of the mRNA used in the test and 2 other controls (positive and negative) are also included. The test uses reagents that can be purchased from commercial suppliers and can be used on any PCR machine. The Metasin test uses different primer-probe combinations to detect the CK-19 and mammaglobin genes than the discontinued commercial test. It takes approximately 26 minutes to produce results after approximately 10 minutes for extracting and purifying mRNA from the tissue. The Metasin test is currently used in the NHS as an in-house test to analyse half of the lymph node. The results are confirmed by follow-up histopathology. The Metasin test could also be used as a replacement test for postoperative histopathology. The Metasin test was CE marked in December 2012 and is available from the manufacturer, TIB MOLBIOL (Berlin).

4.4 Pre-screening of tumour biopsies for CK19 mRNA and mammaglobin mRNA expression may be carried out before using the Metasin test because like the CK19 biomarker, mammaglobin is not expressed in all breast tumours. The proportion of breast cancer tumours that do not express mammaglobin mRNA is not known.

The comparator: postoperative histopathology

4.5 Postoperative histopathology is the usual approach used in the NHS, in which the sentinel lymph nodes are fixed in paraffin blocks, sliced very thinly to produce sections that are mounted on slides, stained and then examined under a microscope by a consultant histopathologist. The time to receive results from histopathology is usually between 5 and 15 working days in the NHS. People who have macrometastases (defined as tumour deposits in which at least 1 dimension is above 2 mm) or micrometastases (tumour deposits that are only discernible microscopically and measure greater than 0.2 mm but have no dimension greater than 2 mm) detected in their sentinel lymph node are regarded as lymph node-positive, and will usually receiveaxillary lymph node dissection. People who have isolated tumour cells in their sentinel lymph node are regarded as lymph node-negative and will not receive axillary lymph node dissection.

4.6 The wait for histopathology test results can cause anxiety for patients and can also lead to the patient having a second operation to remove all of the relevant axillary lymph nodes if the test result is positive. This second operation can be more difficult and result in a higher risk of complications because it will involve operating on the same area of the breast and armpit as the first operation.

4.7 There are several levels of histopathology that can be performed. In one-level histopathology, 1 section of the lymph node is examined, in three-level, 3 sections and, in five-level histopathology, 5 sections. The level of histopathology used may affect the accuracy of histopathology because of the risk of tissue allocation bias. There is also uncertainty about the accuracy of histopathology associated with the different staining techniques used, the thickness of the section examined and the experience of the pathologist reading the section slides.

5 Outcomes

The Diagnostics Advisory Committee (appendix A) considered evidence from a number of sources (appendix B).

How outcomes were assessed

5.1 The assessment was performed by an external assessment group and consisted of a systematic review of the evidence on test performance and clinical effectiveness data for the RD-100i OSNA system and the Metasin test compared with postoperative histopathology.

5.2 The outcome measures relevant to test performance included diagnostic test accuracy, test failure rate, discordant test results, and time to test result.

5.3 The outcome measures relevant to clinical effectiveness included: patient anxiety associated with the waiting time for results and with not knowing what the extent of surgery would be before the operation; number of repeat operations (excluding those for re-excision of positive margins); time to start and nature of adjuvant therapy; morbidity and mortality from biopsies, axillary dissections, first and second operations and treatment of cancer; and adverse events from false test results, including patient distress and sequelae.

5.4 No study was excluded on the basis of intervention, population, comparator or outcome, provided it appeared relevant to the scope of the evaluation.

Clinical effectiveness

RD-100i OSNA system

5.5 The external assessment group included 16 studies in the systematic review that investigated the performance of the RD-100i OSNA system in detecting metastases in the sentinel or axillary lymph nodes,although 2 of these studies reported the same trial. Fourteen of these 16 studies reported test accuracy as an outcome and 2 of these 14 studies also reported time to analysis. An additional observational study reported time to analysis as an outcome alone. One other study reported time in operating theatre, days in hospital, costs and postoperative complications. No data were found for the clinical outcomes of patient anxiety and number of repeat operations.

5.6 Eleven of the 16 studies were single-gate studies in which people with unknown disease status were assessed using both the intraoperative test and the reference standard (histopathology) to compare the results of the 2 tests and confirm diagnosis. The remaining studies comprised 4 cohort studies and 1 observational study. In the cohort studies, different patient samples were used for each test, which enabled whole-node analysis with the RD-100i OSNA system.

5.7 There was heterogeneity across studies in their definitions of histopathology. Three studies reported using five-level histopathological analysis to detect metastases in the node. Two other studies used five-level histopathological analysis during the validation phase of the study and one-level histopathological analysis during routine use. Five studies reported using three-level histopathological analysis and 1 other study reported using one-level histopathological analysis. The level of histopathological analysis used in the remaining studies is unclear. These varying levels of histopathological analyses may impact the accuracy of histopathology because five-level analysis may be more likely to detect micrometastases than one-level analysis. In addition, depending on the level of analysis used some studies did not reflect current NHS practice for histopathological analysis.

5.8 The external assessment group found that, in all of the studies, there was a lack of detail on patient recruitment and patient characteristics so the risk of bias in the studies is unclear. Spectrum bias may arise if the severity of the cancer is greater in one study population than another because it is more likely that a test will detect metastases in people with severe disease and it will therefore appear to have a higher sensitivity. In addition, study populations may vary depending on the upstream diagnostic pathway because some clinics may be better at detecting metastases with fine needle aspiration cytology so sentinel lymph node biopsy would not be needed in some cases. This could result in a patient population with higher levels of micrometastases than macrometastases receiving sentinel lymph node biopsy and so the sensitivity of the intraoperative test or histopathology may appear lower in this population. There was also a lack of information on sampling methods so there was no evidence of sample replicates and reproducibility for molecular analysis.

5.9 The assessment of test accuracy for the RD-100i OSNA system was hindered by tissue allocation bias and by comparison with an inconsistent reference standard (different levels of histopathology). In some studies, discordant samples were further analysed (by extensive histopathology, molecular analysis or Western blotting) and attributed to tissue allocation bias. In these cases, the test accuracy analyses could be adjusted by excluding the data from the discordant samples. No adjustment could be made for the varying levels of histopathology used across the studies.

5.10 The range of estimates for sensitivity and specificity by patient before adjustment for tissue allocation bias from the studies were 77.8–80.0% and 88.0–97.2% respectively. The range of estimates for sensitivity and specificity by patient after adjustment for tissue allocation bias from the studies were 89.8–100% and 93.3–97.2% respectively.

5.11 The external assessment group performed meta-analysis (bivariate method) of diagnostic test accuracy from studies that reported the numbers of true positives, true negatives, false negatives and false positives in the text (or sufficient data for these test statistics to be calculated). Five studies were included that did not adjust for tissue allocation bias and 3 studies were included that did adjust for tissue allocation bias. The sensitivity and specificity by patient without adjustment for tissue allocation bias were 84.5% (95% confidence interval [CI] 74.7% to 91.0%) and 91.8% (95% CI 87.8% to 94.6%) respectively. The sensitivity and specificity by patient with adjustment for tissue allocation bias were 91.3% (95% CI 83.6% to 95.6%) and 94.2% (95% CI 91.2% to 96.2%) respectively.

5.12 Four studies reported the time to analysis by the RD-100i OSNA system. The estimates for time to analysis from the studies ranged from less than 30 minutes to 39.6 minutes for 1 node and increased by 5–10 minutes per additional node analysed. One study reported that the longest and most variable time period corresponded to transporting the node from the operating room to the pathology department. The least variable time period corresponded to the homogenisation of tissue, preparation of the diluted sample and PCR amplification.

5.13 One other study compared the number of postoperative complications between the use of histopathology and the use of RD-100i OSNA to analyse lymph node samples. The aim of this study was to analyse the economic costs of intraoperative testing with the RD-100i OSNA system compared withpostoperative histopathology. Overall, the patients having intraoperative lymph node testing using the RD-100i OSNA system experienced fewer postoperative complications than patients having postoperative histopathology analysis, although the only major complication reported occurred in the OSNA group (no further details of the major complication were reported). This study was conducted in Spain and the assessment group stated that it was uncertain to what extent the study findings may be generalisable to the UK owing to possible differences in clinical practice.

Metasin test

5.14 Two draft unpublished non-peer reviewed studies that investigated the performance of the Metasin test in detecting metastases in the lymph nodes of breast cancer patients were included in the systematic review. The results of one of the studies are considered academic in confidence.

5.15 The study by Sundaresan et al. (unpublished) was designed as a single-gate study in which individuals were assessed by both the intraoperative test and histopathology. It reported test accuracy and time to analysis as outcomes. The study compared the accuracy of the Metasin test to histopathology. It reported using three-level histopathological analysis to detect metastases in the nodes but did not report details of patient recruitment or patient characteristics so the risk of bias in the study is unclear. There was also a lack of information about sample replicates and reproducibility for molecular analysis, and the external assessment group did not consider the study to meet the STARD (Standards for the reporting of diagnostic accuracy studies) criteria.

5.16 Similar to the studies investigating the performance of the RD-100i OSNA system, one of the main issues with assessing the test accuracy of the Metasin test was tissue allocation bias. No discordant analyses were performed in the study but discordant results were reported. Discordant results were observed in 56 out of 1265 patients (4.4%): 36 patients received a positive Metasin test and negative histopathology, and 20 patients received a negative Metasin test and positive histopathology. The authors considered that tissue allocation bias was responsible for the discordant results although no evidence or analysis was presented in the study.

5.17 The estimates for test accuracy by patient without adjustment for tissue allocation bias from Sundaresan et al. were 92% sensitivity (95% CI 89% to 95%) and 97% specificity (95% CI 95% to 97%). No meta-analysis was performed by the external assessment group for the 2 studies investigating the performance of the Metasin test because at least four studies are needed in order to use the bivariate method of meta-analysis. The accuracy values from Sundaresan et al. were used in the cost-effectiveness analyses.

Cost effectiveness

5.18 The external assessment group identified 2 studies that were considered relevant for the systematic review on the cost effectiveness of intraoperative tests for the detection of sentinel lymph node metastases. Both studies were single centre observational studies that compared an intraoperative test with histopathology for assessing sentinel lymph node biopsy. One study was based in the UK and assessed the GeneSearch BLN assay and the other study was conducted in Spain and evaluated the RD-100i OSNA system. Both studies found their respective intraoperative tests to be cost effective compared with histopathology, with both assays being cost saving while reducing theatre time and length of hospital stay. Neither study considered outcomes beyond the diagnostic phase. The UK study provided evidence on resource use and costs of intraoperative testing in the UK but evaluated the GeneSearch BLN assay, which has been withdrawn from the market. The Metasin test uses the same markers as the GeneSearch BLN assay, CK19 and Mammaglobin, but different primer-probe combinations, so it is expected to perform differently to the GeneSearch BLN assay. Therefore, this UK study is not directly relevant to this evaluation. The study conducted in Spain also provided evidence on resource use and costs but was limited in the extent to which it was generalisable to the UK. Therefore, the external assessment group did not consider the study directly relevant to this evaluation.

5.19 The external assessment group performed an economic analysis to assess the cost effectiveness of using intraoperative tests to detect sentinel node metastases compared with using histopathology. The economic model was divided into 2 separate sections (diagnostic and management) to encompass both the short-term and long-term outcomes of intraoperative testing. The costs were evaluated from the perspective of the NHS and personal social services. Outcomes were expressed as quality-adjusted life years (QALYs). Both costs and outcomes were discounted using a 3.5% annual discount rate.

5.20 The external assessment group developed a decision tree to model the short-term diagnostic outcomes outlined in the decision problem. People enter the model as patients who have sentinel lymph node biopsy performed during their initial tumour removal. The model then splits into 3 different diagnostic strategies: postoperative histopathology (current practice) alone, intraoperative testing alone and intraoperative testing combined with postoperative histopathology confirmation. In the model, patients who are diagnosed with sentinel lymph node metastases receive axillary lymph node dissection, either during the same operation as their sentinel lymph node biopsy if intraoperative testing is used or during a second operation if postoperative histopathology is used.

5.21 Once the diagnostic subgroups have been identified (true positive sentinel lymph node, false positive sentinel lymph node, true negative sentinel lymph node and false negative sentinel lymph node), the model moves into the management pathway and the subgroups are separated based on whether or not patients receive an axillary lymph node dissection. This section of the model calculates the long-term outcomes for each subgroup and at this point, a discrete event simulation model, previously developed at the University of Sheffield (ScHARR-TAG), was used to model the natural disease history of the patients once their outcome from the diagnostic decision tree had been determined. In the model, after surgery, patients receive adjuvant therapy comprising chemotherapy and hormonal therapy (where appropriate) for patients diagnosed with metastases and hormonal therapy alone (where appropriate) for patients diagnosed without. After adjuvant therapy, patients can move into a disease-free state or, experience locoregional or metastatic relapse. Patients can also move between these states.

5.22 The meta-analysed accuracy values without adjustment for tissue allocation bias for the RD-100i OSNA system and the accuracy values from one of the unpublished papers for the Metasin test were used in the base case. The node positive prevalence was set at 20% in the base case, in line with the studies in the clinical systematic review.

5.23 Unit costs for the intraoperative tests were taken from the sponsors of the technologies and the cost of histopathology was based on data provided by the NHS Technology Adoption Centre. The unit cost of the RD-100i OSNA system was £350 and the unit cost of histopathology was £472. The surgery costs were mainly based on NHS reference costs and the costs of short-term adverse events were taken from Jeruss et al. (2006). The costs associated with lymphoedema were obtained from the Sheffield Lymphoedema Service and the length of additional hospital stay was calculated from a study by the York Health Economics Consortium. The costs of additional time in surgery were estimated from a study by Ng et al. (2011) and from the report by York Health Economics Consortium. All costs were updated to 2010 levels.

5.24 The QALY decrement associated with a 2-week wait for histopathology results was calculated by the external assessment group to be 0.019 (undiscounted). For patients having a separate second operation, the disutility was estimated as 0.03.

5.25 The external assessment group considered the results of the cost-effectiveness analyses for the Metasin test to be illustrative because of the high levels of uncertainty associated with the unpublished evidence base relating to the diagnostic accuracy of the test.

5.26 The cost-effectiveness analyses of the short-term outcomes examined the diagnostic accuracy of the intraoperative tests compared with postoperative histopathology, and the disutility of waiting for histopathology results and having a second operation. For strategies that did not involve histopathology, the utility was 1 because there was no wait for test results or any second operations directly resulting from the intraoperative test. Only short-term QALY gains were included in these analyses.

5.27 Using the NHS reference costs in the short-term model, whole-node OSNA analysis and half-node OSNA analysis dominated histopathology analysis because they were less costly and more effective. Whole-node OSNA analysis also dominated half-node OSNA analysis. It was estimated that 4.1% of the 76.5% of patients who received a negative test result from half-node OSNA analysis would end up with a positive result while waiting for confirmation by histopathology analysis, compared with 20% of patients who would receive a positive result using postoperative histopathology analysis alone.

5.28 In the short-term model, whole-node and half-node Metasin analyses dominated histopathology analysis because they were less costly and more effective. Whole-node Metasin also dominated half-node Metasin analysis. It was estimated that, of the 78.5% of patients who received a negative result by half-node Metasin analysis, 1.9% would receive a positive result while waiting for confirmation by histopathology analysis, compared with 20% of patients who would receive a positive result using postoperative histopathology analysis alone.

5.29 The cost-effectiveness analyses of the long-term outcomes examined all the costs and benefits from accurate diagnosis through to improved patient management. In these analyses, the diagnostic strategies were ordered by the number of QALYs associated with them, with whole-node OSNA analysis producing the least QALYs (9.22) and postoperative histopathology producing the most QALYs (9.32). The QALY difference is equal to 0.1 (that is, equivalent to 5 weeks of full-health life) and this difference occurs because the higher accuracy of histopathology assumed in the model leads to more correct diagnoses and appropriate subsequent treatment.

5.30 Using the NHS reference costs in the long-term model, the incremental cost-effectiveness ratio (ICER) was £4324 saved per QALY lost for whole-node OSNA analysis compared with histopathology analysis and £24,863 saved per QALY lost for whole-node Metasin analysis compared with histopathology analysis. The ICERs for whole-node analysis compared with histopathology analysis suggest that the intraoperative testing strategies save money but that there is a loss of approximately 0.1 QALY, compared with histopathology analysis.

5.31 In the modelling, histopathology analysis was assumed to be the ‘gold standard’ and was given an accuracy of 100% sensitivity and 100% specificity. The level of uncertainty in this assumption is unclear and the estimated ICERs may change depending on the assumed absolute accuracy of histopathology.

5.32 The sensitivity and specificity of OSNA analysis were changed to use values from studies that adjusted for tissue allocation bias (Frere Belda et al. [2012], Snook et al. [2011] and Khaddage et al. [2011]). Using NHS reference costs, the ICER was £9493 saved per QALY lost for whole-node OSNA analysis compared with histopathology analysis using accuracy values from the Frere Belda et al. study (91.4% sensitivity and 93.3% specificity) and £8840 saved per QALY lost for whole-node OSNA analysis compared with histopathology analysis using values from the Snook et al. study (89.8% sensitivity and 94.5% specificity). However, using the higher accuracy values from the Khaddage et al. study (100% sensitivity and 97.2% specificity) resulted in ICERs for whole-node OSNA analysis that dominated both half-node OSNA analysis and histopathology analysis.

5.33 The change in ICERs when accuracy values adjusted for tissue allocation bias were used showed that test accuracy has a direct impact on the cost effectiveness of the tests. Threshold analysis was used to investigate sensitivity by increasing sensitivity over a range of 70–100% while specificity was held constant. The opposite was also performed to investigate specificity. These analyses were conducted on the results for the whole-node OSNA analysis. Short-term utility results were not reported as the utility of OSNA was not affected by the accuracy of the test.

5.34 The results of the threshold analysis for the long-term results showed that, when the sensitivity of OSNA increased (and specificity was kept at the 91.8% base-case value), the saving per QALY lost increased when comparing whole-node OSNA with histopathology. When comparing OSNA with histopathology, the ICERs for OSNA ranged from £2,119 saving per QALY lost when OSNA had a sensitivity of 70% to £14,193 saved per QALY lost when OSNA had 95% sensitivity. At 100% sensitivity, OSNA dominated histopathology, having more QALYs gained and lower costs.

5.35 For specificity, the long-term cost of OSNA decreased and the QALY gain increased as the specificity increased. At a specificity of 70%, whole-node OSNA analysis was dominated by histopathology analysis because it was more expensive and had fewer QALYs. The largest ICER for OSNA was £8,430 saved per QALY lost compared with histopathology, when whole-node OSNA analysis had 100% specificity (and sensitivity was kept at the base case value of 94.5%).

5.36 Overall, the threshold analyses suggested that, if the true values of sensitivity and specificity for whole-node OSNA analysis lie within the range of 90–100%, the cost effectiveness of whole-node OSNA analysis may increase. The results also imply that changes to specificity may have more of an impact in the short-term than the long-term, but that changes to sensitivity may have a much greater impact on the long-term cost effectiveness.

5.37 Sensitivity analysis was also conducted on the effect of prevalence of sentinel lymph node metastases in the patient population. When the prevalence was reduced to 10%, histopathology analysis dominated half-node OSNA analysis and had an ICER of £2,626 per QALY gained compared with whole-node OSNA analysis. When the prevalence was increased to 40%, half-node OSNA analysis dominated histopathology analysis and had an ICER of £2208 per QALY gained compared with whole-node OSNA analysis.

5.38 Changing individual costs and utility parameter values in the short-term or long-term sections of the model had very little impact on the overall cost-effectiveness results. This highlighted the importance of the diagnostic accuracy of the tests because this was the most influential parameter.

6 Considerations

6.1 The Committee considered the heterogeneity and uncertainty in the studies. The Committee heard from the clinical specialists on the Committee that histopathology practices can vary between hospitals in the UK. It also heard that, in routine clinical practice, the standard of histopathology may be different to that in the studies. It concluded that the well-conducted research standard histopathology described in the studies may not be representative of the histopathology conducted in routine clinical NHS practice and that there was variation in how histopathology was performed across the NHS.

6.2 The Committee considered the unpublished non-peer reviewed evidence for the Metasin test. The Committee acknowledged the findings of the 2 draft unpublished studies but noted that the studies, one of which was designated as academic in confidence by the sponsor of Metasin, had not been peer reviewed and therefore the results should be interpreted with caution. The Committee concluded that the test appeared promising but that there was too much uncertainty associated with the non-peer-reviewed evidence to recommend use of the test in routine NHS practice. The Committee was encouraging of further peer-reviewed research on this promising test.

6.3 The Committee considered the 20% prevalence of sentinel lymph node metastases in the patient population used in the base case for the cost-effectiveness analyses. It heard from the clinical specialists that the prevalence was likely to be higher than 20%, and possibly around 30%. The Committee noted the sensitivity analysis, which showed that an increase in the prevalence of sentinel lymph node metastases increased the cost effectiveness of the RD-100i OSNA system (see section 5.37). Therefore, the Committee concluded that using the RD-100i OSNA system was likely to be more cost effective than the base-case ICERs suggested.

6.4 The Committee considered the accuracy of histopathology. The Committee heard from a specialist committee member that the sensitivity of histopathology in routine UK practice was likely to be lower than 100%, which was what was assumed in the cost-effectiveness analysis. The committee heard that the main source of inaccuracy in histopathology is tissue allocation bias, but that other sources of inaccuracy are possible, including user variability. The Committee noted that it was unlikely that a macrometastasis would be missed if the current histopathology guidelines were followed but that it was possible that micrometastases could be missed. However, the clinical significance of this is not known. The Committee concluded that it was very difficult to determine the absolute accuracy of histopathology because of the nature of the technique and the inherent variation from qualitative judgement of different pathologists. However, it was likely to be lower than 100% in routine clinical practice.

6.5 The Committee also considered the assumption in the modelling that histopathology was 100% accurate following discussions about the accuracy of histopathology in clinical practice. The Committee concluded that the accuracy of histopathologyin any setting could not be 100% because time and resources did not allow every slice of a node to be analysed for metastases. The Committee therefore concluded that use of the RD-100i OSNA system was likely to be more cost effective than the base-case ICERs suggested.

6.6 The Committee considered that whole-node analysis had more benefit than half-node analysis and histopathology because there was no risk of tissue allocation bias when the whole node was analysed. The Committee considered it acceptable for half-node OSNA analysis with postoperative histopathology confirmation to be used while the RD-100i OSNA system is being validated locally but recommended that, after validation, whole-node OSNA analysis should be fully implemented in local clinical practice.

6.7 The Committee discussed the scenario in which intraoperative analysis of sentinel lymph nodes is not performed and when a second operation is needed following identification of lymph node metastases with histopathology. The Committee noted that, in the economic model, the disutility due to a second operation was 0.03. The Committee heard from the clinical specialists that a second breast operation is technically more difficult because of disrupted tissue structure. It also heard that a second operation had an increased risk of complications compared with breast surgery being done for the first time. The Committee also heard from a patient expert that patients generally found the prospect of having a second operation very worrying and that the option of not having to have a second operation was an important consideration for patients. The Committee considered that the disutility from the second operation was likely to be larger than that assumed in the economic model and the cost effectiveness of intraoperative testing was therefore likely to be underestimated in the base case. The Committee concluded that intraoperative analysis of sentinel lymph nodes had considerable advantages over traditional histopathology testing and had the potential to reduce both clinical complications, and patient anxiety and distress.

6.8 The Committee considered the impact on patients of not knowing what the extent of their surgery would be before the operation. The Committee heard from a patient expert the importance of patients being informed before surgery about the different types of surgery they may have, depending on the results of the intraoperative test. The Committee noted that there was strong patient preference for all procedures to be done in a single operation and noted that, in routine clinical practice, the uncertainty about the extent of surgery would be explained to the patient.

6.9 The Committee considered the cost-effectiveness analyses of the RD-100i OSNA system. The Committee acknowledged that the model contained several assumptions that could potentially increase the uncertainty of the cost-effectiveness analysis. However, the Committee considered that it was appropriate to assume a higher prevalence than that used in the base case (see section 6.3), an accuracy value of less than 100% for histopathology(see sections 6.4 and 6.5), and that the technical difficulty and complications associated with a second operation may have been underestimated (see section 6.7). The Committee concluded that, taking these considerations together, the RD-100i OSNA system was likely to be more cost effective than the base-case ICERs suggested and that it was likely that the RD-100i OSNA system was equally or more cost effective than postoperative histopathology. The Committee also concluded that the substantial patient benefits associated with using the RD-100i OSNA system and the strong patient preference for all procedures to be done in a single operation were not fully captured in the cost-effectiveness analyses. The Committee therefore concluded that the RD-100i OSNA system would represent a cost-effective use of NHS resources if used as an option for detecting sentinel lymph node metastases in people with early invasive breast cancer who have a sentinel lymph node biopsy and in whom axillary lymph node dissection is being considered.

6.10 The Committee heard from clinical specialists about therecent publication of the Z0011 trial, which reported no improvement in survival after axillary lymph node dissection in women who received a positive result for lymph node metastases. The Committee heard from the clinical specialists that there was concern that the Z00011 trial was under-powered and there was no radiotherapy quality assurance programme to monitor the dose of radiation given.  The Committee concluded that using intraoperative tests for detecting sentinel lymph node metastases offered substantial benefits to patients in current clinical practice and that these benefits were likely to remain until the uncertainty in the effectiveness of performing axillary lymph node dissection is resolved.

6.11 The Committee considered that a quality assurance scheme for the use of the RD-100i OSNA system and any other relevant intraoperative tests is needed.

6.12 The Committee considered that biomedical scientists can carry out testing using the RD-100i OSNA system or the Metasin test, although a higher level of molecular biology expertise may be needed for the Metasin test. This expertise may not be available in all hospitals performing breast surgery, particularly once laboratory services are centralised. The Committee noted that a biomedical scientist may need to travel to the site of surgery to perform the intraoperative test, which would result in a loss of resources from the laboratory during that time. The Committee also considered that there is a shortage of pathologists in the NHS and that using these intraoperative molecular tests may free pathology resources for other uses in the NHS.

6.13 The Committee considered that, for the efficient use of intraoperative testing, surgical theatre lists may need to be carefully scheduled and multiple analysers may be needed for sentinel lymph node testing if breast operations occur in parallel. The Committee heard from clinical specialists that the sentinel lymph node biopsy can be performed first so that the lymph nodes can be analysed using the RD-100i OSNA system while the primary tumour is being removed. This prevents the time in surgery being significantly increased by use of an intraoperative test.

6.14 Clinical specialists on the Committee also informed the Committee that fewer sentinel lymph node biopsies may be performed during an operating theatre list to allow time to perform axillary lymph node dissections when the intraoperative test results are positive. However, theatre time is made available in the subsequent weeks because the patients are not returning for a second operation, which would occur if patients had to wait for postoperative histopathology results. The Committee concluded that any disruption to theatre lists can be overcome with careful planning and scheduling.

7 Proposed recommendations for further research

7.1 NICErecommends that a national registry is developed to collect data on the use of the RD-100i OSNA system in detecting sentinel lymph node metastases during breast cancer surgery. It is alsorecommends that data on all patients having whole lymph node analysis by the RD-100i OSNA system should be submitted to this registry when it is available.

8 Implementation

NICE intends to develop tools, in association with relevant stakeholders, to help organisations put this guidance into practice.

9 Related NICE guidance

Published

Eribulin for the treatment of locally advanced or metastatic breast cancer. NICE technology appraisal 250 (2012).

Breast reconstruction using lipomodelling after breast cancer treatment. NICE interventional procedure guidance 417 (2012).

Fulvestrant for the treatment of locally advanced or metastatic breast cancer. NICE technology appraisal 239 (2011).

Early and locally advanced breast cancer: diagnosis and treatment. NICE clinical guideline 80(2009).

Advanced breast cancer: diagnosis and treatment. NICE clinical guideline 81 (2009).

Image-guided radiofrequency excision biopsy of breast lesions. NICE interventional procedure guidance 308 (2009).

Familial breast cancer: the classification and care of women at risk of familial breast cancer in primary, secondary and tertiary care. NICE clinical guideline 41 (2006).

Improving outcomes in breast cancer. Cancer Service guidance (2002).

Under development

NICE is developing the following guidance (details available from www.nice.org.uk):

Familial breast cancer: the classification and care of women at risk of familial breast cancer in primary, secondary and tertiary care – update of clinical guideline 41. NICE clinical guideline. Anticipated publication date to be confirmed.

Gene expression profiling and expanded immunohistochemistry tests to guide selection of chemotherapy regimes in breast cancer management: MammaPrint, Oncotype DX, IHC4 and Mammostrat. NICE diagnostics guidance. Anticipated publication date to be confirmed.

10 Review

NICE will update the literature search at least every 3 years to ensure that relevant new evidence is identified. NICE will contact product sponsors and other stakeholders about issues that may affect the value of the diagnostic technology. NICE may review and update the guidance at any time if significant new evidence becomes available.

Adrian Newland
Chair, Diagnostics Advisory Committee
March 2013

Appendix A: Diagnostics Advisory Committee members and NICE project team

Diagnostics Advisory Committee

The Diagnostics Advisory Committee is an independent committee consisting of 22 standing members and additional specialist members. A list of the Committee members who participated in this assessment appears below.

Standing Committee members

Professor Ron Akehurst
Professor in Health Economics, School of Health & Related Research (ScHARR), University of Sheffield

Dr Trevor Cole
Consultant Clinical and Cancer Geneticist, Birmingham Women's Hospital

Dr Paul Collinson
Consultant Chemical Pathologist, St George's Hospital

Dr Sue Crawford
General Practitioner (GP) Principal, Chillington Health Centre

Professor Ian A Cree
Senior Clinical Advisor, NIHR Evaluation Trials and Studies Coordinating Centre, University of Southampton

Professor Erika Denton
National Clinical Director for Imaging, Department of Health, Honorary Professor of Radiology, University of East Anglia and Norfolk & Norwich University Hospital

Dr Simon Fleming
Consultant in Clinical Biochemistry and Metabolic Medicine, Royal Cornwall Hospital

Professor Lisa Hall
Professor of Analytical Biotechnology, University of Cambridge

Professor Noor Kalsheker
Professor of Clinical Chemistry, University of Nottingham

Dr Mark Kroese
Vice Chair, Diagnostics Advisory Committee and Consultant in Public Health Medicine, PHG Foundation, Cambridge and UK Genetic Testing Network

Professor Adrian Newland
Chair, Diagnostics Advisory Committee

Dr Richard Nicholas
Consultant Neurologist; Honorary Senior Lecturer, Heatherwood and Wexham Park Hospitals

Ms Margaret Ogden
Lay representative

Dr Diego Ossa
Director of Market Access Europe, Novartis Molecular Diagnostics

Mr Stuart Saw
Director of Finance, North East London and the City PCTs

Professor Mark Sculpher
Professor of Health Economics at the Centre for Health Economics, University of York

Dr Steve Thomas
Consultant Vascular and Cardiac Radiologist at Sheffield Teaching Hospitals Foundation Trust

Mr Paul Weinberger
CEO, DiaSolve Ltd, London

Mr Christopher Wiltsher
Lay representative

Specialist Committee members

Ms Marie Hecht
Lay member

Professor Ian Kunkler
Consultant and Honorary Professor in Clinical Oncology, Edinburgh Cancer Centre

Professor Graham T Layer
Consultant surgeon and Director of Professional Standards, Royal Surrey County Hospital NHS Foundation Trust

Mr Simon Pain
Consultant Breast and Endocrine Surgeon, Norfolk and Norwich University Hospital

Mr Zenon Rayter
Consultant surgeon, Bristol Royal Infirmary

Dr Deidre Ryan
Consultant pathologist, Barts Health NHS Trust

Dr Abeer Shaaban
Consultant pathologist, St James's University Hospital

NICE project team

Each diagnostics assessment is assigned to a team consisting of a Technical Analyst (who acts as the topic lead), a Technical Adviser and a Project Manager.

Sarah Byron
Topic Lead

Pall Jonsson
Technical Adviser

Appendix B: Sources of evidence considered by the Committee

The diagnostics assessment report was prepared by Peninsula Technology Assessment Group (PenTAG):

• Huxley N, Jones-Hughes T, Coelho H et al. A systematic review and economic evaluation of intraoperative tests (RD-100i OSNA system and Metasin test) for detecting sentinel lymph node metastases in breast cancer. (2012)

Registered stakeholders

The following organisations accepted the invitation to participate in this assessment as registered stakeholders. They were invited to attend the scoping workshop and to comment on the diagnostics assessment report.

Manufacturers/sponsors

• Sysmex UK
• TIB MOLBIOL
• Princess Alexandra Hospital Trust

Professional/specialist and patient/carer groups:

• Beatson West of Scotland Cancer Care
• Breakthrough Breast Cancer
• Breast Unit, Royal Surrey County Hospital
• Breast Cancer Care
• Department of Health
• Leeds and Wakefield NHS Trust
• NCRI Breast Clinical Studies Group
• NHS Bristol
• NHS Technology Adoption Centre
• Peony Breast Care Unit
• Royal College of Nursing
• Royal College of Pathologists
• Royal College of Physicians
• The Royal Marsden NHS Foundation Trust