Advice
Evidence review
Evidence review
Clinical and technical evidence
Regulatory bodies
A search of the Medicines and Healthcare Products Regulatory Agency website revealed no manufacturer Field Safety Notices or Medical Device Alerts for this device. No reports of adverse events were identified from a search of the US Food and Drug Administration (FDA) database: Manufacturer and User Device Facility Experience (MAUDE).
Clinical evidence
A literature search and information from the manufacturer identified 12 publications. Three diagnostic test accuracy studies were published in full and are described in detail in this briefing (Harrington et al. 2015; Biswas et al. 2014; Anderson et al. 2014). The other 9 articles were available only as abstracts or conference proceedings and brief summaries are included (Ashman et al. 2013; Beucher et al. 2014; Buchan et al. 2013; Chapin et al. 2013; McAulay et al. 2014; Mortensen et al. 2014; Perry et al. 2014; Porter et al. 2014; Rebec et al. 2013).
Fully‑published studies
The study by Harrington et al. (2015) was a cross‑sectional multicentre study funded and conducted by the manufacturer, BD Diagnostic Systems. The objective was to evaluate the BD MAX Enteric Bacterial Panel (EBP) compared with routine diagnostic culture for detecting Salmonella, Shigella, Campylobacter (coli and jejuni) and Shiga‑like toxin genes stx1 and stx2 from Shiga‑toxin‑producing organisms (bacterial species not specified). Testing took place in 6 clinical laboratories in the USA and 1 in Canada, using samples collected from sites in the USA, Canada and Mexico.
The study included 4242 soft or diarrhoeal stool specimens from adults and children whose samples were submitted for routine analysis for bacterial stool pathogens. Of these, 3457 were prospective samples, 1345 of which were unpreserved and 2112 were preserved. The remaining 785 samples were retrospective; 321 were unpreserved and 464 were preserved. To increase the number of positive samples of the rarer Shiga‑like toxin genes (stx1 and stx2) from Shiga‑toxin‑producing organisms (bacterial species not specified), retrospective samples were included. These samples had previously been identified as positive for Shiga toxin using enzyme immunoassay (EIA) methods. Where possible these were paired with 1 or more culture‑negative specimens from the same time period.
The primary outcome was positive percentage agreement (PPA) and negative percentage agreement (NPA) with culture or EIA results, rather than sensitivity and specificity because bacterial culture is not a true gold standard. Samples that gave different results between the BD MAX EBP and culture or EIA were tested with an alternative polymerase chain reaction (PCR) method (PCR methods have greater sensitivity than culture). This PCR result was then used as the reference standard where BD MAX EBP was positive but culture or EIA negative.
The overview and results of this study are summarised in tables 1 and 2 respectively. For prospective samples, the PPA ranged from 80% to 100% and the NPA ranged from 98.2% to 99.7%. For retrospective samples, the PPA ranged from 97% to 100% and the NPA ranged from 99.5% to 100%.
Of discrepant results, 6 were positive by culture but not by the BD MAX EBP (1 Campylobacter, 5 Salmonella). A number of samples were positive by BD MAX EBP but negative by culture or EIA (see table 2 for details).
The authors concluded that the BD MAX EBP showed superior sensitivity compared with conventional methods and excellent specificity for the detection of bacterial pathogens in stool specimens.
Table 1 Overview of the Harrington et al. (2015) study
|
Study component |
Description |
|
Objectives/hypotheses |
To evaluate the BD MAX EBP assay compared with routine diagnostic culture for detecting Salmonella spp., Shigella spp., C. coli, and C. jejuni and enzyme immunoassay for Shiga toxins. |
|
Study design |
Cross‑sectional, multicentre study. |
|
Setting |
Clinical laboratories, 6 in the USA and 1 in Canada. Study developed and supported by the manufacturer. |
|
Inclusion/exclusion criteria |
Soft or diarrhoeal stool specimens from adults and children submitted for routine analysis for bacterial stool pathogens. Prospective and retrospective specimens in either a clean dry container or preserved in Cary–Blair transport medium. To increase the number of possible positive samples of the rarer pathogen Shiga toxin, retrospective samples of positive samples for Shiga toxin EIA were also included if frozen at −20˚C. Where possible these were paired with one or more culture negative specimens from the same time period. Formed stools or rectal swabs were excluded. |
|
Primary outcomes |
PPA and NPA rather than sensitivity or specificity because no true reference method is available. Reference standard: Standard culture methods for the laboratory. EIA for Shiga toxins. Samples with discrepant results for the BD MAX EBP and culture or EIA were tested with an alternative PCR method, which was then used as the reference standard if BD MAX EBP was positive but culture negative. |
|
Statistical methods |
PPA and NPA with 95% confidence intervals. Chi–square or Fisher's exact test to compare PPA and NPA between preserved and unpreserved specimens. |
|
Conclusions |
The BD MAX EBP showed superior sensitivity compared with conventional methods and excellent specificity for the detection of bacterial pathogens in stool specimens. |
|
Abbreviations: EIA, enzyme immunoassay; PCR, polymerase chain reaction; PPA, positive percentage agreement; NPA, negative percentage agreement. |
|
Table 2 Summary of results from the Harrington et al. (2015) study
|
Patients included |
4242 specimens from adult and paediatric patients. 3457 prospective samples collected between December 2012 and September 2013: 1345 were unpreserved and 2112 were preserved. 785 retrospective samples collected between 2007 and 2013: 321 unpreserved and 464 preserved. The age distribution of patients was as follows: <1 year 3.7% 1–4 years 10.3% 5–12 years 11.5% 13–18 years 10.5% 19–65 years 48.6% >65 years 15.3%. Approximately 25% of specimens were from children <12 years. No further details regarding the patient spectrum are provided. Prevalence of each target pathogen in the sample of prospective specimens (based on culture or EIA methods): Campylobacter: 1.5% Salmonella: 1.2% Shigella: 0.8% Shiga toxin: 0.4%. Prevalence of each target pathogen in the sample of prospective specimens (based on culture, EIA and PCR methods): Campylobacter: 2.3% Salmonella: 2% Shigella: 1.1% Shiga toxin: 0.8%. |
|
|
Primary outcome |
Prospective specimens PPA Campylobacter: 97.8% Salmonella: 87.2% Shigella: 100% Shiga toxins: 80%. NPA Campylobacter: 98.2% Salmonella: 99.1% Shigella: 99.7% Shiga toxins: 99.3%. |
Retrospective specimens PPA Campylobacter: 97.0% Salmonella: 99.4% Shigella: 98.9% Shiga toxins: 100%. NPA Campylobacter: 99.5% Salmonella: 99.8% Shigella: 100% Shiga toxins: 100%. |
|
Of discrepant results: Six were positive by culture but negative by the BD MAX EBP of which 1 grew Campylobacter and 5 grew Salmonella not detected by BD MAX EBP or alternative PCR. Two samples were positive for Shiga toxin EIA and negative by the BD MAX and alternative PCR. Samples that were positive by the BD MAX EBP but negative by culture or EIA: Campylobacter: 51 (of which 22 were positive with alternative PCR) Salmonella: 26 (of which 19 were positive with alternative PCR) Shigella: 10 (of which 9 were positive with alternative PCR) Shiga toxins: 17 (of which 9 were positive with alternative PCR). |
||
|
Abbreviations: EIA, enzyme immunoassay; PCR, polymerase chain reaction; PPA, positive percentage agreement; NPA, negative percentage agreement. |
||
The UK‑based study by Biswas et al. (2014) investigated the diagnostic accuracy and laboratory turnaround time of 3 PCR assays for the identification of bacterial pathogens in cases of gastroenteritis. In this cross‑sectional study, funded by the National Institute for Health Research, the BD MAX EBP was compared with the RIDAGENE Bacterial Stool Panel and EGEC/EPEC Panels (made by R‑Biopharm AG) and the FTD Bacterial Gastroenteritis (made by Fast Track Diagnostics). The study took place in a single laboratory at a London teaching hospital in the UK. Unpreserved diarrhoeal stool samples submitted for routine bacterial culture between November 2013 and February 2014 were included. Samples came from both hospital inpatients and people in the community. The reference standard for a true positive was either a positive culture, or 2 out of 3 PCR methods positive for a pathogen. The primary outcomes were sensitivity, specificity, positive predictive value and negative predictive value.
There were 434 samples collected, of which 318 were prospectively collected and 116 were retrospective culture‑positive samples. The results are presented in tables 3 and 4.
PCR led to the detection of an additional 9 cases of Campylobacter and 4 cases of Shigella. The reported laboratory turnaround time was 163 minutes, with 20 minutes hands‑on time, compared with 66.5 hours for bacterial culture methods. It is not clear from the paper whether these figures represent the average time per test. The authors concluded that PCR panels were more sensitive than culture‑based methods, allowing faster detection of a larger number of infectious people. Of the 3 PCR systems the BD MAX System was the fastest, needed the least hands‑on time and appeared to have slightly better performance characteristics in terms of sensitivity, specificity, positive predictive value and negative predictive value than the alternatives.
Table 3 Overview of the Biswas et al. (2014) study
|
Study component |
Description |
|
Objectives/hypotheses |
To investigate the diagnostic accuracy and laboratory turnaround time of 3 PCR assays for the detection of bacterial gastroenteritis. The assays included: BD MAX EBP RIDAGENE Bacterial Stool Panels FTD Bacterial Gastroenteritis |
|
Study design |
Cross‑sectional |
|
Setting |
UK single‑centre laboratory study. NIHR‑funded. |
|
Inclusion/exclusion criteria |
Unpreserved diarrhoeal stool samples submitted for routine bacterial culture between November 2013 and February 2014. |
|
Primary outcomes |
Sensitivity and specificity. Reference standard true positive if either culture or 2 out of the 3 PCR methods tested positive. |
|
Statistical methods |
Sensitivity, specificity, PPV, NPV |
|
Conclusions |
PCR panels are more sensitive than culture‑based methods allowing faster detection of a larger number of infectious people. The BD MAX EBP was fastest of all methods and required the least hands‑on time. The BD MAX EBP appeared to have slightly greater performance characteristics than the other PCR panels. |
|
Abbreviations: EIA, enzyme immunoassay; NIHR, National Institute for Health Research; PCR, polymerase chain reaction; PPV, positive predictive value; NPV, negative predictive value. |
|
Table 4 Summary of results from the Biswas et al. (2014) study
|
Patients included |
434 samples: 318 prospectively collected and 116 retrospective culture‑positive samples. This was described as a mostly community‑ or outpatient‑based patient cohort. |
|
Results |
Prevalence of pathogens in the prospective samples Campylobacter: 5.3% Shigella: 4.4% Salmonella: 1.3% Shiga‑toxin‑producing E. Coli: 0.3%. BD MAX EBP — prospective samples only Campylobacter: Sensitivity 100% (95% CI 80.5 to 100) Specificity 100% (95% CI 98.8 to 100) PPV 100% (95% CI 80.5 to 100) NPV 100% (95% CI 98.8 to 100). Shigella: Sensitivity 100% (95% CI 76.8 to 100) Specificity 100% (95% CI 98.8 to 100) PPV 100% (95% CI 76.8 to 100) NPV 100% (95% CI 98.8 to 100). Salmonella: Sensitivity 100% (95% CI 39.8 to 100) Specificity 99.7% (95% CI 98.2 to 100) PPV 80% (95% CI 28.4 to 99.5) NPV 100% (95% CI 98.8 to 100). All samples (prospective and retrospective) NB: Sensitivity and specificity only presented in paper. Campylobacter: Sensitivity 92.1% (95% CI 85 to 96.5) Specificity 100% (95% CI 98.9 to 100). Shigella: Sensitivity 94.4% (95% CI 81.3 to 99.3) Specificity 100% (95% CI 99.1 to 100). Sensitivity 75% (95% CI 50.9 to 91.3) Specificity 100% (95% CI 99.1 to 100). Shiga‑toxin‑producing E. Coli: Sensitivity 100% (95% CI 25 to 100) Specificity 99.5% (95% CI 98.3 to 100). PCR led to the detection of an additional 9 cases of Campylobacter and 4 cases of Shigella. Laboratory turnaround time: BD MAX EBP: 163 minutes (with 20 minutes hands‑on time) Culture methods: 66.5 hours. |
|
Abbreviations: CI, confidence interval;EIA, Enzyme Immunoassay; NPA, negative percentage value; PCR, Polymerase Chain Reaction; PPV, positive predictive value. |
|
The study by Anderson et al. (2014) was a cross‑sectional study in a single laboratory in the USA that aimed to evaluate the performance of the BD MAX EBP in detecting Salmonella, Campylobacter jejuni, Shigella and Shiga‑toxin‑producing E. coli in preserved stool specimens. The organisms tested for included 4 unique strains each of Salmonella, Campylobacterjejuni, Shigella and enterohaemorrhagic E. coli (a total of 16 strains). The study used Cary–Blair‑preserved stool specimens (specimens preserved in Cary–Blair transport medium, a transport medium for Gram‑negative and anaerobic organisms) that had previously tested negative for enteric pathogens by routine culture and the BD MAX EBP. These negative samples were mixed with known concentrations of different species of bacteria. The total number of samples in this study is unclear although it appears to be 1 sample per bacterial species at each concentration level. The reference standard was the known true result of these artificially produced samples.
The BD MAX EBP demonstrated 100% sensitivity for all bacteria tested at the following concentrations of bacteria in the sample: 107 colony‑forming units (CFU)/ml, 106 CFU/ml and 105 CFU/ml. The results of this study are summarised in tables 5 and 6.
For all of the bacterial species and concentrations tested, the BD MAX EBP was as sensitive as culture methods. At lower concentrations BD MAX EBP was more sensitive than culture methods. The authors concluded that the BD MAX EBP has a higher sensitivity at low levels of concentration for enteric pathogens compared with culture.
Table 5 Overview of the Anderson et al. (2014) study
|
Study component |
Description |
|
Objectives/hypotheses |
To evaluate the performance of the BD MAX EBP in detecting 16 strains in total of Salmonella,Campylobacter jejuni and coli, Shigella and Shiga‑toxin‑producing E. coli in preserved stool specimens. |
|
Study design |
Cross‑sectional. |
|
Setting |
Single‑centre laboratory study, USA. |
|
Inclusion/exclusion criteria |
Cary–Blair‑preserved stool samples, negative for enteric pathogens by routine culture methods and the BD MAX EBP that were artificially spiked with pathogen strains at the following levels of concentration: 107 CFU/ml 106 CFU/ml 105 CFU/ml 104 CFU/ml 103 CFU/ml. |
|
Primary outcomes |
Sensitivity. Reference standard determined by known true result of artificially produced samples. |
|
Statistical methods |
Sensitivity % |
|
Conclusions |
The BD MAX EBP has a higher sensitivity at low levels of concentration for enteric pathogens compared with culture. |
|
Abbreviations: CFU/ml, colony forming unit/millilitre; EIA, enzyme immunoassay; PCR, polymerase chain reaction. |
|
Table 6 Summary of results from the Anderson et al. (2014) study
|
Patients included |
Cary–Blair‑preserved specimens from clinical patients negative for enteric pathogens by routine stool culture and BD MAX EBP artificially spiked with pathogens at a range of concentrations. |
|
Primary outcome results |
At the following concentrations: 107 CFU/ml;106 CFU/ml 105 CFU/ml; the BD MAX EBP demonstrated 100% sensitivity for all organisms tested. At 104 CFU/ml the sensitivity of the BD MAX EBP was: Campylobacter: 100% Shiga‑toxin‑producing E. coli: 87.5% Salmonella: 68.8% Shigella: 100%. At 103 CFU/ml the sensitivity of the BD MAX EBP was: Campylobacter: 100% Shiga‑toxin‑producing E. coli: 13.3% Salmonella: 43.8% Shigella: 81%. For all pathogens at all concentrations the BD MAX EBP was as sensitive as culture methods. At lower concentrations the BD MAX EBP was more sensitive than culture methods. |
|
Abbreviations: CFU/ml, colony forming unit/millilitre; EIA, enzyme immunoassay; PCR, polymerase chain reaction. |
|
Studies available as abstracts
Seven abstracts reported cross‑sectional studies of the diagnostic performance of the BD MAX EBP compared with culture or enzyme immunoassay methods in a total of 4569 samples (Ashman et al. 2013; Beucher et al. 2014; Buchan et al. 2013; Chapin et al. 2014; McAulay et al. 2014; Porter et al. 2014; Rebec et al. 2013). These studies, including 1 in which stool samples from children were tested (Beucher et al. 2014), all reported that the BD MAX EBP was more sensitive than conventional methods.
One abstract (Perry et al. 2014) reported a study comparing the BD MAX EBP with the Luminex xTAG Gatrointestinal Pathogen Panel (GPP) for the detection of Campylobacter, Salmonella,Shigella and Shiga toxin‑producing E coli using artificially spiked samples. The authors concluded that the BD MAX EBP demonstrated superior limits of detection compared with the Luminex xTAG GPP.
One abstract (Mortensen et al. 2014) reported the results of a time‑motion study comparing the use of the BD MAX EBP with conventional culture techniques in a US hospital‑based laboratory. Following the processing of 86 samples, the average time to report for routine culture was 44 hours 37 minutes compared with 7 hours and 6 minutes for the BD MAX EBP, which represented an average reduction of turnaround time of 85%.
Costs and resource consequences
The BD MAX EBP is currently in use in a number of NHS trusts. According to the manufacturer there are currently 31 BD MAX Systems installed across 27 NHS trusts in the UK, processing a variety of infectious disease assays. Of those sites, 13 are using the BD MAX EBP routinely, and several others have submitted business cases to use the panel.
Using the technology could eliminate the need for culture of stool specimens to detect the pathogen included in the panel, if specimens are negative on PCR testing. This could lead to a reduction in culture tests and in the time needed for trained scientists to carry out these tests.
The automated nature of the test means that it could be carried out by lower grades of laboratory staff than is needed for bacterial culture, provided that biomedical scientists train them and help with data checking and trouble‑shooting. Therefore, there is potential for savings through revision of skills mix.
No published evidence on resource consequences solely attributable to the BD MAX EBP was identified.
Strengths and limitations of the evidence
From the perspective of evaluation of the BD MAX EBP as a diagnostic test, there is no clear reference standard and relevant studies have used composite reference standards incorporating both culture and molecular results. This issue applies to the evaluation of any diagnostic method to detect bacterial pathogens in gastroenteritis and is not specific to the BD MAX EBP.
The study by Harrington et al. (2015) was a large diagnostic study that included samples from a variety of patients across a range of ages. There is little information about other patient characteristics. It is unclear whether all consecutive samples submitted to the laboratory were included or whether the included samples were randomly selected. As with all studies, the reference standard was imperfect, and in this study the PCR method used to resolve discrepant results was not described. The study did not describe whether staff were blinded to the status of samples or the order and timescale of testing. Although blinding should not affect the qualitative result offered by the BD MAX EBP, it might impact upon culture interpretation.
The study by Biswas et al. (2014) was an independent study that received no financial support from the manufacturer. It was conducted in a laboratory at a London hospital and received samples from inpatients and people in the community, so this might be reflective of standard use in a UK hospital. The sample size was small, resulting in few numbers of positive samples and some imprecise estimates of diagnostic accuracy. It is not clear whether all consecutive samples submitted to the laboratory were included or whether the included samples were randomly selected. The order of testing was unclear, nor was it clear whether technicians were blinded to the status of the samples throughout. As in the other studies, the reference standard was imperfect. However, in this study, PCR results, including the BD MAX EBP, were included to determine true positives. This may have reduced the problems associated with an imperfect reference standard, but could also introduce a risk of incorporation bias which might overestimate the sensitivity of the test.
The study by Anderson et al. (2014) was very small, although the exact number of samples tested was unclear. It offers experimental evidence of the sensitivity of the BD MAX EBP for detecting different bacterial concentrations, but all samples were artificially produced by mixing stool samples with bacteria, resulting in lower clinical relevance. No negative samples were tested, so no conclusions can be drawn regarding specificity. The study was materially funded by the manufacturer.
The evidence presented in the included abstracts broadly concurs with the findings of the 3 fully‑published papers. Studies available only as abstracts may not have been fully peer‑reviewed, may be more susceptible to a publication bias, and the limited information presented precludes critical appraisal.