Evidence review
Clinical and technical evidence
The evidence available for The NGAL Test (BioPorto) relating to acute kidney injury was limited to 3 observational studies, summarised in tables 1, 2 and 3.
Study component | Description |
---|---|
Objectives/hypotheses |
To evaluate whether plasma and urinary NGAL concentrations are useful early markers of AKI developing during and after liver transplantation surgery. |
Study design |
Prospective observational |
Setting |
|
Inclusion/exclusion criteria |
|
Variables |
Early diagnosis of AKI using corrected uNGAL and pNGAL. |
Statistical methods |
This study used the mean plus or minus standard deviation to report the results, and did comparisons using the chi-squared test, Mann–Whitney U test and Wilcoxon signed rank test. Statistical significance was defined as p<0.05. |
Results |
Using RIFLE criteria, 11 out of 19 patients (57.9%) were diagnosed with AKI after reperfusion. Baseline sCr values were within normal range, and there was no statistically significant difference between the AKI group and the non-AKI group. AKI group:
Non-AKI group:
|
Conclusions |
|
Abbreviations: AKI, acute kidney injury; CI, confidence interval; pNGAL, plasma neutrophil gelatinase-associated lipocalin; RIFLE, risk injury failure loss end stage renal disease; sCr, serum creatinine; uCr, urinary creatinine; uNGAL, urinary neutrophil gelatinase-associated lipocalin.
Study component | Description |
---|---|
Objectives/hypotheses |
To look at whether sepsis could have an impact on the sensitivity of sNGAL, uNGAL and CysC for AKI diagnosis in critically ill children. |
Study design |
Single-centre prospective observational cohort study. |
Setting |
|
Inclusion/exclusion criteria |
|
Variables |
Evaluate levels of sNGAL, uNGAL, sCysC and uCysC in children with severe sepsis, with and without AKI. |
Statistical methods |
|
Results |
|
Conclusions |
|
Abbreviations: AKI, acute kidney injury; analysis of variance, ANOVA; CI, confidence interval; OR, odds ratio; PICU, paediatric intensive care unit; pRIFLE, paediatric modified risk injury failure loss end stage renal disease; sCysC, serum Cystatin C; sNGAL serum neutrophil gelatinase-associated lipocalin; uCysC, urinary Cystatin C; uNGAL, urinary neutrophil gelatinase-associated lipocalin.
Study component | Description |
---|---|
Objectives/hypotheses |
To assess acute variations in NGAL and creatinine after doing strenuous exercise. |
Study design |
Prospective observational study. |
Setting |
|
Inclusion/exclusion criteria |
The study recruited 16 white men who had done an endurance training regime for the previous 3 to 10 years. Exclusion criteria:
|
Variables |
Variation in uCr, sCr, uNGAL, sNGAL, eGFR and uNGAL/uCr ratio between pre- and post-run blood and urine tests. |
Statistical methods |
Statistical analysis was done using Analyse-it for Excel. Data were presented using medians and interquartile ranges, and statistical significance was set at p<0.05. |
Results |
Post run: sCr increased by 38% (IQR 14% to 66%) uCr increased by 78% (IQR 21% to 158%) eGFR decreased by 31% (14% to 44%) sNGAL increased 1.6 fold (IQR 1.3 to 2.0 fold) uNGAL increased 7.7 fold (IQR 1.9 to 37.3 fold) nNGAL/uCr ratio increased 2.9 fold (IQR 1.6 to 25.8 fold) 6/16 athletes met the AKIN criteria for AKI. No significant relationship was found between pre-exercise sNGAL and uNGAL concentrations (r −0.147; p=0.564) or between sNGAL and uNGAL/uCr ratio (r −0.110; p=0.673). Likewise, no significant relationship was found between the same categories post exercise (r 0.085; p=0.746 and r 0.192; p=0.460). No significant correlation was found between the pre- and post-exercise variation in sNGAL and either uNGAL (r 0.025; p=0.925) or uNGAL/uCr ratio (r 0.016; p=0.950), or between sCr and either uNGAL (r 0.248; p=0.338) or uNGAL/uCr ratio (r 0.254; p=0.325). A highly significant correlation was found between the pre- and post-exercise changes of sCr and sNGAL (r −0.813; p<0.001). |
Conclusions |
The authors concluded that there is evidence that strenuous physical exercise increases NGAL and creatinine concentration, which may indicate AKI. |
Abbreviations: AKI, acute kidney injury; AKIN, Acute Kidney Injury Network; eGFR, estimated glomerular filtration rate; IQR, interquartile range; r, sample correlation coefficient; sCr, serum creatinine; sNGAL serum neutrophil gelatinase-associated lipocalin; uCr, urinary creatinine; uNGAL, urinary neutrophil gelatinase-associated lipocalin.
Jeong et al. (2012) (see table 1) used 2 separate NGAL test methods – the Abbott ARCHITECT NGAL assay for urinary neutrophil gelatinase-associated lipocalin (NGAL), and The NGAL Test from BioPorto for plasma NGAL. Consequently, the results for plasma NGAL are those of interest.
In this study, 11 of the 19 patients were diagnosed with acute kidney injury by the risk, injury, failure, loss of kidney function, and end-stage renal failure (RIFLE) criteria, using an increase of greater than 50% from baseline in serum creatinine as the formal definition of onset of acute kidney injury. This is one of the recommended criteria in NICE's guideline on acute kidney injury for defining the onset of acute kidney injury, but may present later than other criteria such as an increase of 26 micromol/litre. This could cause some of the patients in the study to be defined incorrectly as not having acute kidney injury.
The authors acknowledged that there are no established criteria for diagnosing acute kidney injury using a NGAL test, so they defined the onset of acute kidney injury as the point at which the level of NGAL in either plasma or urine had risen by greater than 50%. Of the 11 patients with acute kidney injury, 8 had an increase in plasma NGAL of greater than 50% with the increase occurring before the increase in serum creatinine in 7 of these patients. However, out of the 8 patients who were not diagnosed with acute kidney injury by the RIFLE criteria, 7 of these also experienced an increase in plasma NGAL of greater than 50%. Sensitivity, specificity, positive predictive values and negative predictive values were not reported.
Di Nardo et al. (2013) (see table 2) also investigated both urinary NGAL and serum NGAL with 2 different NGAL test methods. The NGAL Test was used to measure urinary NGAL while an ELISA NGAL kit, also from BioPorto, was used to measure serum NGAL. The ELISA NGAL kit is outside of the scope of this briefing.
The study included 11 paediatric patients with severe sepsis admitted consecutively to the paediatric intensive care unit, of which 4 were diagnosed with acute kidney injury using paediatric RIFLE criteria. Ten paediatric patients who were having elective surgery were recruited to form a control group.
A statistically significant difference (p=0.0019) was found in urinary NGAL levels between patients with sepsis and acute kidney injury (median 300.8 nanogram/dl, interquartile range 15.05 to 5,952) and those with sepsis without acute kidney injury (median 26.59 nanogram/dl, interquartile range 11.5 to 543.6). No statistically significant difference was found in the urinary NGAL levels between the group of patients with sepsis without acute kidney injury and the control group (p>0.05). The study did not report a comparison between urinary NGAL levels in patients with sepsis with acute kidney injury and the control group. Therefore it is not possible to draw a conclusion about this.
Lippi et al. (2012a) (see table 3) measured urinary NGAL, corrected urinary NGAL and serum NGAL using The NGAL Test in 16 athletes to examine whether strenuous physical exercise could cause acute kidney injury.
Within this study, 6 of the 16 athletes were diagnosed with post-run acute kidney injury using the acute kidney injury network criteria. This is one of the methods recommended in NICE's guideline on acute kidney injury for detecting acute kidney injury. However, the results for these 6 athletes were not separated from the group results, and so it is not possible to conclude which results were related to those athletes who were diagnosed with acute kidney injury. The result from 1 athlete with a high baseline serum NGAL (but not urinary NGAL), which increased more after the run, was considered anomalous with an explanation that this may have been due to a high leukocyte count, which was normal in all the other athletes.
In addition, 3 technical validations were identified, summarised in tables 4, 5 and 6.
Study component | Description |
---|---|
Objectives/hypotheses |
To do an independent technical validation and comparison of 5 commercially available urinary NGAL assays, 2 CE marked in vitro diagnostics and 3 research-use only ELISAs. |
Study design |
Technical Validation. |
Setting |
The CE marked assays were the ARCHITECT uNGAL assay, Abbott, and The NGAL Test, BioPorto. These were analysed on the i2000SR ARCHITECT analyser and the Siemens ADVIA 1800 respectively. The research-only use assays were the Human NGAL ELISA kit, Hycult Biotech, the human NGAL ELISA kit 036, BioPorto, and the Quantikine Human Lipocalin-2/NGAL Immunoassay, R&D systems. Validation stages:
|
Inclusion/exclusion criteria |
|
Variables |
Determined which assays display satisfactory performance for the validation tests. |
Statistical methods |
Modified Bland–Altman plots, Passing–Bablok analysis. |
Results |
|
Conclusions |
|
Abbreviations: AKI, acute kidney injury; AKIN, Acute Kidney Injury Network; ELISA, enzyme-linked immunoassay; NGAL, neutrophil gelatinase-associated lipocalin; UTI, urinary tract infection.
Study component | Description |
---|---|
Objectives/hypotheses |
To evaluate the precision and reportable range of results in comparison to a widely used point of care test. |
Study design |
Performance evaluation. |
Setting |
|
Inclusion/exclusion criteria |
|
Variables |
Comparison of the precision and reportable range of results of The NGAL Test to a point of care device. |
Statistical methods |
Only stated method was that Analyse-it for Microsoft Excel was used. |
Results |
Precision:
Reportable range: BioPorto specify the analytical measurement range to be 25 to 5,000 nanogram/ml. The author measured the analytical range from 57 to 3,182 nanogram/ml and found the coefficient of correlation equal to 0.999 within the range tested. Coefficient of determination: The comparison between The NGAL Test and the Triage NGAL Test found the coefficient of determination to be r²=0.791. No confidence intervals were reported. The paper noted that the correlation improved when the highest 8 results were removed from the analysis (r²=0.959), but no justification for removing these results was given. |
Conclusions |
The results were compared with the Triage NGAL point of care test, using results reported by Dent et al. (2007). The coefficients of variation for The NGAL Test from BioPorto were much lower than that for Triage NGAL Test, showing there is less variation in results for The NGAL Test between test runs. Additionally, the reportable range of the Triage NGAL Test was reported as 60 to 300 nanogram/ml, narrower than The NGAL Test and significantly reducing the use at higher NGAL concentrations. |
Abbreviations: NGAL, neutrophil gelatinase-associated lipocalin.
Study component | Description |
---|---|
Objectives/hypotheses |
To evaluate an NGAL immunoassay and compare it with the Abbott ARCHITECT NGAL. |
Study design |
Technical validation/Comparison study. |
Setting |
|
Inclusion/exclusion criteria |
70 patients who needed urinalysis presenting consecutively to the emergency department. |
Variables |
Comparison study, comparing the results of The NGAL Test to the Abbott ARCHITECT NGAL. |
Statistical methods |
Statistical evaluation was done with Analyse-it for Microsoft Excel. |
Results |
Imprecision:
|
Conclusions |
The authors concluded that there were many benefits with The NGAL Test, such as fast turnaround time, low imprecision, optimal linearity in the most clinically significant range and a wide dynamic range. |
Abbreviations: CI, confidence interval; NGAL, neutrophil gelatinase-associated lipocalin.
Kift et al. (2013) compared the analytical performance of 5 commercially available assays, including 2 CE marked assays (the Abbott ARCHITECT and The NGAL Test), as well as 3 research-only assays. This study found that The NGAL Test performed well for all tests except for the limit of quantification, which was found to be 150 nanogram/ml, when the manufacturer had stated that it was 25 nanogram/ml in the application notes (see table 4).
Lippi et al. (2012b) evaluated the imprecision and linearity of The NGAL Test from BioPorto. In addition, they compared The NGAL Test with the Abbott ARCHITECT NGAL (see table 5).
The imprecision and linearity results from The NGAL Test were compared with results found in the literature from papers by Cavalier et al. (2011) and Grenier et al. (2010) for the Abbott ARCHITECT NGAL and the Biosite Triage. The methods used by Cavalier et al. (2011) varied slightly from the methods used by Lippi et al. (2012b), so it is not certain whether the comparison between results was valid.
The results showed a good correlation between The NGAL Test and the ARCHITECT, but also a mean bias towards The NGAL Test of 65%.
Seo et al. (2013) compared The NGAL Test from BioPorto with the Biosite Triage NGAL Test, which is a point of care assay (see table 6). The precision coefficients of variation for The NGAL Test were found to be less than 5% compared with less than 14% for the Biosite Triage precision, as reported by Dent et al. (2007). Seo et al. claimed that 'Plasma used in The NGAL Test was more reliable than that of urine' (in the Triage Test).
Seo et al. found that the results of the Triage NGAL Test correlated better with those obtained using The NGAL Test for NGAL concentrations of less than 500 nanogram/ml. Between 500 nanogram/ml and 1300 nanogram/ml (the upper limit of quantification for the Triage test), the results were not linearly correlated.
Costs and resource consequences
Extrapolations from academic studies suggest that there may be as many as 360,000 admissions of people with a diagnosis of acute kidney injury in the UK annually (HSJ supplement 2011). The NGAL Test can be done in a standard laboratory and can be run on most automated clinical chemistry analysers, suggesting that there would be no need to change service organisation or delivery if the test is adopted.
No published evidence on resource consequences of The NGAL Test was identified. Only 1 study (Shaw et al. 2011), which assessed the cost effectiveness of a different NGAL as a biomarker in general rather than specific use of The NGAL Test was identified and included in the review. The study assessed the cost effectiveness of urinary NGAL to diagnosis acute kidney injury after cardiac surgery compared with current diagnostic methods (that is, monitoring of serum creatinine, blood urea nitrogen and urine output).
A decision model was created from the societal perspective using a base case to depict a typical patient in the UK without a history of chronic kidney disease (67-year-old man) having coronary artery bypass graft surgery and admitted to the intensive care unit immediately after the procedure. Cost per quality-adjusted life-year (QALY) was used as the relevant measure for the economic evaluation. A 25% improvement in outcomes from acute kidney injury because of early diagnosis, based on NGAL levels, was assumed in the model. This was varied to 12.5% and 50% levels in the sensitivity analysis. Probabilities and costs were obtained from literature sources and also from expert opinion when data were not available.
The base-case analysis showed an expected cost of £4,244 and 11.86 QALYs for the NGAL strategy compared with £4,672 and 11.76 QALYs for current practice. The cost-effectiveness ratio, presented in the study, was £358 per QALY for NGAL compared with £396 per QALY for current practice. The NGAL strategy dominated (was more effective and less expensive than) current practice and the conclusions remained the same for the sensitivity analysis with therapeutic effect size. Probabilistic sensitivity analyses showed that the NGAL strategy was cost effective for 100% of the 1,000 patient simulations. The results therefore suggested that using urinary NGAL is likely to economically beneficial because acute kidney injury is diagnosed earlier, which appears to result in lower than expected costs.
The results of Shaw et al. (2011) need to be considered with caution because of gaps in the information presented or potential weaknesses in the study design. It is difficult to identify the respective literature source and expert opinion used in the model because the presentation of the parameter estimates was not clearly described. Specifically, the cost estimates used were arbitrary. The serum creatinine test cost estimate was £15 and the urinary NGAL test cost estimate was £25, which were not based on accepted sources such as National Health Service tariffs. There was zero cost assigned to urine output measurement, but this could include measurement costs in terms of time and nursing staff. There was uncertainty surrounding these estimates. The authors did the appropriate sensitivity analysis to account for this but this did not mitigate the study limitation. Secondly, the authors claimed that the study used a societal perspective for evaluation, but they did not consider indirect and opportunity costs that may be associated with acute kidney injury and its potential sequelae in patients who survive hospitalisation and develop some of the long-term complications associated with chronic and end-stage renal disease. Thirdly, the model was based on data from coronary artery bypass graft surgery patients only, and it is very likely that it underestimated overall surgical mortality and the likelihood of developing acute kidney injury because these are likely to be higher for other cardiac procedures. Finally, the model assessed the cost effectiveness of NGAL as a biomarker in general rather than specific use of The NGAL Test.
Adoption of The NGAL Test based only on study results for urinary NGAL biomarker testing is not reliable because The NGAL Test can use both urine and plasma samples, not just urine samples, for the assay procedure.
Strengths and limitations of the evidence
The evidence is limited to 3 observational studies, and 3 technical validations. No published randomised controlled trials investigating The NGAL Test were identified during the literature search. Additionally, no ongoing clinical trials specifying the use of The NGAL Test have been found. However, some UK trials are due to start in 2014 that will evaluate The NGAL Test in different clinical settings, including urology, intensive care units and accident and emergency units (BioPorto: personal communication [2014]).