5 Outcomes
The Diagnostics Advisory Committee considered evidence from a number of sources. Full details are in the project documents for this guidance.
How outcomes were assessed
5.1
The External Assessment Group (EAG) conducted a systematic review of the evidence on tests to monitor levels of tumour necrosis factor (TNF)‑alpha inhibitors and antibodies to TNF‑alpha inhibitors in people with Crohn's disease treated with infliximab or adalimumab. The review had 4 key objectives:
-
compare the performance of the different tests available
-
compare optimal cut‑off thresholds identified in different studies
-
analyse the correlation between test results and clinical state
-
describe and compare test‑informed algorithms used in studies, and review the clinical effectiveness of these test‑informed algorithms compared with standard care (no testing done).
5.2
For the purpose of this assessment and to aid understanding, tests have been split into 3 groups: index tests, alternative tests, and other tests. The 6 different tests are summarised in table 1. Because there were no direct clinical outcome data for the index tests (LISA‑TRACKER enzyme‑linked immunosorbent assay [ELISA] kits, IDKmonitor ELISA kits and Promonitor ELISA kits), the clinical‑effectiveness review considered alternative tests for which clinical outcome data were available. Evidence on the comparative performance of the index tests and the alternative tests was then sought in order to make a link between the index tests and the clinical outcomes. Other tests are also mentioned in the review because they form an indirect link between the index tests and clinical outcomes through the alternative tests.
Evidence on clinical outcomes
5.3
Three studies were identified that implemented a test‑informed algorithm in managing Crohn's disease treated with infliximab or adalimumab and reported clinical outcomes.
Steenholdt et al. 2014 and 2015
5.4
This was a single‑blind randomised controlled trial of 69 adults with Crohn's disease on maintenance infliximab treatment whose disease had lost response to treatment. Patients were randomised to either an infliximab intensified arm (n=36) or to an algorithm arm (n=33). In the infliximab intensified arm, the dose frequency of 5 mg/kg infliximab was increased from every 8 weeks to every 4 weeks. In the algorithm arm, patients had treatment according to a defined algorithm based on serum concentrations of infliximab and of antibodies to infliximab. Samples were taken immediately before infliximab infusion and were analysed by radioimmunoassay. The algorithm categorised patients into one of 4 groups and guided treatment as described in table 2.
5.5
In the dose‑intensification arm, all patients had treatment according to the protocol. In the algorithm arm, 14 of 33 patients did not have treatment according to the algorithm (13 in group 3; 1 in group 4). Most of these 14 patients continued to have infliximab. There were 2 withdrawals from the algorithm arm and 8 withdrawals from the dose intensification arm.
5.6
In the intention to treat population (n=69), clinical response at week 12 was seen in 53% of patients in the dose intensification arm and in 58% of patients in the algorithm arm (relative risk [RR] 1.09; 95% confidence interval [CI] 0.713 to 1.673; p=0.810). At week 20, clinical response was seen in 56% of the dose intensification arm and in 76% of the algorithm arm (RR 1.4; 95% CI 1.0 to 1.9; p=0.128). Remission was achieved at week 20 in 39% of patients in the dose intensification arm and in 55% of patients in the algorithm arm (RR 1.4; 95% CI 0.8 to 2.4; p=0.232).
Vaughn et al. 2014
5.7
This was a retrospective observational pilot study of patients with inflammatory bowel disease in clinical remission who were having infliximab. Patients were identified from records and classified into those who had proactive drug monitoring and those who did not (control group). Samples were analysed first by ELISA (Prometheus) and later with a homogenous mobility shift assay (HMSA; Prometheus). In the proactive monitoring group, serum trough levels of infliximab guided dose change to achieve target drug levels according to the algorithm presented in table 3. Reactive testing was done in both groups if the disease lost response or there was a concern for side effects because of antibody formation.
5.8
There were 48 patients in the proactive drug monitoring group and 78 patients in the control group. In the proactive drug monitoring group, infliximab dose was adjusted in 35% of patients after initial testing (71% dose escalation, 18% dose decrease, and 12% stopped infliximab). After subsequent proactive tests, the dose was adjusted in 25% of patients (80% dose escalation and 20% dose decrease).
5.9
After 5 years, the probability of staying on treatment was 86% in the proactive drug monitoring group and 52% in the control group (hazard ratio 0.3; CI 0.1 to 0.6; p=0.0006). In the control group, the main reasons for stopping infliximab treatment were recurrence of symptoms and acute infusion reactions. In the proactive drug monitoring group, the main reasons for stopping infliximab treatment were adverse events and high antibody levels.
Vande Casteele et al. 2015 – the TAXIT trial
5.10
This was a randomised controlled trial of 251 patients with inflammatory bowel disease (173 with Crohn's disease and 78 with ulcerative colitis). Patients were randomised to clinically‑based dosing or to infliximab trough‑level‑based dosing. Before randomisation, patients were screened and had an optimisation treatment phase, to identify patients whose trough levels of infliximab could be brought to the target range. Therefore, all randomised patients entered the maintenance phase of the study with trough infliximab levels in the target range of 3 to 7 micrograms/ml. In the clinically‑based dosing arm, all subsequent infliximab dosing was according to clinical symptoms and C‑reactive protein levels. In the trough‑level‑based dosing arm, all subsequent infliximab dosing was according to the algorithm presented in table 4. Samples were analysed using Leuven in‑house ELISAs.
5.11
In the optimisation phase, 74% of patients with Crohn's disease were in remission before dose optimisation, and 80% were in remission after optimisation. Dose escalation was done in 43 of 178 patients and the percentage of patients in remission in this group increased from 65% to 88%. Dose reduction was done in 51 of 178 patients and the percentage of patients in remission in this group decreased from 80% to 69%. For the dose escalation group, an average of 2.1 optimisations were needed to reach target trough infliximab levels, and at the end of optimisation the median infusion interval was 6 weeks (range 4–8 weeks). For the dose‑reduction group an average of 1.4 optimisations were needed and the median infusion interval was 8 weeks (range 6–12 weeks).
5.12
In the maintenance phase, similar rates of clinical remission were seen in both groups: 69% in the concentration‑based dosing group, and 66% in the clinically‑based dosing group. When restricted to patients with Crohn's disease, rates of clinical remission were 63% in the concentration‑based dosing group, and 55% in the clinically‑based dosing group.
5.13
There was little difference between groups in the probability of maintaining durable remission (26% in the concentration‑based dosing group and 27% in the clinically‑based dosing group). More patients in the concentration‑based dosing group than in the clinically‑based dosing group (74% compared with 57%) had an infliximab trough concentration between 3 micrograms/ml and 7 micrograms/ml. The risk of patients in the clinically‑based dosing group having undetectable trough levels of infliximab was statistically significantly greater than in the concentration‑based dosing group (RR 3.7; 95% CI 1.7 to 8.0; p<0.001). None of the patients in the concentration‑based dosing group were positive for anti‑drug antibodies, but 3 patients in the clinically‑based dosing group had anti‑drug antibodies.
5.14
No deaths occurred in either group. However, 2 patients in the clinically‑based dosing group needed hospital admission: one for acute appendicitis and the other for ileostomy complications. There were 12 discontinuations in the clinically‑based dosing group and 13 discontinuations in the concentration‑based dosing group. More patients in the clinically‑based dosing group (17%) relapsed and needed rescue therapy than in the concentration‑based dosing group (7%; RR 2.4; 95% CI 1.2 to 5.1; p=0.018).
Summary
5.15
Key conclusions from the 3 studies are summarised in table 5.
Evidence on optimal cut‑off thresholds
5.19
Receiver operating characteristic threshold analyses to determine optimal cut‑off thresholds predictive of clinical response for infliximab, adalimumab or both were reported in 24 studies. Different studies used different markers to assess clinical response. When identifying optimal cut‑offs, some studies aimed for high sensitivity (0.90) at the expense of specificity (0.37), whereas others favoured high specificity (1.00) at the expense of sensitivity (0.33). Reported cut‑offs for infliximab ranged from 0.6 to 7 micrograms/ml. Reported cut‑offs for adalimumab ranged from 3 micrograms/ml to 6.85 micrograms/ml.
5.20
The range of cut‑off thresholds reported across the included studies shows that no validated threshold has been established. Cut‑off thresholds strongly depend on the assay used, the drug measured, the clinical marker investigated and the time of testing.
Evidence on the correlation between test results and clinical state
5.21
The review identified 34 studies that reported on the relationship between test results and the clinical status of patients with Crohn's disease or inflammatory bowel disease. Of these, 3 were systematic reviews that included a meta‑analysis, and 31 were primary studies.
5.22
The test accuracy of drug‑level tests and anti‑drug antibodies tests as predictors of clinical status was moderate. Positive and negative predictive values across clinical prevalence ranges showed that 20% to 30% of test results were wrong.
5.23
Nanda et al. (2013) included 11 studies in a meta‑analysis and reported a 3‑fold greater risk of the disease losing response in patients with a positive anti‑drug antibodies test result compared with patients who had a negative anti‑drug antibodies test result (RR 3.16; 95% CI 2.00 to 4.98). Hierarchical meta‑analysis gave a sensitivity of 0.70 (95% CI 0.55 to 0.82) and specificity of 0.81 (95% CI 0.67 to 0.89) for the anti‑drug antibody test in predicting loss of response. At a loss of response prevalence of 34.7%, the positive predictive value was 65% and the negative predictive value was 84%.
5.24
Lee et al. (2012) included 10 studies in a meta‑analysis and reported no statistically significant decrease in rates of remission in patients with a positive test result for anti‑drug antibodies compared with patients with a negative test result for anti‑drug antibodies (RR 0.96; 95% CI 0.77 to 1.19). Hierarchical meta‑analysis gave a sensitivity of 0.42 and specificity of 0.69 for the anti‑drug antibody test in predicting remission.
5.25
Lee et al. (2012) also examined the association between developing anti‑drug antibodies and having immunosuppressant therapies. Meta‑analysis of 11 studies indicated a 50% reduction in risk of developing anti‑drug antibodies when immunosuppressants were administered (0.50; 95% CI 0.42 to 0.59).
5.26
Paul et al. (2014) included 3 studies in adults and 2 studies in children and reported statistically significantly greater odds of a lack of clinical response in patients with sub‑therapeutic adalimumab levels compared with patients with therapeutic levels of adalimumab (odds ratio 2.60; 95% CI 1.79 to 3.77). They also reported statistically significantly greater odds of a lack of clinical response in patients with antibodies to adalimumab compared with patients who had no antibodies to adalimumab (odds ratio 10.15; 95% CI 3.90 to 26.40).
Systematic review of cost‑effectiveness evidence
5.27
The External Assessment Group (EAG) conducted a search to identify studies investigating the cost effectiveness of LISA‑TRACKER ELISA kits, IDKmonitor ELISA kits, and Promonitor ELISA kits for measuring levels of TNF‑alpha inhibitors and of anti‑drug antibodies.
5.28
Four relevant studies were identified. All studies indicated that a testing strategy might be cheaper than a no‑testing strategy. However, studies reported variable small effects on effectiveness, with some indicating small reduced benefits and some indicating small increased benefits.
5.29
Vande Casteele et al. (2015) conducted a randomised controlled trial to determine whether concentration‑based infliximab dosing is more cost effective than clinically‑based infliximab dosing in people with moderate to severe Crohn's disease or ulcerative colitis (TAXIT trial). The time horizon of the model was 1 year and the perspective was that of the third‑party payer. The base‑case results showed that concentration‑based dosing was slightly less effective (0.8227 quality‑adjusted life years [QALYs] compared with 0.8421 QALYs) and less costly (€20,700 compared with €21,000) than clinically‑based dosing, but overall differences were small.
5.30
Steenholdt et al. (2014) assessed the cost‑effectiveness of having treatment based on serum concentrations of infliximab and antibodies to infliximab compared with having infliximab at an increased dose frequency of 5 mg/kg every 4 weeks. In all patients, the disease lost response to infliximab while the patient was having maintenance treatment. The authors reported that costs at 12 weeks were statistically significantly lower in the algorithm group than in the infliximab intensification group. Mean costs in the intention to treat population at 12 weeks were €6038 in the algorithm group compared with €9178 in the infliximab intensification group (p<0.001).
5.31
Steenholdt et al. (2015) conducted a follow‑up to the original study (Steenholdt et al. 2014), which extended the time horizon to 1 year to assess the long‑term costs of treating Crohn's disease that lost response to infliximab maintenance therapy. Costs were assessed at 20 weeks and at 1 year. The authors reported that the algorithm group had significantly lower costs than the infliximab intensification group at 20 weeks and this was maintained throughout the year. At 20 weeks, the average costs in the algorithm group were US$11,900 compared with US$17,200 in the infliximab intensification group. At 1 year, the average costs in the algorithm group were US$22,100 compared with US$29,100 in the infliximab intensification group.
5.32
Velayos et al. (2013) used a decision analytical model to assess the cost effectiveness of a testing‑based strategy compared with an empiric‑dose‑escalation strategy for patients with moderate to severe Crohn's disease whose disease lost response to infliximab. The study had a third party payer perspective and a 1‑year time horizon. The base‑case results showed that that the testing strategy was cheaper and marginally more effective than the empiric dose‑escalation strategy.
Economic analysis
Model structure
5.33
The EAG constructed 2 new economic models designed to assess the cost effectiveness of monitoring levels of TNF‑alpha inhibitor and anti‑drug antibody compared with standard care in patients with Crohn's disease. The first model focuses on patients whose disease responds to infliximab maintenance therapy and the second model focuses on patients whose disease loses response to infliximab maintenance therapy.
5.34
Both models have a 10‑year time horizon, a 4‑week cycle length and assume a cohort of people aged 30 years with moderate to severe Crohn's disease. In each model, patients can have either standard care, treatment according to an algorithm based on concurrent testing, or treatment according to an algorithm based on reflex testing.
5.35
Patients in the responder model enter in the responder health state, that is, their disease responds to treatment with maintenance infliximab. Patients may stay in this state or their disease may lose response to infliximab, that is, a recurrence of active symptoms while on maintenance infliximab treatment. After a dose change or switch in the TNF‑alpha inhibitor, the disease may regain response or may continue to lose response and the TNF‑alpha‑inhibitor treatment is stopped. Disease that regains response may continue to respond or may lose response again. Patients who stop TNF‑alpha‑inhibitor treatment will have best supportive care and some may need surgery. After surgery, patients move to a post‑surgery health state and may have a TNF‑alpha inhibitor, immunosuppressant, a combination of TNF‑alpha inhibitor and immunosuppressant or no treatment. Patients who have a TNF‑alpha inhibitor alone or in combination will re‑enter the model in the regain response state or the loss of response state. Patients who have an immunosuppressant or no treatment will stay in the post‑surgery state until they need further surgery or they die.
5.36
Patients in the loss of response model enter the model in the loss of response to TNF‑alpha inhibitor state, that is, active symptoms have recurred while on maintenance infliximab treatment. The model then follows the same structure as the responder model.
5.37
In the standard care pathway:
-
people whose disease is categorised as a responder continue having infliximab maintenance therapy every 8 weeks until they lose response
-
people whose disease loses response will have an increased dose; as a result, the disease may regain response or continue with loss of response
-
people whose disease continues to lose response will have another drug in addition to their current treatment; as a result, the disease may regain response or continue with loss of response
-
people whose disease continues to lose response will switch TNF‑alpha‑inhibitor treatment
-
people whose disease does not respond to a different TNF‑alpha inhibitor will be considered for surgery.
5.38
In the concurrent‑testing scenario, tests for infliximab levels and antibodies to infliximab would be done at the same time. Patients would fall into one of 4 categories:
-
drug absent and antibodies present
-
drug and antibodies absent
-
drug and antibodies present
-
drug present and antibodies absent.
5.39
In the reflex‑testing scenario, a test for infliximab levels is done first. If the drug is absent, a test for antibodies to infliximab would be done. If the drug is present, no further testing would be done. Patients would fall into one of 3 categories:
5.40
For patients whose disease is in the responder state, treatment options for each of the categories are based on the algorithm used in the TAXIT trial by Vande Casteele et al. (2015); table 6.
5.41
For patients whose disease loses response, treatment options for each of the categories are based on the algorithm used in the study by Steenholdt et al. (2014); table 7.
Model assumptions
5.46
The following assumptions were applied in the base‑case analysis:
-
Patients have had intravenous infusions of infliximab of 5 mg/kg at weeks 0, 2 and 6.
-
Patients weigh more than 70 kg.
-
Patients whose disease regained response have the same utility as those whose disease is categorised as a responder.
-
People with Crohn's disease are not at increased risk of dying from the disease over the lifetime of the model, and there is no difference in mortality between the test‑algorithm group and the standard‑care group.
-
For people who have had surgery, there is an increased risk of 0.0015 of dying from the procedure.
-
The treatment effects for people having a dose increase (from 5 mg/kg to 10 mg/kg of infliximab) and a decreased interval (from 8‑week to 6‑week intervals) are the same.
-
People whose disease is categorised as a responder and who have trough concentrations within the range that the treatment algorithm suggests receive no dose change.
-
Transition probabilities in the test‑algorithm group are the same as the transition probabilities in the standard‑care group for the following transitions:
-
loss of response to infliximab maintenance therapy (Juillerat et al. 2015)
-
loss of response with dose escalation (Ma et al. 2014)
-
loss of response to adalimumab maintenance therapy (Karmaris et al. 2009).
-
People whose disease stays in the loss of response health state (TNF‑alpha inhibitor stopped) have symptoms of Crohn's disease that in time may need surgery. People will have best supportive care until active symptoms develop that need surgery.
5.47
The testing schedules in the base‑case models were:
-
In the responder model, testing was done every 3 months while patients' disease was responding to a TNF‑alpha inhibitor. If patients' disease lost response to a TNF‑alpha inhibitor they would also be tested every 3 months until the TNF‑alpha inhibitor was stopped.
-
In the loss of response model, patients whose disease lost response were tested on entry into the model. If their disease regained response they would then enter onto the 3‑monthly testing regimen. If their disease continued to lose response to a TNF‑alpha inhibitor they would also be tested every 3 months until the TNF‑alpha inhibitor was stopped.
5.48
Two sets of base‑case results were provided. The first base‑case results use non‑constant hazard time‑to‑event transition probabilities. The second base‑case results use exponential transition probabilities (which assume constant hazard of time‑to‑event transition probabilities). These different sets of transition probabilities reflect different assumptions on the time taken for people with Crohn's disease to leave one health state and pass to another in the model. The EAG states that the constant hazard transition probabilities appear to be more appropriate for the model.
Results – responder model
5.49
The second base‑case results for the responder model show that the testing strategies are cheaper but less effective than the standard care strategy. Incremental costs (savings) compared with no testing are £11,800 for reflex testing and £10,700 for concurrent testing. Incremental QALYs (lost) compared with no testing are 0.2323 for reflex testing and 0.2447 for concurrent testing. Incremental cost‑effectiveness ratios (ICERs) show that if testing strategies were adopted, savings of between £43,700 and £50,800 would be made for each QALY lost.
5.50
Scenario analyses of the responder model included:
-
testing done annually in patients whose disease responds to treatment with a TNF‑alpha inhibitor
-
testing done first at 3 months and then annually in patients whose disease responds to treatment with a TNF‑alpha inhibitor
-
testing done only at 3 months in patients whose disease responds to treatment with a TNF‑alpha inhibitor, and in patients whose disease regains response after loss of response to TNF‑alpha‑inhibitor treatment
-
testing done only at 3 months in patients whose disease responds to treatment with a TNF‑alpha inhibitor (no testing of patients whose disease regains response after losing response to TNF‑alpha‑inhibitor treatment).
5.51
Results of scenario analyses show that the testing strategies are cheaper and less effective than the standard‑care strategy. Incremental costs (savings) compared with no testing range from £36,400 (annual testing) to £48,500 (testing at 3 months only in people whose disease responds). Incremental QALYs (lost) compared with no testing range from 0.2694 (testing at 3 months in people whose disease responds or regains response) to 0.2823 (annual testing). ICERs show that if testing strategies were adopted, savings of between £126,600 and £176,300 would be made for each QALY lost.
Results – loss of response model
5.52
The second base‑case results for the loss of response model show that the testing strategies are cheaper but less effective than the standard‑care strategy. Incremental costs (savings) compared with no testing are £84,800 for reflex testing and £86,100 for concurrent testing. Incremental QALYs (lost) compared with no testing are 0.2985 for reflex testing and 0.3154 for concurrent testing. ICERs show that if testing strategies were adopted, savings of between £273,000 and £284,100 would be made for each QALY lost.
5.53
A scenario analysis of the loss of response model examined a test schedule in which patients whose disease lost response to a TNF‑alpha inhibitor are tested, but patients whose disease regains response to treatment with a TNF‑alpha inhibitor are not tested. Testing is done every 3 months until the patient's disease regains response to the TNF‑alpha inhibitor, or the patient stops treatment with the TNF‑alpha inhibitor. Results show that the testing strategies are cheaper but less effective than the no‑testing strategy. Incremental costs (savings) compared with no testing are £118,100 for reflex testing and £119,600 for concurrent testing. Incremental QALYs (lost) compared with no testing are 0.3331 for reflex testing and 0.3508 for concurrent testing. ICERs show that if testing strategies were adopted, savings of between £340,900 and £354,500 per QALY lost could be made.
Sensitivity analyses
5.54
In addition to the scenario analyses, a range of univariate sensitivity analyses were done. These included:
-
Changing the time horizon from 10 years to 1 year.
-
In the no‑testing strategy arm, transition probabilities derived from Juillerat et al. (2015) were used for people whose disease lost response after dose escalation.
-
In the responder model, transition probabilities derived from Vande Casteele et al. (2015) were used.
-
Reducing the proportion of people with infliximab and antibodies to infliximab from 0.7878 to 0.200.
-
Changing the transition probabilities from exponential transition probabilities (which assume a constant hazard rate over time) to time‑to‑event transition probabilities.
-
Patients whose disease did not regain response after best supportive care.
5.55
Most of these changes had no impact on the direction of the results. However, changing the transition probabilities from exponential transition probabilities to time‑to‑event transition probabilities resulted in the testing strategies becoming more costly and less effective than the no‑testing strategy. Also, in the responder model, if patients' disease was assumed not to regain response after best supportive care, this resulted in the no‑testing strategy becoming cheaper than the testing strategies. Incremental QALYs also reduced, but the no‑testing strategy remained slightly more effective than the testing strategies.
5.56
In further sensitivity analyses, key model input parameters were varied to determine which inputs influence the ICER. Results showed that the models are stable to most changes, but sensitive to a 10% increase in the utility value for people whose disease regains response.
5.57
Probabilistic sensitivity analyses were done on the revised base‑case models. In the responder model, the scatterplot shows considerable uncertainty around both the incremental costs and incremental QALYs. The cost‑effectiveness acceptability curve suggests that there is a 50% probability of the no‑testing strategy being cost effective if the maximum acceptable ICER is £20,000 per QALY gained. It should be noted however, that this analysis is of the base‑case model in which testing was done every 3 months.
5.58
In the loss of response model, the scatterplot shows less uncertainty in the incremental costs but considerable uncertainty in the incremental QALYs. The cost‑effectiveness acceptability curve suggests that there is no preference between a no‑testing strategy and a testing strategy if the maximum acceptable ICER is £20,000 per QALY gained. However, if the maximum acceptable ICER is greater than £30,000 per QALY gained, a no‑testing strategy is likely to be the most cost‑effective strategy. Again, it should be noted that this analysis uses the base‑case model in which patients whose disease regained response to a TNF‑alpha inhibitor were tested every 3 months, in addition to testing of patients whose disease lost response.