Single technology appraisal and highly specialised technologies evaluation: User guide for company evidence submission template

Sections 2.2 to 2.6 of the submission should include only the trials that were included in the economic model. If you wish to include additional studies in sections 2.2 to 2.6, which were not included in the economic model but are relevant to your submission (for example, natural history data to support severity assumptions), please provide your rationale using the following format:

[Study name] was not used to populate the economic model but is included in sections 2.2 to 2.6. The results of this study support [include details of why they are relevant]. This study was not included in the economic model because [add rationale].

Items 3 to 6b of the CONSORT checklist should be provided for all RCTs identified in section 2.2 as relevant to your submission.

• Trial design – brief description of trial design, including details of randomisation if applicable.

• Eligibility criteria – a comprehensive description of the eligibility criteria used to select the trial participants, including any definitions and any assessments used in recruitment.

• Settings and locations where the data were collected – describe the locations where the trial was carried out, including the country and, if applicable, the care setting (for example, primary care [GP or practice nurse], secondary care [inpatient, outpatient, day case]).

• Trial drugs and concomitant medications – provide details of trial drugs and comparator(s), with dosing information and titration schedules if appropriate. Provide an overview of concomitant medications permitted and disallowed during the trial.

• Outcomes used in the economic model or specified in the scope, including primary outcome. This should always include the primary outcome even if it is not used in the economic model. Please state if the outcomes were pre-specified or post-hoc analyses.

Provide a comparative summary of the methodology of the trials in a table. See the following suggested table format.

In a table, describe the characteristics of the participants at baseline for each of the trials in your submission. Include separate tables for the full trial population and if different, the population included in the decision problem and any subgroups. Provide details of baseline demographics, including age, sex and relevant variables describing disease severity and duration and appropriate previous treatments and concomitant treatment. Highlight any differences between trial groups. See the following suggested table format.

Clearly describe the methods used for expert elicitation or expert opinion, including the identification and selection of experts, and the reporting of results including the consensus of opinions or data aggregation. Follow existing reporting guidelines when possible.

See section 3.3.14 of NICE's health technology evaluations manual for additional guidance on the design, conduct and reporting of non-randomised and real-world studies.

During completion of this section consider items 7a (sample size), 7b (interim analyses and stopping guidelines), 12a (statistical methods used to compare groups for primary and secondary outcomes) and 12b (methods for additional analyses, such as subgroup analyses and adjusted analyses) of the CONSORT checklist.

For each study identified in 2.2 as relevant to your submission, provide details of the study population included in the primary analysis of the primary outcome and methods used to take account of missing data (for example, a description of the intention-to-treat analysis carried out, including censoring methods, or whether a per-protocol analysis was carried out).

For each study, provide details of the statistical tests used in the primary analysis. Also provide details of the primary hypothesis or hypotheses under consideration, the power of the trial and a description of sample size calculation, including the rationale and assumptions in a table. If the outcomes were adjusted for covariates, provide the rationale. A suggested table format is presented at the end of this section.

For non-randomised and non-controlled evidence such as observational studies, the potential biases should be identified before data analysis, either by a thorough review of the subject area or discussion with experts in the clinical discipline. Ideally these should be quantified and adjusted for.

The validity of the results of an individual RCT or non-randomised or non-controlled study will depend on the robustness of its overall design and execution, and its relevance to the decision problem. The quality of each source of evidence identified as relevant to your submission in section 2.2 should be appraised. Whenever possible, the criteria for assessing published studies should be used to assess the validity of unpublished and part-published studies. The quality assessment will be validated by the evidence review group.

Describe the methods used for assessing risk of bias and generalisability of individual trials (including whether this was done at the study or outcome level), and how this information is to be used in any data synthesis.

• The following are the minimum criteria for assessment of risk of bias and generalisability in parallel group RCTs, but the list is not exhaustive:

  • Was the randomisation method adequate?

  • Was the allocation adequately concealed?

  • Were the groups similar at the outset of the study in terms of prognostic factors, for example, severity of disease?

  • Were the care providers, participants and outcome assessors blind to treatment allocation? If any of these people were not blind to treatment allocation, what might be the likely impact on the risk of bias (for each outcome)?

  • Were there any unexpected imbalances in dropouts between groups? If so, were they explained or adjusted for?

  • Is there any evidence to suggest that the authors measured more outcomes than they reported?

  • Did the analysis include an intention-to-treat analysis? If so, was this appropriate and were appropriate methods used to account for missing data?

  • Also consider whether the authors of the study publication declared any conflicts of interest.

• In addition to parallel group RCTs, there are other randomised designs (for example, randomised crossover trials and randomised cluster trials) in which further quality criteria may need to be considered when assessing bias. Key aspects of quality to be considered can be found in Systematic reviews: CRD's guidance for undertaking reviews in health care (University of York Centre for Reviews and Dissemination).

• For the quality assessments of non-randomised and non-controlled evidence, use an appropriate and validated quality assessment instrument. Key aspects of quality to be considered can be found in Systematic reviews: CRD's guidance for undertaking reviews in health care (University of York Centre for Reviews and Dissemination). This includes information on several initiatives aimed at improving the quality of research reporting. Include consideration of the following:

  • Was the cohort recruited in an acceptable way?

  • Was the exposure accurately measured to minimise bias?

  • Was the outcome accurately measured to minimise bias?

  • Have the authors identified all important confounding factors?

  • Have the authors taken account of the confounding factors in the design or analysis, or both?

  • Was the follow up of patients complete?

  • How precise (for example, in terms of confidence intervals and p values) are the results?

Consider how closely the studies reflect routine clinical practice in England.

If there is more than 1 study, tabulate a summary of the responses applied to each of the quality assessment criteria. Suggested table formats for the quality assessment results are as follows.

Present results for all outcomes that inform the economic model or are specified in the scope from the studies identified as relevant to your submission (including real-world studies when applicable). The primary outcome of the studies must be reported. Data from intention-to-treat analyses should be presented whenever possible and a definition of the included participants provided. If participants have been excluded from the analysis, the rationale for this should be given. If the decision problem is for a narrower population than in the trial, the same information should also be presented for the population in the decision problem as well as any subgroups.

If you are presenting pooled results for multiple parallel trials, also present the results for each individual trial.

The information may be presented graphically to supplement text and tabulated data. If appropriate, please present graphs such as Kaplan–Meier plots.

For each outcome, provide the following information from each study:

• The unit of measurement.

• The size of the effect; for dichotomous outcomes, the results ideally should be expressed both as relative risks (or odds ratios) and risk (or rate) differences. For time-to-event analysis, the hazard ratio is an equivalent statistic. Both absolute and relative data should be presented.

• A 95% confidence interval.

• The number of people in each group included in each analysis and whether the analysis was intention to treat. State the results in absolute numbers when feasible.

• When interim data are quoted, this should be clearly stated, along with the point at which data were taken and the time remaining until completion of the trial. Analytical adjustments should be described to cater for the interim nature of the data.

• Other relevant data that may help interpret the results may be included, such as adherence to medication or study protocol.

• Discuss and justify any clinically important differences in the results between the different arms of a trial and between trials.

• Specify whether unadjusted and adjusted analyses were performed, and whether the results were consistent.

Present information on the subsequent treatments received in the trials identified as relevant to your submission. This should include the number and percentage of people receiving each treatment by arm and by line of treatment if multiple subsequent lines of treatment are applicable. Indicate where subsequent treatments are used in combination when reporting this data. A suggested table for reporting this data is provided at the end of this section.

Include information on both drug and non-drug subsequent treatments (for example, stem cell transplants, surgery and radiotherapy).

Provide details of the subsequent treatment data collected in the trial protocol, such as how many lines of treatment were considered and whether subsequent treatment data was collected by individual drug or by class.

Report the mean time spent on each subsequent treatment including a measure of uncertainty such as the 95% confidence interval. Report time to next treatment and PFS2 data where these have been collected with associated Kaplan–Meier plots.

Provide details of any subgroup analyses carried out. Specify the rationale and whether they were pre-planned or post-hoc.

Clearly specify the characteristics of the participants in the subgroups and explain the appropriateness of the analysis to the decision problem.

Provide details of the statistical tests used in the primary analysis of the subgroups, including any tests for interaction.

If a meta-analysis cannot be conducted and instead a qualitative overview is considered to be appropriate, summarise the overall results of the individual studies with reference to their critical appraisal.

If a meta-analysis has been performed, include the following in the results:

• The characteristics and possible limitations of the data (that is, population, intervention, setting, sample sizes and the validity of the evidence) should be fully reported for each study included in the analysis and a forest plot included.

• A statistical assessment of heterogeneity. If the visual presentation and/or the statistical test indicate that the RCT results are heterogeneous, try to explain the heterogeneity.

• Statistically combine (pool) the results for both relative risk reduction and absolute risk reduction using either a fixed effects or random effects model as appropriate.

• Provide an adequate description of the methods of statistical combination and justify their choice.

• Carry out sensitivity analysis when appropriate.

• Tabulate and/or graphically display the individual and combined results (such as through the use of forest plots).

If any of the relevant studies listed in section 2.1 are excluded from the meta-analysis, the reasons for doing so should be explained. The impact that each excluded study has on the overall meta-analysis should be explored.

In a table provide a summary of the trials used to carry out the indirect comparison or mixed treatment comparison. The following is a suggested table format. When there are more than 2 treatments in the comparator sets for synthesis, include a network diagram.

If the table or network diagram provided does not include all the trials that were identified in the search strategy, the rationale for exclusion should be provided.

Provide the results of the analysis. For examples of how to present the results, see the NICE Decision Support Unit technical support documents 1 to 3.

Provide the results of the statistical assessment of heterogeneity. The degree of heterogeneity, and the reasons for it, should be explored as fully as possible.

If there is doubt about the relevance of particular trials, present separate sensitivity analyses in which these trials are excluded.

Discuss any heterogeneity between results of pairwise comparisons and inconsistencies between the direct and indirect evidence on the technologies.

Evidence from comparative RCTs and regulatory summaries is preferred, but findings from non-comparative trials may sometimes be relevant. For example, post-marketing surveillance data may demonstrate that the technology shows a relative lack of adverse reactions commonly associated with the comparator, or that the occurrence of adverse reactions is not statistically significantly different to those associated with other treatments.

In a table, summarise the adverse reactions reported in the studies identified in section 2.2, as relevant to your submission, including those for the full trial population and the population in the decision problem, if different. For each intervention group, give the number with the adverse reaction and the frequency, the number in the group, and the percentage with the adverse reaction. Then present the relative risk and risk difference and associated 95% confidence intervals for each adverse reaction.

Provide a brief overview of the safety of the technology in relation to the decision problem.

Provide details of all completed and ongoing studies that should provide additional evidence in the next 12 months for the indication being appraised.

A statement of principal (interim) findings from the clinical evidence highlighting the clinical benefits and harms of the technology.

A discussion of the strengths and limitations of the clinical evidence base for the technology. This should include the following:

• A brief statement on the internal validity of the studies included in the clinical evidence base.

• A brief statement on the external validity of the studies included in the clinical evidence base. Include the relevance of the evidence base to the decision problem and the relevance of the outcomes assessed in clinical trials to the clinical benefits experienced by patients in practice. Identify any factors that may influence the external validity of study results to patients in routine clinical practice.