NICE process and methods

11 The consultation process and dealing with stakeholder comments

11 The consultation process and dealing with stakeholder comments

Consultation with stakeholders, which lasts 6 weeks for standard clinical guidelines, is an integral part of the NICE clinical guideline development process. Comments received from stakeholders are a vital part of the quality-assurance and peer-review processes, and it is important that they are addressed appropriately. This chapter advises staff in National Collaborating Centres (NCCs) and the NICE Internal Clinical Guidelines Programme[16] on responding to stakeholder comments following consultation.

This chapter also includes information on what to expect during the consultation process. Circumstances in which a second consultation may be needed are also covered.

11.1 Principles of responding to stakeholder comments

This section describes how to respond to comments received from stakeholders about the draft guideline. The same principles apply when responding to comments on the draft scope (see section 2.6).

11.1.1 Responding to comments

Most comments will be received from registered stakeholders. These comments, and the responses to them, are sent to stakeholders with the advance copy of the full guideline, and are posted on the NICE website when the guideline is published (see section 12.2). Comments received from non-registered stakeholders, and comments received after the deadline for submission, are not considered and are not responded to; such comments will be returned to the sender.

11.1.2 Format of comments

All comments received by NICE are entered into a 'guideline consultation table' in a Microsoft Word file, which is sent to the NCC. The table contains the following information:

  • Organisation – name of the organisation that submitted the comments.

  • Document – full guideline or NICE guideline.

  • Section – this column can be used by the NCC and Guideline Development Group (GDG) to facilitate the identification of comments by section.

  • Page number.

  • Comments – comments received from stakeholders, which are entered unchanged.

  • Responses – blank column for the NCC and GDG to complete.

The GDG considers the comments received, and the NCC then responds to the comments. The following key points should be taken into account when responding to comments from stakeholders.

  • Each comment must be acknowledged and answered as fully and as factually as possible. It is important to acknowledge that each point has been seen and understood. Some comments may be presented as general commentary, but they should still be acknowledged.

  • If changes are made to the guideline as a result of the comment, this must be made clear in the response. If no changes have been made, it should be made clear why not.

  • For draft guidelines, responses to comments and changes to the guideline must be made with the agreement of the GDG before publication. The NCC must maintain an audit trail of changes.

Examples of responses to types of comments received during consultation on a clinical guideline are given in table 11.1.

Table 11.1 Examples of responses to stakeholder comments received on the clinical guideline Drug misuse: psychosocial interventions (NICE clinical guideline 51 [2007]; NCC for Mental Health)

Type of comment

Example of a response

Compliments about the guideline

Thank you for your comments.

A specific change was recommended and has subsequently been made

Thank you; we have changed 'legal' to 'pharmacy provided medication'.

Thank you for your comment; we have addressed this issue in section 7.6 of the full guideline.

A specific change was recommended and has subsequently been partially made

Thank you for your comment; we have added a section on families and carers in the introduction which draws together material on families and carers discussed in other parts of the guideline. We have incorporated some of your suggestions into the text.

A specific change was recommended and has subsequently NOT been made

Although we accept your comments on the use of oral fluid testing as an option for contingency management programmes there are a number of factors supporting the decision to consider urinalysis as the preferred method. Firstly, the longer drug detection time afforded by urinalysis. Secondly, there is a larger evidence base for urinalysis which is still the most established method of testing. Thirdly, urinalysis is less costly.

Asks for something that is outside the scope of the guideline

In response to your comment on alcohol, the scope of the guideline was concerned with drug misuse and did not include alcohol, although the issue of alcohol misuse in addition to primary drug misuse was considered where appropriate.

Concern about impact of the guideline

We appreciate that the impact upon benefits is an important issue and it is under consideration by the implementation team where it is within their remit to do so.

11.2 Consultation on the guideline

This section describes what to expect during the consultation phase. Draft versions of both the full guideline and the NICE guideline are consulted on.

11.2.1 Stakeholders

Draft versions of the full guideline and the NICE guideline are made available on the NICE website for the consultation. Registered stakeholders are informed by NICE that the documents are available.

11.2.2 External expert review

Although NICE does not routinely commission peer review from external experts, NCCs may occasionally consider arranging additional external expert review of part or all of a clinical guideline. These experts may include healthcare professionals, those commissioning care, social care professionals or people with a patient and carer perspective. This review may take place during guideline development or at the consultation stage. If it occurs during development, the process and comments remain confidential, but the adviser(s) should be named in the final full guideline. Comments from external expert advisers during the development of the guideline should be discussed by the whole GDG. If external advisers comment during consultation, their comments are responded to in the same way as comments from registered stakeholders and are published in the guideline consultation table on the NICE website under 'expert advisers'. All expert advisers are required to complete a declaration of interests form (see section 3.2.1).

11.2.3 NICE staff

NICE staff also comment on the consultation draft of the guideline, before and/or during the consultation. These staff include the Patient and Public Involvement Programme (PPIP) lead, the implementation lead and the lead editor for the guideline, as well as technical advisers (including a health economist), the Guidelines Commissioning Manager and the Centre for Clinical Practice (CCP) lead for the guideline.

Comments from NICE received during consultation are entered into the guideline consultation table and are responded to by the NCC in the same way as comments from registered stakeholders, but are not posted on the NICE website.

11.3 Considering a second consultation

In exceptional circumstances, the CCP Director may consider the need for a further 4-week stakeholder consultation. This additional consultation may be required after the standard 6-week consultation has ended if either of the following criteria has been met:

  • Information or data that would significantly alter the guideline has been omitted from the first draft.

  • Evidence was misinterpreted in the first draft of the guideline and the amended interpretation significantly alters the guideline.

The final decision on whether to hold a second consultation will be made by NICE.


[16] Information throughout this manual relating to the role of the National Collaborating Centres in guideline development also applies to the Internal Clinical Guidelines Programme at NICE.