NICE process and methods

13 The consultation process and management of stakeholder comments

13 The consultation process and management of stakeholder comments

Consultation with stakeholders lasts 4 weeks for NICE good practice guidance (GPG), and is an integral part of the development process. Comments received from stakeholders are a vital part of the quality-assurance and peer‑review processes, and it is important that they are addressed appropriately.

13.1 Principles of the consultation process

The draft GPG is made available on the NICE website for consultation and registered stakeholders are able to submit comments within the specified timeframe. External expert advisers may submit comments during the consultation period (see section 5.5).

NICE staff may also comment on the consultation draft of the GPG before and/or during the consultation. These staff could include the Public Involvement Programme lead, the implementation lead and the lead editor for the guidance. Comments from NICE staff received during consultation are not posted on the NICE website.

13.1.1 Communications

The consultation period is publicised in a number of ways:

  • alerting registered stakeholders via email when the public consultation period opens

  • notifying individuals signed up to receive NICE medicines and prescribing news and alerts

  • updating the specific topic guidance development page on the NICE website

  • listing the consultation on the 'current NICE consultations' page.

13.1.2 Submitting comments

When the draft GPG is published for consultation, the following are also published:

  • instructions and guidance for submitting comments

  • a 'comments proforma' (to be used by stakeholders).

Completed proformas are sent to the dedicated email address for the GPG. Comments not submitted on the correct proforma, or which do not use the proforma correctly, are returned to the sender for re-submission. The NICE project team will only accept 1 response from each registered stakeholder organisation. Comments received from non-registered stakeholders are only considered if the organisation subsequently registers as a stakeholder. Comments received after the deadline for submission are not considered and are not responded to.

Because all comments from registered stakeholders are made public on the NICE website, confidential information (such as information about individual patients) must not be included.

The document Protocol for managing guidance consultation comments contains further details about how NICE deals with stakeholder comments received during consultation.

13.2 Principles of responding to stakeholder comments

Most comments are received from registered stakeholders. These comments, and the responses to them, are published on the NICE website when the GPG is published (see section 14).

13.2.1 Documentation of comments

All comments received by NICE are entered into a GPG 'consultation comments table' in a Microsoft Word file, which contains the following information:

  • Stakeholder organisation – name of the organisation that submitted the comments.

  • Section, page and line number – these columns can be used by the project team and Guidance Development Group (GDG) to facilitate the identification of comments by section.

  • Comments – comments received from stakeholders, which are entered unchanged.

  • Responses – to be completed by the NICE project team and GDG.

13.2.2 Responding to comments

The GDG considers the comments received, and the NICE project team then responds to the comments on their behalf. The following key points are taken into account when responding to comments from stakeholders.

  • Each comment must be acknowledged and answered as fully and as factually as possible. It is important to acknowledge that each point has been seen and understood. Some comments may be presented as general commentary, but they should still be acknowledged.

  • If changes are made to the GPG as a result of the comment, this must be made clear in the response. If no changes are made, it should be made clear why not.

  • For draft GPG, responses to comments and changes to the GPG must be made with the agreement of the GDG before publication. The NICE project team must maintain an audit trail of changes.

Examples of responses to types of comments received during consultation on the GPG are given in table 13.1.

Table 13.1 Examples of responses to stakeholder comments received on Developing and updating local formularies (NICE good practice guidance 1 [2012])

Type of comment

Example(s) of a response

Compliments about the GPG

Thank you for your comments. No response required.

A specific change was recommended and has subsequently been made

Thank you for your comment. The NICE definition of the term 'option for treatment' has now been incorporated into the good practice guidance. See recommendation 2.6.2.

A specific change was recommended and has subsequently been partially made

Thank you for your comment. While the GDG recognised the importance of medicines optimisation, it concluded that this good practice guidance is concerned with systems and processes relating to local formulary development, and it was not appropriate to include a detailed approach to medicines optimisation. To do so would be outside scope. However, the GDG agreed that optimising the use of medicines to improve patient care may be a potential benefit of a local formulary and this has been incorporated into the guidance.

A specific change was recommended and has subsequently NOT been made

Thank you for your comment. While the GDG acknowledge that this may be potentially complex, it would still constitute good practice.

Thank you for your comment. The GDG concluded that it was not appropriate to have definitive lists of possible clinicians and organisations. These will be for local determination.

Asks for something that is outside the scope of the GPG

Thank you for your comment. The NICE review process for technology appraisals is outside the scope of this good practice guidance.

The consultation comments table is signed off by the project lead, the Programme Director and the Centre for Clinical Practice (CCP) Director.

The NICE project team may refer some comments, if appropriate, to the Programme Director for consideration before sign off.

13.3 Considering a second consultation

In exceptional circumstances, the Centre for Clinical Practice Director may consider the need for a further 4‑week stakeholder consultation, which may be required after the standard 4‑week consultation has ended if either of the following criteria has been met:

  • Information or data that would significantly alter the GPG was omitted from the first draft.

  • Evidence was misinterpreted in the first draft of the GPG and the amended interpretation significantly alters the guidance.

The final decision on whether to hold a second consultation is made by NICE.