Appendix M Fieldwork

Introduction

This appendix outlines points to consider when the decision has been made to conduct fieldwork for draft recommendations on a particular topic. Fieldwork is carried out only on an exceptional basis for public health guidance in new or sensitive areas, and not as a matter of routine on all guidance. Often, particularly for those topics where there is already related NICE guidance, fieldwork findings add little to the insights generated by the consultation with stakeholders. Nevertheless, for areas where NICE does not yet have good links with key practitioners and stakeholder groups it can be a valuable part of the process.

Exceptions where fieldwork would be carried out include occasions when NICE develops public health guidance in a new or (scientifically or politically) controversial topic area. For an exception to be made, the CPHE team will need to put the case for fieldwork to the centre director for approval. During fieldwork, the draft recommendations are tested with policy makers, commissioners and practitioners to see how easy they are to implement.

This appendix provides an overview of the fieldwork process including:

points CPHE consider when deciding to conduct fieldwork
aim of fieldwork
guiding principles of fieldwork
commissioning process
approaches used
way ethical approval is achieved
methods used
analysis of data
structure of the fieldwork report.

It concludes by describing how the fieldwork findings are used to finalise the recommendations.

Points to consider when deciding to conduct fieldwork

A number of issues help CPHE decide whether to conduct fieldwork in addition to the standard public consultation and evidence gathering processes. These issues can occur at different points during guidance development and will vary depending on the topic under consideration. Examples include considering whether NICE has developed any guidance at all (including any clinical guidelines) for a particular area, for example oral health, or taking into account any concerns raised by key stakeholders during the topic advisory workshops.

In other cases it may be that additional health inequalities are a concern, for example a referral to develop guidance for populations who find it difficult to engage with healthcare services.

In other circumstances it may be when a topic is particularly complex and requires a whole system approach. In this case the configuration of services may be a central component of the efficacy of a set of recommendations and input from a particular sub-set of healthcare professionals may be required.

Other issues to consider will be whether the topic includes an area of rapidly changing practice. For example, advances in some areas of frontline practice can occur quickly on the ground and reports or publication of evidence lags behind. In this example it may be necessary to test the draft recommendations with frontline practitioners, or providers or commissioners of services. In other cases it may be that key stakeholders express particular concerns during public consultation about the implementation or content of recommendations and request the CPHE consider testing some or all of them with particular service user groups.

These are just a few examples of the types of considerations CPHE will take into account at particular points during guidance development when deciding whether to commission fieldwork.

Aim of fieldwork

Fieldwork tests how easy it will be for policy makers, commissioners and practitioners to implement the draft recommendations and how the recommendations might work in practice. Practitioners may include members of the community, volunteers, parents and carers as well as professionals such as GPs, nurses and teachers.

Sometimes the views of people who are affected by the recommendations ('end users') will also be sought, either at this stage or earlier in the process. However, this is quite separate from fieldwork and would be commissioned as primary research or a separate 'end user consultation'. (Please refer to chapter 3 for more details.)

Practitioners' experience and views are used to fine-tune the recommendations. The aim is to ensure they are understood and interpreted as the public health advisory committee intended, even without supporting information. This is important because they should stand alone: the pathway and the implementation tools developed for the guidance contain little supporting information.

Guiding principles

Recommendations developed by public health advisory committee are based largely on evidence presented in the reviews, particularly evidence about the effectiveness and cost effectiveness of interventions and programmes. This allows assessments to be made about the plausibility of an intervention or programme, under different circumstances and with different populations.

However, the context in which practitioners will be implementing the recommendations also needs to be considered. The aim is to determine:

how feasible the recommendations are
what challenges practitioners will face
what opportunities exist to develop practice
what support might be needed.

Practitioner knowledge and experience is gathered during fieldwork to provide this type of evidence.

The following principles underpin the fieldwork phase.

Successful implementation depends on evidence-based recommendations informed by practical experience.
Practitioners know what is relevant to their current practice and the policy context.
Practitioners know what is feasible in practice.
High-quality fieldwork elicits practitioners' knowledge in a transparent, reliable and systematic way.
Draft recommendations are presented as indicative only of what might work.
Fieldwork allows participants' needs to be taken into account in the final recommendations.
Fieldwork seeks to test the impact of draft recommendations on inequalities in health, equity and diversity.

The exact questions participants are asked will vary from topic to topic, but 3 areas always need to be considered:

Content of the recommendations Are the recommendations relevant and appropriate for the groups responsible for delivering them? Are they clear and easy to understand for all these groups?
Practice What is current practice in the area? How might these recommendations build on or change current practice or service provision? What are the implications of this? Should any other elements be covered in the recommendations?
Impact What are the barriers to, and opportunities for, implementing the recommendations? What further resources, training or support might be needed? Are the recommendations relevant to other professional groups? What is the best way to get information to the range of professional groups involved? Are the recommendations sustainable? How would participants prioritise the draft recommendations?

Fieldwork commissioning

The CPHE project team should commission an academic or research organisation to carry out the fieldwork. This organisation should be separate from the review team involved in compiling evidence reviews for the guidance, unless there are exceptional circumstances. (An exception might be made if, for example, specific expertise in the topic, or access to specialist networks, is needed). However, the review team may be asked to help the fieldwork contractor, for example, by generating a list of participants.

The fieldwork contractor should have a good track record in health-related qualitative or participatory research and, ideally, should have experience in the guidance topic area.

The commissioning process should adhere to NICE's Standing Financial Instructions. This involves developing a project specification, inviting expressions of interest, issuing invitations to tender and selecting a contractor based on clear and auditable criteria.

Approaches to fieldwork

Fieldwork is a qualitative exercise; a range of views are required and it can involve a number of methods. The CPHE project team and the public health advisory committee should consider the choice of method(s) carefully, taking into account the topic, the professional groups involved and other issues. It may include the use of groups, 1-to-1 or paired in-depth interviews or surveys. In some cases – for example, if a range of professional groups are involved – a combination of approaches may be used.

The fieldwork contractor should agree the approaches to use with the CPHE project team. Similarly, the methodology and any questions or support materials used must be developed and agreed with the CPHE project team.

Group-based methods

Group-based methods include focus groups, participative workshops and 'virtual' (electronic) groups. These may be appropriate when:

potential participants have clear professional 'identities' and the 'field' is well established
NICE can contact enough professionals in a geographical region to set up a focus group or workshop
the issues discussed are unlikely to be confidential or professionally sensitive and anonymity will not be necessary.

1-to-1 or paired in-depth interviews

Interviews may be carried out face-to-face, by telephone or electronically. They may be appropriate when:

it is not possible to get groups of professionals together because it's a relatively new area, the number of possible participants is limited or there are geographic or time constraints
the issues discussed are likely to be confidential or professionally sensitive and anonymity may be needed
in-depth responses are needed.

Surveys

Group-based methods and 1-to1 or paired interviews (as outlined above) are the best way to find out participants' opinions. But they are not suitable in all circumstances, for example, because of the sensitivity of the topic, confidentiality issues, or difficulties in recruiting participants. In such cases, surveys that use semi-structured and open-ended questions could be more appropriate. Surveys may be carried out by telephone, electronically, on paper or by using vote casting or polling.

Surveys gather opinions in a quick, less obtrusive manner than group-based approaches and interviews. In addition, the responses can be quantified. But they do not allow the same depth of exploration and, generally, should only be used if other methods are unsuitable.

Ethical approval

In principle, fieldwork falls into the category of 'service evaluation' and so is outside the remit of NHS research ethics committees. However, the CPHE project team and the fieldwork contractor should consider the ethical issues each time fieldwork is planned. (If there is any doubt, the contractor should consult the National Research Ethics Service). The fieldwork contractor is responsible for seeking ethical approval, if required.

Fieldwork methods

The fieldwork contractor should agree any changes to the protocol for fieldwork with the CPHE project team before the fieldwork starts.

Fieldwork participants

Participants should be chosen to represent a broad range of professional groups in the statutory, non-statutory and voluntary sectors. This may include people who work with the target populations covered by the guidance and other users of the guidance, such as service commissioners. It may also include professionals working indirectly to promote the aspect of health covered by the guidance. Please note, fieldwork participants do not have to be from an organisation that is registered as a NICE stakeholder.

Equality issues should be fully considered when choosing fieldwork participants. This may mean getting a representative spread of professionals, but it may also mean focusing on participants with recent experience of working with disadvantaged groups. The approach should be based on the content of the recommendations, whether or not they refer to the whole population or subgroups of it and service delivery and policy issues.

Participants can include commissioners, policy officers, outreach workers, the police, representatives of the emergency services and the probation service, GPs, health visitors and educational welfare officers.

Participants should also:

have relevant experience and knowledge of the guidance topic
operate mainly (but not exclusively) in a regional or local capacity.

Sampling

Sampling should be guided by the topic. It will depend on the:

professional groups identified as being responsible for taking action (listed in the 'Who should take action?' part of the recommendation)
ultimate beneficiaries of the guidance (listed in the 'Who is the target population?' part of the recommendation)
scope
research questions
inclusion criteria for the evidence reviews.

'Snowballing' (gathering participants via other participants or networks) and purposive or other non-random techniques may be used to ensure all relevant professional groups are represented.

Random sampling (randomly selecting participants from the relevant professional groups) or quota sampling (selecting a fixed number of participants, randomly or purposively from these groups) may be useful for large-scale surveys. Random and quota sampling may also be useful where there are a large number of professionals, but there are not enough of them in each relevant geographical area.

The fieldwork report should explicitly state how participants were identified, sampled and recruited. The fieldwork contractor should ensure the final sampling frame and sample take account of the equity focus. It should be agreed with the CPHE project team.

Conducting the fieldwork

Once the method(s) has been chosen (see Approaches to fieldwork), a summary of the issues to be covered should be developed. The summary should be based on the draft recommendations (and related evidence statements) and the 3 key areas of inquiry outlined in Guiding principles. The quality of the fieldwork approach (including sample and method[s] selected) should be quality assured by the CPHE project team. The team should:

brief the fieldwork contractor in detail before work begins
agree final documents and comment on draft recruitment letters
help develop topic guides (summaries of the recommendations and key questions for discussion)
agree interview schedules or sampling frames and samples, and other supporting materials
discuss how to get participants who work with key or vulnerable groups involved
attend fieldwork groups and/or observe interviews (where possible)
have access to transcripts of all data
discuss and agree techniques for data analysis and themes for data presentation
comment on the fieldwork report before the final draft is submitted.

Group-based approaches

The criteria for organising group-based fieldwork are outlined below. They can be adjusted to accommodate a particular fieldwork process, although the fieldwork contractor should agree any deviations from the set criteria with the CPHE project team. For further information about preliminary work in this area, see Kelly et al. (2004).

The final sampling frame, recruitment and sample must be agreed with the CPHE project team and must take into account any planned comparisons between, for example, professions or geographical areas.
At least 6 fieldwork workshops or focus groups should be convened for guidance produced using the programme development process; at least 4 should be convened for guidance produced using the intervention process. These should take place in more than 1 geographical region and may be up to a day long. If it is not feasible to organise this many workshops or groups, the decision on how many should be convened must be agreed with the CPHE project team.
Independent professional facilitators should be selected to lead each meeting. A background in the topic under consideration is not essential, but it is essential to have knowledge of the public health community and the methods used to translate research evidence into practice. Facilitators also need an understanding of the wider public sector (and other sectors, as appropriate, according to the range of practitioners attending the fieldwork sessions). In addition, they need some knowledge of public health as a discipline.
A sample of up to 35 professionals (not including researchers) with a remit for the relevant public health topic should be invited to each meeting. If it is not feasible to invite 35 participants, the decision on how many to invite must be discussed and agreed with the CPHE project team.
If it suits the needs of the project, separate workshops or focus groups can be arranged for different professional groups. This will depend on the number of participants and should be agreed with the CPHE project team.
For some topic areas, researchers need to be included in the fieldwork. In such cases, a separate meeting should be convened for them, using the same processes. This should be agreed with the CPHE project team.
Topic guides, prompts or supporting materials (such as the draft recommendations, supporting evidence statements and the key areas of concern – see Guiding principles) must be developed in collaboration with, and agreed by, the CPHE project team.
A member of the NICE implementation team should attend at least 1 fieldwork meeting.
If possible, a member of the CPHE project team should attend each workshop or focus group to answer any technical questions and present the evidence statements and draft recommendations.

1-to-1 or paired, in-depth interviews

The sampling for 1-to-1 or paired, in-depth interviews should follow the procedures outlined in the Sampling section in Fieldwork methods. Specifically:

Sampling frames, techniques and recruitment must be agreed with the CPHE project team.
Samples and sample sizes must be agreed with the CPHE project team, taking into account any planned comparisons of professional groups that may be needed.
Interview schedules must be developed in collaboration with, and agreed by, the CPHE project team.
Any prompts or supporting materials (such as the draft recommendations, supporting evidence statements and the key areas of concern – see Guiding principles) should also be developed with, and agreed by, the CPHE project team.

Interviews may be structured or semi-structured, depending on the topic and the professional groups involved. Semi-structured interviews allow complex or difficult issues to be explored and so are likely to be more useful than a fixed-format interview. They should focus on the draft recommendations (and related evidence statements) and the 3 key areas of inquiry, as outlined in Guiding principles.

Individual or paired interviews are usually more expensive to set up than group work, and the need for in-depth or individual contact should be weighed against the available resources at the planning stage.

Survey methods

If survey methods are used (see the Surveys section in Fieldwork methods), sampling and recruitment should follow the principles outlined in the Sampling section in Fieldwork methods. The fieldwork contractor should agree the approach, the sampling frame, final sample, how the survey will be done, and survey questions or the use of supporting materials with the CPHE project team.

Distribution and day-to-day management of any surveys used is the responsibility of the fieldwork contractor.

Recording groups and interviews

The way groups and interviews are recorded depends on the method(s) used. For example, software that automatically produces transcripts is available for online focus groups. This would differ from the way a 'traditional' face-to-face focus group would be recorded. The plenary discussions, group work and interviews should all be recorded (for example, on tape or digitally and then transcribed, or by using a scribe). Previous experience has shown that stenography is the best way for a scribe to record the points. All participants should consent in writing to the recording – and to its use in discussions and group work.

If a scribe is used, he or she should accurately record points raised against each question (mediating factors, barriers and solutions). They may also categorise each point according to whether it relates to strategy and policy development, commissioning, management or individual practice.

Structure of the fieldwork

The structure of fieldwork events depends on the method(s) adopted. Presentations may be used, for example, to give an overview of the recommendations (and if appropriate, the guidance document). In such cases, they should be kept succinct to make best use of time and strike the right balance between passive and active participation.

Topic guides, generated by the contractor (and approved by the CPHE project team), should be circulated to fieldwork participants before or at the beginning of sessions to facilitate discussions.

Workshops, focus groups and interviews should be based on the structure outlined below. The structure will differ for surveys, but the elements outlined below should still be covered. If the fieldwork contractor wants to make any changes to the content and structure of sessions, it should agree them beforehand with the CPHE project team.

Session 1: purpose of the meeting

The lead facilitator explains the aims and objectives for the day and the values underpinning NICE's fieldwork process (see Points to consider when deciding to conduct fieldwork and Aims of fieldwork). The facilitator describes the guidance development process, introduces the draft recommendations and describes how participants will help refine them. They should make explicit reference to inequalities – and to the importance of judging the impact of interventions on different segments of the target population. The initial presentations may take around 20 minutes.

Session 2: participants' working environments

Participants consider the draft recommendations and comment on the context in which they operate. Social, political and economic factors relevant to participants' work – and the communities that they serve – may be raised here. This session may last for up to an hour.

Session 3: appraisal of draft recommendations and evidence statements

Group work can be run in different ways, depending on the project. For example, a large group convened on the same day can be subsequently divided into 4 or 5 multidisciplinary groups, each working with a facilitator. Alternatively, different sessions set up on different days can be arranged for each professional group – each with a facilitator.

If interviews (rather than groups) are being used, a series of 1-to-1 or paired sessions may also be set up.

Discussion focuses on the following question:

'Given that the evidence suggests that a particular kind of intervention/activity has worked in the following circumstances, and that this should form the basis of a recommendation, what would need to be done to make it work in your local situation?'

A follow-up prompt is:

'If this would not work, why not – and what would?'

Social and marketing research techniques can sometimes be useful (such as role play). It can also be useful to develop tools to help participants assess the feasibility and impact of each recommendation (for example, electronic key pads or q-sort techniques to help prioritise and sort sets of standard statements). Any techniques or tools used would need to be agreed with the CPHE project team.

Participants should also be asked to address the implications for health inequalities and generally for their own practice or profession. To conclude, discussion could consider barriers to – and facilitators of – change, including potential local drivers for change.

Other issues that may be raised include:

political drivers and imperatives for activity planning
decision and influence
partnerships
budgets
stakeholders
consultation
commissioning
shared data and information services
performance management
prioritisation of recommendations
examples of local good practice that may support the recommendations.

Session 4: feedback

At the end of group sessions, the facilitator may provide plenary feedback about the participants' view on implementation barriers, opportunities and solutions. Case study templates should be distributed for participants to note any points that have come up during the day or to submit case studies of local 'good practice'. These should be collected at the close of the session, or returned to the team by a specified date. Forms should be clearly marked with instructions for completion and return.

Evaluation and follow-up

At the end of each workshop, an evaluation of it may be completed by all participants. The facilitators and members of the CPHE project team should then liaise to share notes and transcripts. The fieldwork contractor should use these as a basis for the fieldwork report. All original notes should be retained. The draft fieldwork report should be circulated to all participants to check for accuracy. If separate reports are produced for each event, participants should check the report about the event they attended. These individual reports should form the basis for the fieldwork report considered by the public health advisory committee.

Fieldwork analysis

Fieldwork analysis is dependent on the method(s) adopted but the following points are a guide. The fieldwork contractor should agree the way data are presented and analysed with the CPHE project team.

Data presentation

The fieldwork groups' discussions and/or interviews should be transcribed in full (electronically or by hand). If a survey approach has been used, responses should be collated, transcribed or recorded in full.

Analysis

The methods used for the fieldwork will affect how the analysis is done.

For group-based fieldwork, data analysis should begin as soon as possible, and preferably in time to be included at the next fieldwork session. This approach may also be feasible if interviews and surveys are being used.
For 1-to-1 interviews and surveys, analysis is usually carried out at the end of data collection.

Analysis may be performed using qualitative research software, or by hand, but the method should be fully reported in the fieldwork report.

The fieldwork data should be broken down into common and consistent themes, framed by the research questions, using a content analysis approach. Usually, 1 researcher should prepare an initial analysis. This should be verified by 'blind' coding and sorting of a sample of the transcript by a second researcher. For examples of this kind of analysis, see part 3 (chapters 7–13) of Silverman (2004) or Ritchie and Spencer (1993).

Once the analysis is complete, participants' quotes should be selected to illustrate each theme. These quotes should be coded to keep participants anonymous and to allow the quotes to be distinguished (see the Data presentation section in Fieldwork analysis).

Fieldwork report

The fieldwork report sent to public health advisory committee is a summary of analysed fieldwork data and key points arising from it. A copy of the fieldwork report should also be sent to all participants.

Style and transcription notation

The fieldwork report should describe the aims of the fieldwork, the methodology used and the findings, drawing conclusions about how the guidance can be improved. The main section should cover the findings, summarising the emerging themes. It should be illustrated with verbatim quotes from participants as follows:

Quotes should not be edited, other than to clarify where text is not clear. If an extra word is needed to make sense of a quote, it should be put in square brackets […] to indicate a word has been inserted.
Short quotes should be inserted into the text and should be clearly marked with double quotation marks (" "). Longer quotes should be presented as inset paragraphs with double quotation marks.
If words from the quote are omitted, they should be replaced by '(…)', but the omission must not alter the meaning of the original quotation.
Quotes should be coded to keep the participant anonymous and allow for the distinction between different participants' comments.
As with data from clinical trials, transcripts should be kept for at least 5 years (see www.ct-toolkit.ac.uk).

Using fieldwork findings to inform final recommendations

The fieldwork contractor should present a summary of the findings to the public health advisory committee. It should use this information to refine and prioritise the recommendations after the consultation. (This includes making them more specific for different groups of practitioners, where appropriate.)

For further details about developing and prioritising recommendations, see chapter 7.

Equality and diversity

Equality and diversity issues should be considered at every stage of the fieldwork process – from commissioning the contractor to finalising the fieldwork report. For example, the fieldwork contractor should make every effort to ensure equality and diversity issues are considered when generating the fieldwork sample and the questions to be asked. These issues should also be considered when deciding on the approaches to use. In addition, fieldwork should specifically seek to determine:

Does the guidance avoid unlawful discrimination?
Are there ways in which the guidance could better promote equality?

For further information, see section 7.6.

References and further reading

Green J, Thorogood N (2004) Qualitative methods for health research: qualitative methods for health. London: Sage

Kelly MP, Chambers J, Huntley J et al. (2004) Method 1 for the production of effective action briefings and related materials. London: Health Development Agency.

Pope C, Ziebland S, Mays N (2000) Qualitative research in health care: analysing qualitative data. BMJ 320 (7227): 114–6

Silverman D, editor (2004) Doing qualitative research: a practical handbook. London: Sage Publications

Ritchie J, Spencer L (1993) Qualitative data analysis for applied policy research. In: Bryman A, Burgess R, editors. Analysing qualitative data. London: Routledge 173–94

Tashakkori A, Teddlie C (2002) Handbook of mixed methods in social and behavioural research. London: Sage

Methods for the development of NICE public health guidance (third edition)

NICE process and methods