The guidelines manual

1.1 Is a qualitative approach appropriate?

A qualitative approach can be judged to be appropriate when the research sets out to investigate phenomena that are not easy to quantify or measure accurately, or where such measurement would be arbitrary and inexact.

Qualitative research in a health setting commonly measures:

If clear numerical measures could reasonably have been put in place, then consider whether a quantitative approach may have been more appropriate.

1.2 Is the study clear in what it seeks to do?

The design of qualitative research tends to be 'theory generative' rather than 'theory testing'; it is therefore unlikely that a research question will be found in the form of a hypothesis or null hypothesis in the way that you would expect in traditional quantitative research. Nevertheless, what the study is investigating should still be set out early and clearly. The research question should be set in context, with a summary of the background literature and the study's underpinning values and assumptions.

2.1 How defensible/rigorous is the research design/methodology?

There are a large number of qualitative methodologies, and a tendency in healthcare studies to 'mix' aspects of different methodologies or to use a generic qualitative method. From a qualitative perspective, this should not compromise the quality of the study if the research design captures appropriate data and has an appropriate plan of analysis for the subject under investigation.

Sampling in qualitative research can be purposive. Qualitative research is not experimental and does not purport to be generalisable, and therefore does not require a large or random sample. People are usually 'chosen' for qualitative research based on being key informers. The choice of sample and sampling method should be described, ideally including any shortcomings of the sample.

3.1 How well was the data collection carried out?

Assess whether the methods of data collection are described with details of the following:

Were these appropriate methods of data collection to use, given the aims of the research?

4.1 Is the context clearly described?

It is important when gauging the validity of qualitative data to consider whether the data are plausible and realistic. To make an accurate assessment of this, it is important to describe the context of the research in terms of the physical context (for example, youth club, GP surgery, gang headquarters) and who else was there (for example, participants are likely to position themselves very differently, and thus to respond very differently, in a discussion with parents present compared with a discussion with peers present). The participants should be described in enough detail to allow some insight into their life and situation and any potential context bias considered by the authors (that is, interpretation of the influence of the setting).

4.2 Were the methods reliable?

It is important that the method used to collect the data is appropriate for the research question and that the data generated map well to the aims of the study. Ideally, more than one method should have been used to collect data.

5.1 Are the data 'rich'?

Qualitative researchers use the adjective 'rich' to describe data that are in-depth, convincing, compelling and detailed enough that they can provide some insight into the research participants' experience. It is also important to know the 'context' of the data – where they came from, what prompted them, what they pertain to, and so on.

5.2 Is the analysis reliable?

The analysis of data can be made more reliable by the researchers putting checks in place. Sections of data should be coded by another researcher or, as a minimum, a second researcher should check the coding for consistency. Participants may also verify the transcripts of their interview (or other data collection, if appropriate). Negative or discrepant results should be highlighted and discussed.

5.3 Are the findings convincing?

The results of the research should be convincing or credible. Findings should be presented clearly and organised logically and the authors should consider and explain any contradictions. Extracts from original data should be included where possible to give a fuller sense of the findings. These data should be appropriately referenced – although you would expect data to be anonymised, they still need to be referenced in relevant ways (for example, if sex differences were important, then you would expect extracts to be marked male or female).

5.4 Are the conclusions adequate?

This section is self explanatory.

6.1 Was the study approved by an ethics committee?

All qualitative research involves ethical considerations, and these should be considered within any research report. Ideally there should be a full discussion of ethics, although this is rare because of space constraints in peer-reviewed journals. Any qualitative research should be approved by a research ethics committee, and this should be stated in the report so that it is clear that every care was taken to protect research participants.

6.2 Is the role of the researcher clearly described?

The researcher should have considered their role in the research; for example, as a reader, interviewer or observer. This is often referred to as 'reflexivity'. The 'status' of the researcher can profoundly affect the data. For example, a middle-aged woman and an 18-year-old man are likely to get different responses to questions about sexual activity when interviewing a group of teenage boys. It is important to consider age, gender, ethnicity and 'insider' status (such as whether the interviewer or researcher is part of the group being researched or has the same condition or illness). The researcher can also profoundly influence the data by use of questions, opinions, judgements and so on, so it is important to know what the researcher's position is in this regard, and how the researcher introduced and talked about the research with the participants.