NICE process and methods
Appendix G Quality appraisal checklist – quantitative studies reporting correlations and associations
Appendix G Quality appraisal checklist – quantitative studies reporting correlations and associations
A correlates review (see section 3.3.4) attempts to establish the factors that are associated or correlated with positive or negative health behaviours or outcomes. Evidence for correlate reviews will come both from specifically designed correlation studies and other study designs that also report on correlations.
This checklist[15] has been developed for assessing the validity of studies reporting correlations. It is based on the appraisal step of the 'Graphical appraisal tool for epidemiological studies (GATE)', developed by Jackson et al. (2006).
This checklist enables a reviewer to appraise a study's internal and external validity after addressing the following key aspects of study design: characteristics of study participants; definition of independent variables; outcomes assessed and methods of analyses.
Like GATE, this checklist is intended to be used in an electronic (Excel) format that will facilitate both the sharing and storage of data, and through linkage with other documents, the compilation of research reports. Much of the guidance to support the completion of the critical appraisal form that is reproduced below also appears in 'pop-up' windows in the electronic version[16].
There are 5 sections of the revised GATE. Section 1 seeks to assess the key population criteria for determining the study's external validity – that is, the extent to which the findings of a study are generalisable beyond the confines of the study to the study's source population.
Sections 2 to 4 assess the key criteria for determining the study's internal validity – that is, making sure that the study has been carried out carefully, and that the identified associations are valid and are not due to some other (often unidentified) factor.
Checklist items are worded so that 1 of 5 responses is possible:
++ |
Indicates that for that particular aspect of study design, the study has been designed or conducted in such a way as to minimise the risk of bias. |
+ |
Indicates that either the answer to the checklist question is not clear from the way the study is reported, or that the study may not have addressed all potential sources of bias for that particular aspect of study design. |
− |
Should be reserved for those aspects of the study design in which significant sources of bias may persist. |
Not reported (NR) |
Should be reserved for those aspects in which the study under review fails to report how they have (or might have) been considered. |
Not applicable (NA) |
Should be reserved for those study design aspects that are not applicable given the study design under review (for example, allocation concealment would not be applicable for case–control studies). |
In addition, the reviewer is requested to complete in detail the comments section of the quality appraisal form so that the grade awarded for each study aspect is as transparent as possible.
Each study is then awarded an overall study quality grading for internal validity (IV) and a separate one for external validity (EV):
-
++ All or most of the checklist criteria have been fulfilled, where they have not been fulfilled the conclusions are very unlikely to alter.
-
+ Some of the checklist criteria have been fulfilled, where they have not been fulfilled, or not adequately described, the conclusions are unlikely to alter.
-
– Few or no checklist criteria have been fulfilled and the conclusions are likely or very likely to alter.
Checklist
Study identification: Include full citation details |
|
|
Study design:
|
||
Guidance topic: |
||
Assessed by: |
||
Section 1: Population |
||
1.1 Is the source population or source area well described?
|
++ + − NR NA |
Comments: |
1.2 Is the eligible population or area representative of the source population or area?
|
++ + − NR NA |
Comments: |
1.3 Do the selected participants or areas represent the eligible population or area?
|
++ + − NR NA |
Comments: |
Section 2: Method of selection of exposure (or comparison) group |
||
2.1 Selection of exposure (and comparison) group. How was selection bias minimised?
|
++ + − NR NA |
Comments: |
2.2 Was the selection of explanatory variables based on a sound theoretical basis?
|
++ + − NR NA |
Comments: |
2.3 Was the contamination acceptably low?
|
++ + − NR NA |
Comments: |
2.4 How well were likely confounding factors identified and controlled?
|
++ + − NR NA |
Comments: |
2.5 Is the setting applicable to the UK?
|
++ + − NR NA |
Comments: |
Section 3: Outcomes |
||
3.1 Were the outcome measures and procedures reliable?
|
++ + − NR NA |
Comments: |
3.2 Were the outcome measurements complete?
|
++ + − NR NA |
Comments: |
3.3 Were all the important outcomes assessed?
|
++ + − NR NA |
Comments: |
3.4 Was there a similar follow-up time in exposure and comparison groups?
|
++ + − NR NA |
Comments: |
3.5 Was follow-up time meaningful?
|
++ + − NR NA |
Comments: |
Section 4: Analyses |
||
4.1 Was the study sufficiently powered to detect an intervention effect (if one exists)?
|
++ + − NR NA |
Comments: |
4.2 Were multiple explanatory variables considered in the analyses?
|
++ + − NR NA |
Comments: |
4.3 Were the analytical methods appropriate?
|
++ + − NR NA |
Comments: |
4.6 Was the precision of association given or calculable? Is association meaningful?
|
++ + − NR NA |
Comments: |
Section 5: Summary |
||
5.1 Are the study results internally valid (i.e. unbiased)?
|
++ + − |
Comments: |
5.2 Are the findings generalisable to the source population (i.e. externally valid)?
|
++ + − |
Comments: |