Table 3

Characteristics of risk of bias tools and ad hoc criteria

ToolConfoundingBias in selection of participants into the studyBias in classification of the exposureBias due to departures from intended exposuresBias due to missing dataBias in measurement of the outcomeBias in selection of the reported resultsReporting qualityGeneralisabilityPrecision/sample sizeOther
Newcastle-Ottawa Scale38XXX XX XX Duration of follow-up
AHRQ Checklist for Cross-Sectional Studies60  X  X X   
Quality in Prognosis Studies61X X  XXXX Appropriateness of the statistical methods
ROBINS-I7XXXXXXX    
SIGN checklist for cohort studies62XXX XX X   
SIGN checklist for case-control studies62XXX XX X   
STROBE63       X   
Cochrane risk of bias tool9 X XXXX    
Critical Appraisal Skills Programme tools for cohort studies30X X XX XXXMagnitude of effect, Bradford-Hill Criteria
Data collection instrument and procedure for systematic reviews in the guide to community preventative services64XXXX X X  Appropriateness of the statistical methods, control for design effects, accounting for different levels of exposure in segments of the study population in the analysis
Effective Public Health Practice Project65X XXXX  X Appropriateness of the statistical methods
Mixed Methods Appraisal Tool66 XX XX XX  
NICE Methodological Checklist for Cohort Studies67X  XXX    Duration of follow-up
NICE Methodological Checklist for Case-Control Studies67XXX  X X   
NIH Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies31XXXXXX X XDuration of follow-up
Quality Assessment Tool for Systematic Reviews of Observational Studies (QATSO)68X    X  X Privacy and sensitive nature of the question considered
Research Triangle Institute Item Bank on Risk of Bias69XXXXXX X XStudy design, prespecification of the outcomes, duration of follow-up, missing outcomes, appropriateness of statistical methods, believability of the results, reporting of funding
Ad hoc criteria
Theal et al. 201870 (combined two sets of ad hoc criteria)XXXXXX X  Appropriateness of statistical methods
Beydoun et al. 201971X    X   XStudy design
Gianfredi et al. 2018
Gianfredi et al. 201872 73		XXX XX     
Asgari-Taee et al. 201874X  X    X Study design, appropriateness of statistical methods, validity of findings
Padilha et al. 201875X          
Dandamudi et al. 201876
(combined two sets of ad hoc criteria)		XXXX       
Dobbels et al. 201977  XXXX X X 
Dallacker et al. 201878  X  X  X Study design

  • AHRQ, Agency for Healthcare Research and Quality; NICE, National Institute for Health and Care Excellence; STROBE, Strengthening the Reporting of Observational Studies in Epidemiology.