Development of the RTI item bank on risk of bias and precision of observational studies

Objective: To create a practical and validated item bank for evaluating the risk of bias and precision of observational studies of interventions or exposures included in systematic evidence reviews.

Study design and setting: The item bank, developed at RTI International, was created based on 1,492 questions included in earlier instruments, organized by the quality domains identified by Deeks et al. Items were eliminated and refined through face validity, cognitive, content validity, and interrater reliability testing.

Results: The resulting item bank consisting of 29 questions for evaluating the risk of bias and precision of observational studies of interventions or exposures (1) captures all of the domains critical for evaluating this type of research, (2) is comprehensive and can be easily lifted "off the shelf" by different researchers, (3) can be adapted to different topic areas and study types (e.g., cohort, case-control, cross-sectional, and case series studies), and (4) provides sufficient instruction to apply the tool to varied topics.

Conclusion: One bank of items, with specific instructions for focusing abstractor evaluations, can be created to judge the risk of bias and precision of the variety of observational studies that may be used in systematic and comparative effectiveness reviews.