Modifiable methodological and reporting practices are associated with reproducibility of health sciences research: a systematic review and evidence and gap map

OBJECTIVES: Map the evidence on factors (eg, research practices) associated with reproducibility of methods and results reported in health sciences research. STUDY DESIGN AND SETTING: Five bibliographic databases were searched from January 2000 to May 2023 with supplemental searches of high-impact journals and relevant records. We included health science records of observational, interventional, or knowledge synthesis studies reporting data on factors related to research reproducibility. Data were coded using inductive qualitative content analysis, and empirical evidence was synthesized with evidence and gap maps. Factors were categorized as modifiable or nonmodifiable; reproducibility outcomes were categorized as related to methods or results. Statistical tests of association between factors and reproducibility outcomes were summarized. RESULTS: Our review included 148 studies, primarily from biomedical/preclinical (n = 62) and clinical (n = 71) domains. Factors were classified into 12 modifiable (eg, sample size and power) and three nonmodifiable (eg, publication year) categories. Of 234 reported evaluations of factors, 76 (32%) assessed methodological reproducibility and 158 (68%) assessed results reproducibility. The most frequently reported factor was transparency and reporting (38 of 234 assessments). A total of 155 factors (66%) were evaluated for statistical associations with reproducibility outcomes. Statistical associations were most frequently conducted for analytical methods (24 of 26 reporting significance), sample size and power (21 of 23 reporting significance), and participants characteristics and study materials (10 of 12 reporting significance). CONCLUSION: Several modifiable factors were associated with reproducibility of health sciences research and represent opportunities for intervention. Applying more stringent statistical testing procedures and thresholds, conducting appropriate sample size and power calculations, and improving transparency and completeness of reporting should be top priorities for improving reproducibility. Experimental studies to test interventions to improve reproducibility are needed. PLAIN LANGUAGE SUMMARY: Many research findings in medicine and health cannot be reproduced by other researchers. This makes it harder to know what evidence to trust when making patient care and health policy decisions. We systematically reviewed 148 studies that examined how specific research practices are linked to whether research findings can be reproduced. We found that several modifiable features of study design, analysis, and reporting were often associated with better reproducibility. Using larger sample sizes, applying more stringent statistical methods, and providing transparent, complete descriptions of methods and results were frequently linked to more reproducible findings. These results suggest that improving how studies are planned, analyzed, and reported may increase the trustworthiness and usefulness of health research.