The I 2 Statistic as Selection Bias Test -Updated Threshold Limits (Preprint)

Aim: This study aims to revise the pre-specified I 2 point estimate threshold limits of the trialadjusted, SCT based I 2 test for selection bias in single randomised controlled trials (RCTs), in order to increase the percentage of testable RCTs. Further objectives were testing the null-hypotheses that, based on the test's revised I 2 point estimate thresholds, H01: the magnitude of trial effect estimates is not significantly positively correlated with the selection bias levels and H02: does not differ significantly between RCTs with identified 'low' and 'high' selection bias. Methods: Revision of the I 2 point estimate threshold limits was based on multiple RCT simulation and the re-testing of 332 real-world RCTs for selection bias. The selection bias level (B%) of each RCT, using the revised limits, was determined. H01 was tested using Spearman's rank correlation, and H02 using the 2-tailed, independent samples t-test. The mean difference (MD) with 95% Confidence interval (CI) between the mean absolute risk difference (RD) values with standard deviation (SD) of RCTs with identified 'low' and 'high' selection bias was computed. Results: All 332 RCTs were testable, and none of the computed I 2 point estimates fell outside the revised thresholds limits, increasing the proportion of testable RCTs from 71 to 100%. There was a statistically significant, positive, small correlation (0.1 ≤ |r| < 0.3) between the absolute RD values and established B% levels (Spearman's rho = 0.268, p < 0.0001). The mean absolute RD for RCTs with identified 'high' selection bias was 0.18 (SD = 0.16), compared with 0.10 (SD = 0.13) for RCTs with 'low' selection bias. The difference was statistically significant: (t =-4.65; p < 0.0001; MD 0.08; 95%CI: 0.05-0.11). The effect size (Cohen's d = 0.52) indicates a medium effect. Both nullhypotheses were rejected. Conclusion: The revised threshold limits increased the utility of the trial-adjusted, SCT based I 2 test from 71 to 100% testable RCTs. The rejection of both null-hypotheses indicates that the previously established statistically significant, positive relationship between selection bias levels (B%) and effect estimates (absolute RD) is independent of the changes in the I² threshold limits.