Correcting for publication bias in multivariate and multilevel meta-analysis: A multivariate step function selection model approach

Univariate meta-analysis models assume that all effect sizes included in the analysis are independent. This assumption is violated if, for example, two outcomes are reported in a study that are of interest to the meta-analyst or a study reports multiple experiments administered by the same researchers in the same lab. The multivariate and multilevel meta-analysis models allow modelling dependent effect sizes. These two models have recently gained in popularity among meta-analysts in psychology, where one of the largest threats to validity of meta-analysis is publication bias where a selective subset of studies are not published. We extend the univariate step function selection model approach, for modelling publication bias, to multivariate and multilevel meta-analyses. We propose a strict and more relaxed selection model that assigns a different publication probability to studies that have only statistically significant outcomes or at least one significant outcome, respectively. We primarily focus on the method’s implementation as a sensitivity analysis where publication probabilities in the selection model are assumed as known but also show how these can be estimated. The method is applied to the data from multivariate and multilevel meta-analyses. Simulation studies tailored to these applications show that the proposed method as a sensitivity analysis outperforms the multivariate and multilevel meta-analysis models that do not correct for publication bias. The simulations also show that the publication probability is imprecisely estimated. We offer guidance for applying the proposed method in practice and share R code to facilitate the application of our proposed methodology.