How Does Model (Mis)Specification Impact Statistical Power, Type I Error Rate, and Parameter Bias in Moderated Mediation? A Registered Report

Moderated mediation models are commonly used in psychological research and other academic fields to model when and how effects occur. Researchers must choose which paths in the mediation model are moderated when specifying this type of model. While the ultimate goal is to specify the model correctly, researchers may struggle to determine whether to err on the side of including too many moderated paths (maximalist approach) or including too few moderated paths (minimalist approach). This registered report examines how the specification of moderation impacts statistical power, type I error rate, and parameter bias for the index of moderated mediation. In a systematic review of papers published using moderated mediation over the course of one year, we found that six model specifications account for 85\% of published moderated mediation analyses and the median sample size was 285. We ran a Monte Carlo simulation study to examine the impacts of model specification on power and type I error rate, and results were analyzed using multilevel logistic regression. In reference to the data-generating process, the data analysis model can either be correctly specified, over-specified, under-specified, or completely misspecified. Over-specified models were hypothesized to have lower statistical power to detect a significant index of moderated mediation compared to correctly specified models and relatively low parameter bias. Under-specified models were hypothesized to have lower statistical power than correctly specified models but unacceptably high parameter bias. Completely misspecified models were hypothesized to have inflated type I error rates and unacceptable parameter bias. Based on these results we recommend researchers lean more toward maximalist approaches to avoid parameter bias, but acknowledge that this comes with a cost in statistical power.