P-curving social and personality psychology experiments: Changes from 2006 to 2019

The replication crisis triggered critical reflection on several issues, most prominently, statistical errors and bias (e.g., p-hacking, irreplicability, false positives). This raises the question of whether published statistical results changed throughout psychology’s replication crisis. One tool for assessing the credibility of a set of statistical results is p-curve. Focusing on social and personality psychology, we present descriptive results from two projects that were initially independent. Following the p-curve guide, both teams extracted information about experiments’ key hypotheses, sample sizes, and key statistical results (among other things). The first team (Study 1) coded 1,526 experiments in 335 articles published in the Journal of Social and Personality Psychology: Attitudes and Social Cognition between 2006 and 2015. The second team (Study 2) coded 1,074 experiments in 507 articles published in Social Psychological and Personality Science between 2010 and 2019. The descriptive results suggest improvement during this critical period, however, they are also compatible with a range of scenarios and we therefore refrain from drawing strong conclusions. We invite readers to explore the data and consider the many different underlying research practices that could have produced these results. Our results provide constraints for discussions about the effects of the replication crisis and reform efforts.