Does Socially Desirable Responding Change After an Intervention? Implications for Estimating Treatment Effects

Survey measures are common in intervention research, but may suffer from biases such as socially desirable responding (SDR). While previous work demonstrates how unaddressed SDR can affect study results in observational contexts, the degree to which SDR may affect treatment effect estimates from randomized controlled trials (RCTs) is less studied. To shed light on this issue, we examine empirical data from 10 RCTs from education, psychology, and health fields and compare estimated treatment effects from models that do and do not account for SDR. Results show that, with two exceptions, estimated treatment effects were mostly unaffected by SDR. We complement our empirical results with a simulation study to determine the degree of SDR necessary to bias treatment effects and find that severe SDR can bias treatment effects when not accounted for in the model. However, models accounting for SDR are more complex and produce estimates that are slightly downwardly biased despite being the data-generating model. We conclude with a discussion of recommendations for researchers using survey instruments as outcomes in RCTs.