Sampling for a cross-national health survey in six African countries using social media advertisements: comparison of different targeting strategies, types of estimates, and selectivity against population benchmarks

Abstract When researchers recruit participants for an online survey via Facebook advertisements, a targeting algorithm determines which individuals receive the survey invitation. Because the algorithm maximizes the number of clicks on the survey link, predominantly easy-to-reach population groups in a country or region might be reached (simple demographic targeting: SDT). To reduce representation bias, researchers can target selected demographic population groups separately (complex demographic targeting: CDT). In this study, we evaluate whether CDT effectively leads to less bias in univariate, bivariate, and multivariate estimates than SDT. This is done by comparing socio-demographic and health-related measures of health surveys recruited via Facebook with and without CDT in six African countries against probability-based survey benchmarks. Independent of the targeting strategy, our results show that many estimates were strongly biased, especially the univariate estimates of education and Internet use, whereas other estimates (e.g. HIV knowledge) were less affected. Although we found minor evidence that the CDT method reduced bias in univariate estimates, bias in relationship estimates stayed similar. Moreover, the study shows that univariate estimates are generally more often biased than relationship estimates and that bias is partially related to coverage issues. The success of weighting in bias reduction was mixed.