The impact of bots and fraudulent responses on online survey data: the development of a 12-point data cleaning protocol used in a study of marginalized communities

Survey research faces the constant obstacle of bot and fraudulent responses that threaten to diminish the voices of marginalized communities. The objective of this study is to describe a data cleaning process, that a study focused on hard-to-reach teen populations undertook, to achieve quality data. Data were collected through a non-probability quota-based survey, designed and programmed in Qualtrics and distributed electronically. A literature review informed the development of a comprehensive 12-step cleaning process. A total of 16,702 responses were collected, only 5.5% (n = 918) were deemed usable. The raw data sample overrepresented Non-Latine (NL) white youth, older youth (ages 15–18 years), and responses from the North Central Healthy Chicago Zone (HCZ). The fraudulent responses presented positivity bias and overrepresented adverse mental health outcomes. Without proper data protection and cleaning protocols, we cannot understand the needs of marginalized communities thus limiting our ability to make effective intervention and meaningful change.