Fraud Detection in Incentive-Based Online Surveys Through Follow-up Demographic Verification

Online survey platforms like Qualtrics, SurveyMonkey, and Amazon Mechanical Turk have become essential tools for fast, cost-effective data collection, but growing concerns over data quality and the risk of fraud have accompanied their rapid growth. The purpose of this study was to determine whether follow-up demographic verification surveys can identify fraudulent participation and improve data quality in survey research conducted through online survey platforms. An orthopedic activity level survey with 35 embedded data quality checks was distributed using Amazon Mechanical Turk. A follow-up demographic verification survey was sent to 28 participants who contacted the survey administrator after completing the original survey. Five datasets were cleaned and merged using R, allowing for the identification of potential patterns of fraud through the direct comparison of demographic information between the original and follow-up surveys. Participants who did not complete the follow-up survey exhibited signs of fraud and poor data quality, including one email being linked to multiple participant accounts and higher counts of failed data quality checks. Although the small sample size limited the ability to detect statistical significance, descriptive patterns indicate practically meaningful differences. In contrast, demographic discrepancies were minimal among those who completed the follow-up. Open-text box similarity detection was the most effective individual data quality check. Integrating follow-up verification surveys into study designs provides a practical and scalable approach to detect fraud and maintain data integrity before compensation is distributed, offering researchers a cost-effective method to protect incentive budgets and data quality within online survey research. KEYWORDS: Online Survey Platforms; Data Quality; Online Research; Fraud Detection; MTurk; Participant Verification; Demographic Consistency; Incentive Structures; Follow-Up Surveys; Crowdsourced Data