Generating Realistic Adversarial User Comment Attacks to Evaluate the Robustness of Fake News Detectors

The rise of social media has amplified the spread of fake news, threatening societal stability. To address this, machine learning-based fake news detection systems analyze news content, including text, images, and user interactions. However, these systems are vulnerable to attacks that manipulate data, undermining their effectiveness. Studying these vulnerabilities is essential for improving fake news detectors. While much research has focused on attacks targeting news articles, less attention has been given to user comments, which play a crucial role in how news is perceived. Existing attack methods either reuse generic comments or generate new ones, but these approaches often fail to generate comments relevant or realistic enough to deceive detection systems. In this article, we focus on generating more realistic comments for specific news articles. We propose three novel attack strategies: two fine-tune a small language model using reinforcement learning algorithms transformer reinforcement learning and reinforced self-training, while the third uses a large language model with self-reflection. We conduct experiments showing that our methods outperform the state-of-the-art MALCOM procedure and that our proposed attack strategies produce more realistic comments, which are consequently more difficult to detect.