Industrialized Deception: The Collateral Effects of LLM-Generated Misinformation on Digital Ecosystems

Read full text
Authors: Alexander Loth, Martin Kappes, Marc-Oliver Pahl
Cited by: 2
References: 62

Abstract

Generative AI and misinformation research have evolved since our 2024 survey. This paper presents an updated perspective, transitioning from literature review to practical countermeasures. We report on changes in the threat landscape, including improved AI-generated content through Large Language Models (LLMs) and multimodal systems -- addressing the adverse impacts of AI on information quality. This work focuses on our practical contributions: JudgeGPT, a platform to evaluate human perception of AI-generated news, and RogueGPT, a controlled stimulus generation engine for research. Together, these tools form an experimental pipeline for studying how humans perceive and detect AI-generated misinformation. Our findings show that detection capabilities have improved, but competition between generation and detection continues. We discuss mitigation strategies such as LLM-based detection, inoculation approaches, and the dual-use nature of generative AI.

Fields of Study & Subjects

Misinformation and Its Impacts Hate Speech and Cyberbullying Detection Ethics and Social Impacts of AI

Publication details

Journal / source
Companion Proceedings of the ACM Web Conference 2026
Publisher
ACM
Pages
295–302
Publication date
28 May 2026