Take off Your Disguise: Detecting Disguised Prompt-Based Jailbreak Attacks Against LLMs

Large language models (LLMs) are typically equipped with alignment mechanisms designed to prevent the generation of harmful content. However, recent advances have led to the emergence of highly evasive disguised prompt jailbreak attacks (DPJAs), where attackers conceal real malicious intent within prompts that appear benign on the surface, thereby inducing the model to produce unsafe outputs. In this work, we propose prompt reconstruction-based detection (RePrompt), a training-free and plug-in detection framework designed to identify such jailbreak attacks. The key insight of RePrompt is that, when guided by carefully designed templates, LLMs possess the capability to uncover disguised adversarial prompts and effectively detect jailbreak attacks. RePrompt leverages the reasoning capabilities of the LLM itself to analyze and uncover the true intent behind a user’s query. When a disguised jailbreak prompt is encountered, RePrompt reconstructs the underlying malicious intent hidden beneath the surface-level disguise. Experiments across multiple LLMs, datasets, and three representative DPJAs demonstrate that RePrompt consistently outperforms state-of-the-art defense methods, reducing the average attack success rate from 47.0% to below 1.6%, achieving a near-zero false positive rate, and limiting the average query cost to only 1.13.