Evaluating Jailbreak Vulnerabilities in LLMs: A Taxonomy and Comparative Analysis in Romance Fraud Scenarios

We see that the romance fraud contexts let the jailbreak prompts get advice that breaks the rules from the language models. We present a taxonomy-driven benchmark. The benchmark evaluates three leading LLM families. We use 60 romance-themed jailbreak prompts in the benchmark. We sort the prompts into 12 strategy primitives. The strategy primitives include roleplay persona, agent mimicry, policy evasion, system override, format smuggling, translation obfuscation, emotional coercion, chain-of-thought coaxing, and no-refusal clause. We also group the prompts into the five higher-level categories. The categories are persona roleplay, safety bypass, system injection, coaxing frame, and obfuscation. We measure the Attack Success Rate, the refusal rate, and the latency. We provide the pairwise confusion analysis and the effect size analysis by the strategy and, by the category. We looked at the results. The results show differences in how easily each model can be fooled. The Gemini model had the ASR (0.983; refusal 0.017; 1.08s) on n=60 prompts. The OpenAI models had a level of vulnerability (ASR 0.533; refusal 0.467; 4.08s), on n=60 prompts. The Claude model had the ASR (0.250; refusal 0.750; 5.75s) on n=60 prompts. The high-yield attacks combined role play with refusal suppression rules or copying many agents. We see specific blind spots in the taxonomy.