LLM Cheat Prevention via Adversarial Question Paraphrasing

As Large Language Model (LLM) chatbots have become increasingly accessible and their misuse for academic dishonesty has raised growing concern. Current methods that attempt to detect LLM-generated text are unreliable and risk producing false positives, which can unfairly harm genuine students. This paper offers an alternative by developing an “inoculation” process that generated paraphrased questions to find semantically similar ones that LLMs answer incorrectly. Using Llama 3.2 3B, create and evaluate paraphrases for MMLU questions, then test GPT-4o mini on them to identify effective inoculated questions. The approach successfully finds inoculations for 26.7% of correctly answered questions, requiring review of on more than 20 paraphrases per question. Detects weakness in instructor LLM responses with low cost.