"SAGE: Specification-Aware Grammar Extraction for Automated Test Case Generation with LLMs"
Abstract
"SAGE (Specification-Aware Grammar Extraction) is a novel framework that fundamentally advances automated test case generation (ATCG) for competitive programming. While prior state-of-the-art methods, such as LogiCase, introduced Context-Free Grammars with Counters (CCFGs) to formalize input constraints, they relied heavily on supervised fine-tuning (SFT), which is often hampered by the scarcity of high-quality, ground-truth labeled grammars. SAGE overcomes these limitations through three primary technical contributions. First, we propose a verifiable reward-guided reinforcement learning strategy utilizing Group Relative Policy Optimization (GRPO). By introducing the metric of \"well-formedness,\" SAGE enables the induction of robust syntactic patterns from unlabeled specifications, effectively mitigating data scarcity. Second, unlike conventional methods that rely on fixed-distribution random sampling, SAGE implements an LLM-driven adversarial strategy to dynamically select test case complexity based on the target solution\u2019s logic. This enables the generation of strategically adversarial test cases tailored to expose latent vulnerabilities in specific code implementations. Experimental results demonstrate that SAGE significantly outperforms 18 baseline LLMs, achieving a 15.92%p improvement in grammar validity and a 10.37%p gain in test effectiveness over existing methods. These contributions provide a more reliable, interpretable, and scalable foundation for software testing in complex algorithmic domains."
Fields of Study & Subjects
Software Testing and Debugging Techniques
Software Engineering Research
Teaching and Learning Programming
Publication details
- Journal / source
- IEEE DataPort
- Publication date
- 4 March 2026
- Language
- EN
- Document type
- Dataset