"SAGE: Specification-Aware Grammar Extraction for Automated Test Case Generation with LLMs"

Read full text
Authors: Sang‐Ki Ko, Hyunwoo Park, Aditi Aditi, Sicheol Sung, Yo-Sub Han
Cited by: 0

Abstract

"SAGE (Specification-Aware Grammar Extraction) is a novel framework that fundamentally advances automated test case generation (ATCG) for competitive programming. While prior state-of-the-art methods, such as LogiCase, introduced Context-Free Grammars with Counters (CCFGs) to formalize input constraints, they relied heavily on supervised fine-tuning (SFT), which is often hampered by the scarcity of high-quality, ground-truth labeled grammars. SAGE overcomes these limitations through three primary technical contributions. First, we propose a verifiable reward-guided reinforcement learning strategy utilizing Group Relative Policy Optimization (GRPO). By introducing the metric of \"well-formedness,\" SAGE enables the induction of robust syntactic patterns from unlabeled specifications, effectively mitigating data scarcity. Second, unlike conventional methods that rely on fixed-distribution random sampling, SAGE implements an LLM-driven adversarial strategy to dynamically select test case complexity based on the target solution\u2019s logic. This enables the generation of strategically adversarial test cases tailored to expose latent vulnerabilities in specific code implementations. Experimental results demonstrate that SAGE significantly outperforms 18 baseline LLMs, achieving a 15.92%p improvement in grammar validity and a 10.37%p gain in test effectiveness over existing methods. These contributions provide a more reliable, interpretable, and scalable foundation for software testing in complex algorithmic domains."

Fields of Study & Subjects

Software Testing and Debugging Techniques Software Engineering Research Teaching and Learning Programming

Publication details

Journal / source
IEEE DataPort
Publication date
4 March 2026
Language
EN
Document type
Dataset