Septemberember/SG4PM: Specification Generation for Program Maintenance via LLMs and Testing-Based Formal Verification
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Official Implementation of SG4PM Introduction π Background Formal specifications play an important role in program comprehension, verification, and maintenance. However, manually constructing precise specifications is costly, while automatically generated specifications may be incomplete or difficult to verify, especially for programs containing loops. Existing LLM-based specification generation approaches still face two major challenges: Loop-Invariant Dependency: Conventional formal verification usually requires correct loop invariants for programs containing loops. Generating such invariants is difficult and error-prone, even with large language models (LLMs). Specification Completeness: A generated specification may pass verification while describing only part of the program behavior, for example, by covering only a subset of the input space or imposing insufficient constraints on program outputs. SG4PM (Specification Generation for Program Maintenance) addresses these challenges by combining LLMs with Functional Scenario Form (FSF) and Testing-Based Formal Verification (TBFV). SG4PM provides the following core capabilities: FSF Generation: LLMs automatically generate Functional Scenario Form specifications from Java programs. An FSF represents program functionality as a set of input-output functional scenarios (Ti,Di). FSF Validation: The generated FSF is automatically checked for mutual exclusion and specification completeness, preventing overlapping or incomplete functional scenarios. TBFV-Based Consistency Verification: TBFV combines specification-based testing with Hoare-logic reasoning to verify the consistency between program code and its FSF without requiring loop invariants. Verifier-Guided Refinement: When validation or verification fails, diagnostic information is returned to the LLM to guide iterative refinement of the generated FSF. FSF-Driven Program Maintenance: Users can modify an FSF to express changed functional requirements. The modified FSF then guides the LLM to update the original program, after which TBFV verifies whether the updated implementation is consistent with the modified specification. The overall workflow is: Program Code β LLM-based FSF Generation β FSF Validation β TBFV Consistency Verification β Verified FSF β User-modified FSF β LLM-based Code Update β TBFV Re-verification β Updated Program Through this workflow, SG4PM connects specification generation, formal verification, and program maintenance within a unified verifier-guided framework. π Quick Start 1. Download & Extract https://github.com/Septemberember/PCaE Download PCaE-release.zip from Releases. Ensure the directory structure is as follows: . βββ PCaE-1.0.jar # Executable Fat JAR βββ resources/ # Resources folder (must be in the same directory as the JAR) 2. Setup Environment Java 17+: Ensure Java 17 or higher is installed. Verify by running: java -version Python 3.9+: Required for the Z3 SMT solver. Install the dependency via: pip install z3-solver 3. Configure API Key Navigate to resources/config/ and edit the configuration file corresponding to your preferred LLM (e.g., gpt-5.2.txt): [name]=openai/gpt-5.2 [url]=https://openrouter.ai/api/v1/chat/completions [apikey]=your_real_openrouter_key_here [!TIP]Formatting Matters: Ensure that there are no spaces around the = sign in the configuration file. Keep your [apikey] secure and never commit it to a public repository. 4. Execution Run the following command from the project root. Windows / Linux / macOS: java -cp PCaE-1.0.jar org.pcae.verification.FSFGenerator --inputDir resources/dataset --model gpt-5.2 --maxRounds 3 --experimentName Demo_Run The main parameters are: --inputDir: Directory containing the Java programs to be analyzed. --model: LLM configuration used for FSF generation. --maxRounds: Maximum number of verifier-guided interaction rounds. --experimentName: Name of the output directory for the current run. 5. Check Results All outputs will be generated in a folder named Demo_Run. The output contains: FSF Files: Functional Scenario Form specifications automatically generated by the LLM. Validation Results: Results of FSF mutual-exclusion and specification-completeness checking. Verification Logs: Detailed TBFV verification information for checking consistency between the Java program and the generated FSF. Intermediate Results: Execution and interaction information generated during verifier-guided FSF refinement. A successfully verified FSF provides an explicit representation of the functional behavior of the corresponding program and can subsequently be modified to support program maintenance. π§ FSF-Driven Program Maintenance SG4PM supports program maintenance through explicit modification of verified FSF specifications. A user can modify the testing condition T and/or definition condition D of selected functional scenarios to represent changed requirements. SG4PM then uses the modified FSF together with the original program to guide the LLM in generating an updated implementation. The maintenance workflow is: Verified FSF β Modify T and/or D β Validate Modified FSF β LLM-based Program Update β TBFV Verification β Functionally Consistent Updated Program TBFV is applied again after code modification to verify that the updated program is consistent with the modified FSF. π» Development & IDE Setup If you prefer to run, inspect, or modify the source code using IntelliJ IDEA, follow the steps below. 1. Project Initialization Open the PCaE folder as a Maven project. Maven will automatically resolve the dependencies defined in pom.xml. 2. Configure Run Arguments To pass parameters to FSFGenerator: Navigate to: src/main/java/org/pcae/verification/FSFGenerator.java Right-click and select Modify Run Configuration... Enter the desired parameters in the Program arguments field: --inputDir resources/dataset --model gpt-5.2 --maxRounds 3 --experimentName IDE_Test 3. Build from Source (Optional) If you have modified the source code and want to package your own executable JAR, run: mvn clean package -DskipTests
Fields of Study & Subjects
Software Testing and Debugging Techniques
Software Engineering Research
Logic
programming
and type systems
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 30 June 2026
- Language
- EN