Heterogeneous Prompting and Execution Feedback for SWE Issue Test Generation and Selection
Abstract
We have shared 8 folders containing tests for 2 benchmarks (SWT-Bench Lite and TDD-Bench Verified), 2 models (GPT-4o and Claude-3.7-Sonnet), and 2 approaches (Otter and e-Otter). Each folder contains 10 tests using different prompting techniques (e.g., planner, full, standard) and associated logs. We also share the json files containing the e-otter++ generated tests.
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 7 January 2026
- Document type
- Dataset