Pact Benchmark: ICPC World Finals — Contract-First Multi-Agent vs Single-Agent Code Generation
Abstract
Benchmark comparing Pact (contract-first multi-agent framework) against Claude Code on 5 ICPC World Finals competitive programming problems (212 test cases). Pact achieves 100% (212/212) vs Claude Code single-shot 79% (167/212) and iterative 92% (196/212). All conditions use Claude Opus 4.6. Includes test data, baseline results, full Pact state for both conditions (research and base), and reproduction scripts. The decisive problem is Trailing Digits (2020 World Finals): Claude Code scores 31/47 even with 5 retry iterations — the naive algorithm times out. Pact's interview and decomposition phases force upfront mathematical analysis, producing the correct O(log n) approach on the first attempt.
Fields of Study & Subjects
Teaching and Learning Programming
Software Engineering Research
Multi-Agent Systems and Negotiation
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 4 March 2026
- Language
- EN
- Document type
- Dataset