Evaluating Tools for Automatic Software Testing (Report on Test-Comp 2026)

Abstract This report presents the results of the 8th Competition on Software Testing (Test-Comp 2026), which is an annual event to provide an overview and comparative evaluation of automatic tools for test-suite generation for C programs. The experiment setup is publicly available and suitable for reuse as a baseline when comparing newly developed approaches for test generation. The benchmark set SV-Benchmarks contains 16217 test-generation tasks for C programs. Each test-generation task consists of a program and a test specification. The test specifications included error coverage (generate a test suite that exhibits a bug) and branch coverage (generate a test suite that executes as many program branches as possible). Test-Comp 2026 evaluated 21 software systems for test generation that are all freely available. This included 11 test-suite generators that participated with active support from teams led by 11 different representatives from 6 countries (actively maintained software systems, participation in competition jury). Test-Comp 2026 had 1 new test generator ( "Image missing" "Image missing" ) and 1 new test-suite validator ( "Image missing" "Image missing" ). The evaluation included also 10 test-generation tools from previous years.