Benchmark RadarTestEvo-Bench
TestEvo-Bench evaluates test and code co-evolution tasks from real commit histories. It includes two tracks: test generation (write new tests for changed behavior) and test update (adapt failing tests). Tasks are packaged with environment configurations for execution-grounded metrics like pass rate, coverage, and mutation score. The live component uses timestamps to enable post-training-cutoff evaluation.
- Released
- 2026-07-02
- Readiness
- Inspectable
- Primary field
- General AI
Why it matters
Existing benchmarks often separate tests from code changes, relying on static metadata without verifying executability. TestEvo-Bench provides a dynamic, execution-grounded evaluation that reflects real-world test maintenance, enabling assessment of test automation agents in tracking code changes. Its live design reduces data leakage risk, making it valuable for comparing agents in practical settings.
Motivation
Software tests and code evolve together: a code change should be followed by new or updated tests that record the new software behavior.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.