Benchmark RadarEvoCode-Bench
EvoCode-Bench evaluates coding agents in multi-turn iterative interactions, with 26 stateful coding tasks and 227 evaluated rounds. Each task preserves the agent's workspace for 5-15 rounds, specifies requirements via observable behavior, and uses cumulative executable tests to verify new and prior requirements. Scoring uses MT@4, a four-attempt fail-stop multi-round score, and SR, a single-round score from a reference-completed prior state.
- Released
- 2026-05-22
- Readiness
- Paper only
- Primary field
- General AI
Why it matters
Existing coding benchmarks typically evaluate a single specification and final output, missing the ability to handle evolving requirements. EvoCode-Bench fills this gap by assessing whether agents can maintain a working codebase over multiple rounds. The gap between SR and MT@4 scores reveals that high single-turn performance does not guarantee sustained multi-turn success, providing a more relevant evaluation for real-world iterative development.
Motivation
Coding agents are increasingly used as iterative development partners, but most benchmarks still evaluate one specification followed by one final assessment.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.