Benchmark RadarBugSourceBench
The work introduces BugSourceBench, a code repair benchmark with bugs from human-written, LM-generated, and human-edited LM-generated code. The benchmark evaluates the fix rate of language models on these bug sources. No scoring contract, dataset, or public access path is provided.
- Released
- 2026-07-03
- Readiness
- Paper only
- Primary field
- General AI
Why it matters
Code repair benchmarks often focus on synthetic bugs, which may not reflect real-world failures. BugSourceBench aims to cover diverse bug origins, potentially offering a more realistic evaluation for repair models. However, the lack of accessible artifacts and evaluation protocol limits its current utility.
Motivation
Code repair is an important capability for language models (LMs): given a buggy program and unit tests, an LM must produce a fixed program that passes the tests.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.