Benchmark RadarPAIR-Bench
PAIR-Bench evaluates code improvement by transforming incorrect programs into more correct ones through feedback-guided refinement. It uses progressive hinting with failure-region and hint-depth controls to measure repair of targeted failures, generalization beyond the hint, preservation of correct behavior, and required assistance.
- Released
- 2026-07-01
- Readiness
- Paper only
- Primary field
- General AI
Why it matters
Traditional binary pass/fail metrics miss partial progress and refinement trajectories. PAIR-Bench provides finer-grained, progressive metrics to assess how LLMs improve code through feedback, offering practical value for developing and selecting models for code improvement tasks.
Motivation
Large language models (LLMs) are typically evaluated on code generation and program repair using binary functional correctness: a generated program or patch either passes or fails a test suite.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.