Benchmark RadarAtrex-Bench
Evaluates coding agents on generating GPU kernels from PyTorch references across 30 operators and 440 shapes derived from production inference traces. Scoring uses a three-stage evaluator measuring compile success, numerical correctness, and speed-of-light (SOL) efficiency against a cached roofline. The benchmark includes 4 DSL backends and supports multi-vendor GPUs.
- Released
- 2026-07-16
- Readiness
- Runnable
- Primary field
- General AI
Why it matters
Prior GPU kernel benchmarks draw from synthetic or curated sources that diverge from deployed workloads. This benchmark provides a production-trace-driven evaluation that emphasizes operators consuming the most serving time, enabling assessment of agent performance on relevant tasks.
Motivation
Existing GPU kernel generation benchmarks draw problems from synthetic or curated sources that diverge from deployed workloads.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.