Benchmark RadarVULBENCH-CPP
VULBENCH-CPP evaluates the safety of AI-generated C++ code using multi-tier verification including functional testing, static analysis, dynamic analysis, and bounded model checking. It includes 8,918 programs from three LLMs and human authors.
- Released
- 2026-06-30
- Readiness
- Paper only
- Primary field
- Cybersecurity
Why it matters
Security of AI-generated code is critical, but evaluations often rely on a single method. VULBENCH-CPP provides a comprehensive multi-tier benchmark that reveals AI code is more prone to runtime violations and that no single verification tier is sufficient.
Motivation
As large language models (LLMs) are increasingly deployed for systems programming, their ability to generate secure C++ code, where a single memory-safety failure creates an exploitable vulnerability, remains a critical concern.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.