Benchmark RadarBackendForge
BackendForge is a benchmark of 56 contract-defined backend generation tasks from real open-source applications. LLMs must generate Dockerized services evaluated through HTTP tests against an OpenAPI contract.
- Released
- 2026-07-13
- Readiness
- Paper only
- Primary field
- General AI
Why it matters
Agentic LLMs need to produce deployable and behaviorally correct software artifacts. BackendForge provides a deterministic, black-box evaluation of backend service generation, exposing gaps between local API implementation and complete service delivery.
Motivation
Large language models (LLMs) are increasingly used in agentic coding settings, where they can inspect files, execute commands, run tests, observe failures, and iteratively revise code.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.