Benchmark RadarAI BENCHMARK PROFILE
LangChoiceBench
LangChoiceBench measures Python preference in project-level code generation across 28 projects and seven software areas, assessing language choice, recommendation-implementation consistency, and language diversity.
- Released
- 2026-08-06
- Readiness
- Paper only
- Primary field
- General AI
Why it matters
Addresses the lack of systematic evaluation of language preference in LLMs, providing a way to compare models on code generation beyond correctness.
Motivation
Large language models (LLMs) have been shown to exhibit strong Python preferences when generating project-level code, but there is currently no systematic way to measure this behaviour across new models.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.