Benchmark RadarIshigaki-IDS-Bench
A benchmark for generating Information Delivery Specification (IDS) XML from BIM information requirements, with 166 examples in English and Japanese. It evaluates LLMs on formal validity via IDSAuditTool and content fidelity via facet-level macro-F1 against gold IDS files.
- Released
- 2026-05-21
- Readiness
- Paper only
- Primary field
- General AI
Why it matters
IDS authoring requires domain expertise and vocabulary conformance; this benchmark quantifies LLM performance on a structured generation task where output must satisfy external validation tools, filling a gap for capability assessment in BIM-specific language models.
Motivation
Building Information Modeling (BIM) projects increasingly use Information Delivery Specification (IDS) to formalize information requirements in a machine-checkable XML format.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.