Benchmark RadarERUnderstand
ERUnderstand evaluates vision-language models on structured understanding of entity-relationship diagrams. The benchmark contains 2,960 diagrams across curated educational sources, real-world schemas, and synthetic data, with standardized machine-readable JSON annotations. Scoring uses F1, BLEU, and graph edit distance to measure how accurately models recover schema elements such as entities, relationships, attributes, and extended ER constructs.
- Released
- 2026-07-27
- Readiness
- Runnable
- Primary field
- General AI
Why it matters
ER diagrams are central to database design but their image-based nature impedes automated processing. Existing VLM benchmarks do not focus on structured schema extraction from ERDs, and no public benchmark provides a standardized protocol for this task. ERUnderstand fills that gap with a reusable dataset and evaluation toolkit, enabling reproducible comparison of VLM performance on conceptual database schemas.
Motivation
Entity-Relationship Diagrams (ERDs) are central to conceptual database design, yet they are typically available only as rendered images rather than machine-readable schemas, limiting AI-assisted database engineering.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.