Benchmark RadarHoloCount
HoloCount evaluates multimodal large language models on visual counting tasks across three levels: semantic counting (atomic and property-based enumeration), analytical counting (logical composition via spatial and set-based reasoning), and robustness testing (adverse scenarios and grounded counter-priors). The benchmark uses a hierarchical taxonomy and provides a dataset for evaluation.
- Released
- 2026-07-07
- Readiness
- Inspectable
- Primary field
- General AI
Why it matters
Existing counting benchmarks fail to capture complex failure modes under logical constraints or adversarial conditions. HoloCount provides a structured diagnostic tool to assess MLLM quantitative precision, revealing performance gaps as tasks shift from perception to analytical reasoning, and guiding development of more grounded multimodal systems.
Motivation
Visual counting is a fundamental pillar of multimodal intelligence, requiring a seamless integration of fine-grained grounding and spatial reasoning.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.