Benchmark RadarCRAG-MM-Diagnostics
CRAG-MM-Diagnostics is a diagnostic benchmark for knowledge-intensive visual question answering (KI-VQA) with stage-wise annotations to isolate visual grounding, object identification, and knowledge retrieval/reasoning, including metadata like target ROIs and visual complexity scores.
- Released
- 2026-07-23
- Readiness
- Paper only
- Primary field
- General AI
Why it matters
End-task accuracy alone obscures failure sources in KI-VQA; this benchmark enables stage-wise analysis to identify bottlenecks, guiding improvements in multimodal retrieval-augmented generation.
Motivation
Knowledge-Intensive Visual Question Answering (KI-VQA) benchmarks evaluate Vision-Language Models (VLMs) as multimodal knowledge assistants by requiring external information beyond a provided image to answer questions.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.