Benchmark RadarSymbalBench
SymbalBench evaluates automated detection of systematic misalignments in MLLM-generated image captions. It comprises 420 vision-language datasets (1.7 million image-text pairs) from natural and medical domains, each annotated with known systematic misalignments.
- Released
- 2026-07-16
- Readiness
- Runnable
- Primary field
- Health & Life Sciences
Why it matters
MLLM-generated captions often contain recurring errors tied to visual features, which can degrade downstream tasks. SymbalBench provides a standardized testbed to assess methods for surfacing such systematic captioning failures, aiding dataset auditing and model improvement.
Motivation
Multimodal large language models (MLLMs) often introduce errors when generating image captions, resulting in misaligned image-text pairs.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.