Benchmark RadarIdioLink
IdioLink is a retrieval benchmark with 10,700 documents and 2,140 queries across 107 idioms, testing the linking of idiomatic expressions to literal or paraphrased equivalents. Annotations mark core meaning spans, and scoring is based on retrieval quality against these spans.
- Released
- 2026-05-21
- Readiness
- Paper only
- Primary field
- General AI
Why it matters
Existing retrieval models often rely on surface similarity and fail on idiomatic expressions. IdioLink provides a reusable testbed to measure semantic abstraction beyond lexical overlap, offering a practical way to evaluate idiom-aware retrieval systems.
Motivation
Idioms pose a fundamental challenge for language models, as their meaning cannot be inferred from surface form alone.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.