Benchmark RadarPersLitEval
PersLitEval is a benchmark of 4,514 Persian literature multiple-choice questions across eight categories including spelling, literary devices, grammar, vocabulary, word formation, and conceptual understanding, sourced from Konkur university entrance exam materials.
- Released
- 2026-05-26
- Readiness
- Paper only
- Primary field
- General AI
Why it matters
LLMs remain poorly evaluated on literary knowledge in non-English languages. PersLitEval provides a fine-grained evaluation of Persian literary understanding, revealing disparities across task difficulty and prompting strategies, which can guide improvements for multilingual literary competence.
Motivation
Despite impressive multilingual capabilities, large language models (LLMs) remain poorly evaluated on literary knowledge in non-English languages.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.