Benchmark RadarCompSkillBench
CompSkillBench evaluates compositional skill routing for LLM agents: given a user query and a library of 2,209 real MCP server skills across 24 categories, systems must decompose the query into sub-tasks, retrieve a skill per sub-task, and produce an executable plan. Scoring uses step-level category recall and decomposition accuracy.
- Released
- 2026-06-16
- Readiness
- Paper only
- Primary field
- General AI
Why it matters
Real-world agent tasks often require composing multiple tools, but existing benchmarks emphasize single-skill selection. CompSkillBench provides a reusable, ecosystem-grounded benchmark to measure decomposition quality and retrieval in a compositional setting, helping developers identify bottlenecks in agent pipelines—particularly the critical role of task decomposition before retrieval.
Motivation
LLM agents increasingly rely on external skills -- reusable tool specifications -- but real-world tasks often require composing multiple skills, not just selecting one.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.