Benchmark RadarMirrorCraft
MirrorCraft is a paired benchmark for evaluating LLM-based agents under hidden rule changes in Minecraft. Each Mirror world copies a Vanilla world with modified server-side rules, while terrain and objectives match. It includes five biomes, six rule suites, three objectives, and uses deterministic milestones and Rule Intervention Effect.
- Released
- 2026-07-31
- Readiness
- Paper only
- Primary field
- General AI
Why it matters
Most Minecraft benchmarks use fixed mechanics, not testing adaptability to rule changes. MirrorCraft provides a controlled setting to study agent performance under hidden rule modifications, important for developing agents that can handle dynamic environments.
Motivation
With the prosperity of the large language models (LLMs), it has become an interesting topic: how do LLM-based agents work in Minecraft?
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.