Benchmark RadarDevicesWorld
DevicesWorld is an executable benchmark for cross-device agent evaluation, comprising 6,140 tasks across Android, Linux, and SmartHome environments, with rule-based verifiers for automatic scoring.
- Released
- 2026-07-15
- Readiness
- Runnable
- Primary field
- General AI
Why it matters
Existing benchmarks focus on single-device environments, leaving a gap in evaluating agents that must coordinate across heterogeneous devices. DevicesWorld aims to address this by providing tasks with cross-device dependencies and automated verification, which could inform development of more capable multi-device agents.
Motivation
LLM-based agents have rapidly improved at operating individual digital environments such as mobile applications, desktop systems, and smart homes.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.