Benchmark RadarAI BENCHMARK PROFILE
SMH-Bench
Benchmark for LLM agents in smart-home environments, with 1,100 tasks across 7 categories and 22 subcategories, built on the executable HomeEnv simulator, stratified by home complexity.
- Released
- 2026-06-01
- Readiness
- Paper only
- Primary field
- General AI
Why it matters
Addresses deficiencies in existing smart-home benchmarks by evaluating state-dependent reasoning across devices, but lacks a public comparison path.
Motivation
Smart homes are evolving toward complex state-dependent living environments, requiring Large Language Models (LLMs) to reason over user intent, preferences, and multi-device interactions.
Primary resources
Benchmark Radar records only publicly supported details and links back to primary sources for verification.