https://benchmark-radar.com/New and updated public AI benchmarks indexed by Benchmark Radar.enSun, 30 Aug 2026 00:00:00 +0000https://benchmark-radar.com/benchmarks/bm_the-illusion-of-textit-what-if-evaluating-_2b6da5c9/https://benchmark-radar.com/benchmarks/bm_the-illusion-of-textit-what-if-evaluating-_2b6da5c9/Fri, 28 Aug 2026 00:00:00 +0000A diagnostic benchmark of 220 open-domain what-if questions across STEM, HSS, and Hybrid scenarios, evaluated with PRISM metrics on causal graphs and explanatory adequacy.https://benchmark-radar.com/benchmarks/bm_vistarget-bench_aaa3c19c/https://benchmark-radar.com/benchmarks/bm_vistarget-bench_aaa3c19c/Fri, 28 Aug 2026 00:00:00 +0000A 150-task human-verified benchmark pairing questions with held-out target images to separate image-retrieval failures from visual-perception failures in multimodal search agents.https://benchmark-radar.com/benchmarks/bm_spar-bench_b75502fb/https://benchmark-radar.com/benchmarks/bm_spar-bench_b75502fb/Fri, 28 Aug 2026 00:00:00 +0000Evaluates spatial reasoning in medical vision encoders via eight probes over multi-organ abdominal CT covering coordinate localization, relational reasoning, and spatial queries.https://benchmark-radar.com/benchmarks/bm_pcbnet_ffc56858/https://benchmark-radar.com/benchmarks/bm_pcbnet_ffc56858/Fri, 28 Aug 2026 00:00:00 +0000Printed circuit boards (PCBs) are fundamental to modern electronic systems, yet AI-driven PCB design automation remains constrained by the lack of large-scale paired schematic-netlist datasets.https://benchmark-radar.com/benchmarks/bm_numbench_cf80a48e/https://benchmark-radar.com/benchmarks/bm_numbench_cf80a48e/Fri, 28 Aug 2026 00:00:00 +0000Text-to-image (T2I) models often generate the wrong number of objects, yet existing benchmarks are too small or weakly controlled to explain why.https://benchmark-radar.com/benchmarks/bm_beyond-global-scalars-synergizing-token-le_0d7c6f3e/https://benchmark-radar.com/benchmarks/bm_beyond-global-scalars-synergizing-token-le_0d7c6f3e/Fri, 28 Aug 2026 00:00:00 +0000Adversarial AIGC text detection benchmark with 16,000 samples spanning a full-granularity attack spectrum.https://benchmark-radar.com/benchmarks/bm_post-training-vlms-for-video-mistake-detec_58e7640e/https://benchmark-radar.com/benchmarks/bm_post-training-vlms-for-video-mistake-detec_58e7640e/Fri, 28 Aug 2026 00:00:00 +0000Tests video models on detecting whether a step was executed correctly according to its description, for seen and unseen actions.https://benchmark-radar.com/benchmarks/bm_looparena_97bc3500/https://benchmark-radar.com/benchmarks/bm_looparena_97bc3500/Fri, 28 Aug 2026 00:00:00 +0000Loop Engineering is emerging as a practice for organizing development work around coding agents.https://benchmark-radar.com/benchmarks/bm_geniac-secbench_393dfc82/https://benchmark-radar.com/benchmarks/bm_geniac-secbench_393dfc82/Fri, 28 Aug 2026 00:00:00 +0000Large language models are increasingly used to author Infrastructure-as-Code (IaC), where a single insecure default can be deployed directly into production.https://benchmark-radar.com/benchmarks/bm_finexam-10k-when-retrieval-helps-financial_fa6458fb/https://benchmark-radar.com/benchmarks/bm_finexam-10k-when-retrieval-helps-financial_fa6458fb/Fri, 28 Aug 2026 00:00:00 +0000Evaluates financial reasoning across CFA Levels I-III and FRM Parts I-II with 10,198 expert-reannotated questions, releasing 5,110 public items and maintaining a leaderboard on 5,088 held-out items.https://benchmark-radar.com/benchmarks/bm_evoharmbench-breaking-content-moderation-w_fad69c38/https://benchmark-radar.com/benchmarks/bm_evoharmbench-breaking-content-moderation-w_fad69c38/Fri, 28 Aug 2026 00:00:00 +0000Dynamic adversarial evaluation framework that evolves evasion strategies across 229 semantic sub-clusters from 5,002 real-world samples to assess content moderation systems.https://benchmark-radar.com/benchmarks/bm_elephantbench_79ce069e/https://benchmark-radar.com/benchmarks/bm_elephantbench_79ce069e/Fri, 28 Aug 2026 00:00:00 +0000Closed-book knowledge probe with 1,094 questions evaluating recall of multiple divergent accounts for long-tail facts, with fixed C/P/F/K scoring metrics.https://benchmark-radar.com/benchmarks/bm_cf-yolo-context-aware-feature-refinement-f_82f82c68/https://benchmark-radar.com/benchmarks/bm_cf-yolo-context-aware-feature-refinement-f_82f82c68/Fri, 28 Aug 2026 00:00:00 +0000Evaluates object detection of micro-defects on copper tube surfaces using 1,847 images and 4,898 bounding box instances across defect types in industrial inspection.https://benchmark-radar.com/benchmarks/bm_cocobench_0d6f9f05/https://benchmark-radar.com/benchmarks/bm_cocobench_0d6f9f05/Fri, 28 Aug 2026 00:00:00 +0000Agent systems powered by multimodal large language models (MLLMs) have advanced rapidly in recent years, yet existing embodied-agent benchmarks still lack fine-grained diagnostics for multi-agent coordination.https://benchmark-radar.com/benchmarks/bm_cneo-bench_bbdf3de1/https://benchmark-radar.com/benchmarks/bm_cneo-bench_bbdf3de1/Fri, 28 Aug 2026 00:00:00 +0000Chinese neologisms exploit diverse and unique linguistic mechanisms, such as phonetic substitution (e.g., 886 for ``bye-bye'') and visual character decomposition that are rare in other languages.https://benchmark-radar.com/benchmarks/bm_ruleweaver-benchmarking-rule-centered-scen_2379b2dc/https://benchmark-radar.com/benchmarks/bm_ruleweaver-benchmarking-rule-centered-scen_2379b2dc/Thu, 27 Aug 2026 00:00:00 +0000Evaluates rule-centered scenario reasoning through corpus-derived IF-THEN rules, composed into QA instances with rubric-based answer quality, rule recall, and rule precision scoring.https://benchmark-radar.com/benchmarks/bm_revico_34e02af8/https://benchmark-radar.com/benchmarks/bm_revico_34e02af8/Thu, 27 Aug 2026 00:00:00 +0000Vision Language Models (VLMs) have shown great success in general visual tasks, yet they still struggle to deeply understand text within images.https://benchmark-radar.com/benchmarks/bm_ratio-a-benchmark-for-retrieval-across-typ_4be3618a/https://benchmark-radar.com/benchmarks/bm_ratio-a-benchmark-for-retrieval-across-typ_4be3618a/Thu, 27 Aug 2026 00:00:00 +0000Evaluates retrieval models on three ideation moves—Address, Broaden, Specify—using relevance judgments derived from full-text scientific papers across computer science literature.https://benchmark-radar.com/benchmarks/bm_r2m-bench_1ea8e143/https://benchmark-radar.com/benchmarks/bm_r2m-bench_1ea8e143/Thu, 27 Aug 2026 00:00:00 +0000High similarity between first-visit and return frames does not necessarily show that a video world model remembered the scene; the intervening rollout may simply have changed very little.https://benchmark-radar.com/benchmarks/bm_plcbench_fb7005e5/https://benchmark-radar.com/benchmarks/bm_plcbench_fb7005e5/Thu, 27 Aug 2026 00:00:00 +0000A hardware-in-the-loop framework evaluating LLM agents across commercial PLCs and physical process simulations, with deterministic scoring for PLC interaction, process-linked manipulation, and sustained physical impact.https://benchmark-radar.com/benchmarks/bm_pawbench_0877eb45/https://benchmark-radar.com/benchmarks/bm_pawbench_0877eb45/Thu, 27 Aug 2026 00:00:00 +0000Evaluates video generators as stochastic samplers of world dynamics across 50 scenarios, comparing the distribution of possible behaviors under identical initial observations and actions.https://benchmark-radar.com/benchmarks/bm_multi2av-safety_13dc6c17/https://benchmark-radar.com/benchmarks/bm_multi2av-safety_13dc6c17/Thu, 27 Aug 2026 00:00:00 +0000Audio-video generation is rapidly moving from prompt-driven synthesis toward multimodal conditioning, where text, images, audio, and video can jointly shape the generated output.https://benchmark-radar.com/benchmarks/bm_mcr-bench_7a6da290/https://benchmark-radar.com/benchmarks/bm_mcr-bench_7a6da290/Thu, 27 Aug 2026 00:00:00 +0000In real-world software development, code review typically involves iterative interactions between developers and reviewers to improve software quality, making the process costly and time-consuming.https://benchmark-radar.com/benchmarks/bm_hug-vis_24e7fc8b/https://benchmark-radar.com/benchmarks/bm_hug-vis_24e7fc8b/Thu, 27 Aug 2026 00:00:00 +0000Visual intelligence seeks to perceive, interpret, and synthesize the visual world and is central to modern computer vision.https://benchmark-radar.com/benchmarks/bm_fault-bench_59abb402/https://benchmark-radar.com/benchmarks/bm_fault-bench_59abb402/Thu, 27 Aug 2026 00:00:00 +0000A benchmark of 200 troubleshooting scenarios across eight network topologies evaluating network troubleshooting LLM agents under unreliable user tickets, including false fault reports and incorrect device attribution.https://benchmark-radar.com/benchmarks/bm_esrp-bench_4847c977/https://benchmark-radar.com/benchmarks/bm_esrp-bench_4847c977/Thu, 27 Aug 2026 00:00:00 +0000This paper introduces Embodied Scene Rearrangement Planning (ESRP), a novel task requiring embodied agents to rearrange furniture in 3D scenes to match a target configuration using only egocentric observations and a top-down target layout.https://benchmark-radar.com/benchmarks/bm_dumatebench_7a085138/https://benchmark-radar.com/benchmarks/bm_dumatebench_7a085138/Thu, 27 Aug 2026 00:00:00 +0000Autonomous agents are increasingly adopted to complete complex, multi-tool workflows in real-world settings.https://benchmark-radar.com/benchmarks/bm_deepchart_94c0e719/https://benchmark-radar.com/benchmarks/bm_deepchart_94c0e719/Thu, 27 Aug 2026 00:00:00 +0000Faithful chart generation in real-world data-science workflows requires grounding visualizations in scattered evidence, computing chart-ready quantities, and rendering them accurately.https://benchmark-radar.com/benchmarks/bm_corporatebench_6a07b031/https://benchmark-radar.com/benchmarks/bm_corporatebench_6a07b031/Thu, 27 Aug 2026 00:00:00 +0000LLMs are increasingly able to answer complex questions about enterprise-scale document collections.https://benchmark-radar.com/benchmarks/bm_braillebench_3cc849b8/https://benchmark-radar.com/benchmarks/bm_braillebench_3cc849b8/Thu, 27 Aug 2026 00:00:00 +0000Although Large language models (LLMs) mediate access to knowledge and computational assistance, their capabilities should benefit vulnerable groups in the same way.https://benchmark-radar.com/benchmarks/bm_bekchiai-measuring-observing-and-controlli_e45a38cc/https://benchmark-radar.com/benchmarks/bm_bekchiai-measuring-observing-and-controlli_e45a38cc/Thu, 27 Aug 2026 00:00:00 +0000BekchiAI-Benchmark evaluates LLM agent skills using 2,057 deterministic tool-using ReAct tasks across 7 categories, scoring accuracy plus tool-call adherence, URL hallucination, source-match, and token cost.https://benchmark-radar.com/benchmarks/bm_behavior2trip_b70cb42e/https://benchmark-radar.com/benchmarks/bm_behavior2trip_b70cb42e/Thu, 27 Aug 2026 00:00:00 +0000Travel planning agents assist users in generating personalized travel plans by modeling their individual preferences.https://benchmark-radar.com/benchmarks/bm_bts-agentbench_70852571/https://benchmark-radar.com/benchmarks/bm_bts-agentbench_70852571/Thu, 27 Aug 2026 00:00:00 +0000Evaluates multi-turn agent performance on 532 telemetry-derived tasks across train/dev/test splits, with additional XAI4HEAT episodes, using deterministic replayable construction and verifiable gold answers.https://benchmark-radar.com/benchmarks/bm_balms_57758cdd/https://benchmark-radar.com/benchmarks/bm_balms_57758cdd/Thu, 27 Aug 2026 00:00:00 +0000Mental health assessment relies on episodic self-report scales, which convert subjective states such as stress into numerical scores but provide only sparse snapshots of wellbeing.https://benchmark-radar.com/benchmarks/bm_ancient-bench_967fba8e/https://benchmark-radar.com/benchmarks/bm_ancient-bench_967fba8e/Thu, 27 Aug 2026 00:00:00 +0000Ancient Chinese artifact text recognition is fundamental to heritage digitization, and benchmarks for ancient texts are essential for evaluating current model capabilities.https://benchmark-radar.com/benchmarks/bm_agentjudgebench_8ae8da3e/https://benchmark-radar.com/benchmarks/bm_agentjudgebench_8ae8da3e/Thu, 27 Aug 2026 00:00:00 +0000LLM judges are widely used to evaluate agentic tool-calling systems, yet their reliability on structured, dependency-driven workflows remains largely unexamined.https://benchmark-radar.com/benchmarks/bm_4dsynth-controllable-procedural-world-synt_fa98bfb4/https://benchmark-radar.com/benchmarks/bm_4dsynth-controllable-procedural-world-synt_fa98bfb4/Thu, 27 Aug 2026 00:00:00 +0000Evaluates embodied agents on interactive navigation tasks in procedurally generated 4D environments with independently tunable difficulty axes.https://benchmark-radar.com/benchmarks/bm_xrepotest_04d9b86c/https://benchmark-radar.com/benchmarks/bm_xrepotest_04d9b86c/Wed, 26 Aug 2026 00:00:00 +0000Evaluates multilingual repository-level unit test generation across Rust, Go, Julia, PHP, and Ruby using containerized execution and context augmentation strategies.https://benchmark-radar.com/benchmarks/bm_video-ifbench_4fc7a0ab/https://benchmark-radar.com/benchmarks/bm_video-ifbench_4fc7a0ab/Wed, 26 Aug 2026 00:00:00 +0000Evaluates instruction following of multimodal LLMs in video understanding with 1.5K samples across constraint categories.https://benchmark-radar.com/benchmarks/bm_vga-benchv2_d6dfef90/https://benchmark-radar.com/benchmarks/bm_vga-benchv2_d6dfef90/Wed, 26 Aug 2026 00:00:00 +0000Evaluates video generation quality and aesthetic value with 1,016 prompts, 60,000 videos, and 36,000 task-level annotations.https://benchmark-radar.com/benchmarks/bm_vbvr-pro_41ab73db/https://benchmark-radar.com/benchmarks/bm_vbvr-pro_41ab73db/Wed, 26 Aug 2026 00:00:00 +0000Provides 300 procedurally generated tasks for native visual reasoning through generation, with verifiable reward scorers and controlled modality comparisons.https://benchmark-radar.com/benchmarks/bm_scimif_1f2ceaa4/https://benchmark-radar.com/benchmarks/bm_scimif_1f2ceaa4/Wed, 26 Aug 2026 00:00:00 +0000Evaluates instruction following of MLLMs across five scientific disciplines with a taxonomy of 10 constraint groups.https://benchmark-radar.com/benchmarks/bm_reconstructing-the-right-episode-evaluatin_98f96c99/https://benchmark-radar.com/benchmarks/bm_reconstructing-the-right-episode-evaluatin_98f96c99/Wed, 26 Aug 2026 00:00:00 +0000SCALE-QA evaluates interleaved conversational memory using 3,000 audited multiple-choice questions across 10 domains, where correct answers depend on causally related evidence from earlier turns in flat unsegmented threads.https://benchmark-radar.com/benchmarks/bm_pivot-a-multi-trajectory-dataset-and-testb_3fccb501/https://benchmark-radar.com/benchmarks/bm_pivot-a-multi-trajectory-dataset-and-testb_3fccb501/Wed, 26 Aug 2026 00:00:00 +0000Evaluates novel-view synthesis under diverse camera trajectories, measured vs optimized poses, and calibrated vs optimized intrinsics using five real-world scenes.https://benchmark-radar.com/benchmarks/bm_omniphys_ae437965/https://benchmark-radar.com/benchmarks/bm_omniphys_ae437965/Wed, 26 Aug 2026 00:00:00 +0000Evaluates multimodal physics understanding, reasoning, and generation on 15,246 questions with 19,850 images from middle-school to university levels.https://benchmark-radar.com/benchmarks/bm_mathadv_85c33564/https://benchmark-radar.com/benchmarks/bm_mathadv_85c33564/Wed, 26 Aug 2026 00:00:00 +0000Evaluates theorem proving and auxiliary tasks including multiple-choice, fill-in-the-blank, and reformulation robustness across 13 mathematical domains.https://benchmark-radar.com/benchmarks/bm_mmjailbench_1b3e3a1a/https://benchmark-radar.com/benchmarks/bm_mmjailbench_1b3e3a1a/Wed, 26 Aug 2026 00:00:00 +0000Evaluates multimodal jailbreak vulnerabilities by factorizing harmful intent, prompt framing, visual semantics, and instruction carrier.https://benchmark-radar.com/benchmarks/bm_finriskatlas_4424cb99/https://benchmark-radar.com/benchmarks/bm_finriskatlas_4424cb99/Wed, 26 Aug 2026 00:00:00 +0000Evaluates Chinese financial LLMs on static operation execution across 9,742 instances and evidence-state control via FinRisk-Ask using pre-action trajectory states.https://benchmark-radar.com/benchmarks/bm_fedcmapss_9057f5e4/https://benchmark-radar.com/benchmarks/bm_fedcmapss_9057f5e4/Wed, 26 Aug 2026 00:00:00 +0000Data-driven prognostics and health management has emerged as a key enabler for Industry 4.0, yet the development of robust remaining useful life (RUL) estimation models is often limited by the scarcity of run-to-failure data.https://benchmark-radar.com/benchmarks/bm_paint-what-you-see-benchmarking-dexterous-_f2ccb455/https://benchmark-radar.com/benchmarks/bm_paint-what-you-see-benchmarking-dexterous-_f2ccb455/Wed, 26 Aug 2026 00:00:00 +0000Evaluates dexterous visual tool use through reference-guided painting, semantic annotation, handwriting, and path planning tasks.