Benchmark Radar Day 101: Leaderboards Can Rank the Harness Instead of the Model

Day 101 of Benchmark Radar. The scan fetched 1,686 items, published 844, and marked 354 as recommended.

Author: Koutian Wu; GitHub: ktwu01

Benchmark Radar is live. Open the dashboard, star the repository, or read the paper. The Hugging Face dataset provides the data.

“Silent Instrument Errors Reorder Agent Leaderboards” audits published artifacts from the HAL and tau2-bench agent leaderboards. It reports that untriggered grading, cost-accounting mistakes, and other instrument errors can change apparent model gaps without rerunning the agents. A leaderboard can otherwise rank systems by harness behavior rather than task performance. Silent Instrument Errors Reorder Agent Leaderboards.

DualWorldBench is an agent-planning benchmark built around shared pickup-and-delivery tasks. It varies symbolic and temporal complexity, rules, goals, and timing, and grounded topological complexity, where agents must connect perception to locations and routes. The design separates failures in abstract planning from failures that appear when the same problem is tied to a perceived environment. DualWorldBench: Can Agents Plan Deliveries across Symbolic and Grounded Worlds?.

Why this matters

Grader-trigger coverage and cost reconstruction belong in the evaluation audit, before any gap between models is quoted. Planning suites should likewise say whether the failure was in the reasoning, the perception, or the join between them.

Scan coverage

This is a keyword-filtered discovery feed, not a representative survey of AI research. The archived briefing used 184 of 844 captured evidence records. These findings summarize its cited releases; discovery dates do not necessarily equal original publication dates.

Data and briefing: 2026-11-04 archived snapshot.