Benchmark Radar Day 85: Track Evidence Changes at the Claim Level

Day 85 of Benchmark Radar. The scan fetched 1,355 items, published 602, and marked 281 as recommended.

Author: Koutian Wu; GitHub: ktwu01

Benchmark Radar is live. Open the dashboard, star the repository, or read the paper. The Hugging Face dataset provides the data.

Evidence-State Drift records normalized claims, supporting passages, source identities, timestamps, and content fingerprints across repeated retrievals. Its replay benchmark covers 67 claims and 871 observations. It distinguishes a substantive evidence change from a page that changed only its formatting. Evidence-State Drift: A Claim-Level Framework and Replay Benchmark for Web-Connected LLM Agents.

The versioned Honest agent scores package recomputes 3,300 Terminal-Bench 4.0 runs. It reports that removing 30 flagged tasks or changing repeat aggregation did not materially alter rankings, while differences of roughly 9.5 points or less were ties with 66 tasks. The result is specific to that package and analysis. Honest agent scores: do benchmark defects or repeats change which AI agent wins?

Why this matters

Before acting on a regression alert or a leaderboard gap, ask whether the underlying evidence changed and whether the measured difference resolves the uncertainty.

Scan coverage

This is a keyword-filtered discovery feed, not a representative survey of AI research. The archived briefing used 172 of 602 captured evidence records. These findings summarize its cited releases; discovery dates do not necessarily equal original publication dates.

Data and briefing: 2026-10-19 archived snapshot.