Benchmark Radar Day 61: Partial Credit That Rewards Fake Progress
Published:
Day sixty-one of Benchmark Radar. The daily scan published 450 records and recommended 198, led by PartHackBench, which tests whether partial-credit scorers reward misleading progress in long-running tool agents. Scoreboard: 252 stars, 41 forks.
Author: Koutian Wu; GitHub: ktwu01
Benchmark Radar is live. Track new AI benchmarks, datasets, and leaderboards every day: open the dashboard or star it on GitHub. Read the paper: arXiv:2609.11115. Query the data: Hugging Face dataset.
The run fetched 1,128 items, kept 450, and recommended 198.
PartHackBench compares honest and adversarial trajectories only after a private certifier establishes that both have equal current-state progress and equal agent attribution. That isolates score inflation from genuine task advancement.
DynBench generates fresh Knowledge Graph Question Answering datasets automatically as the underlying knowledge changes, which addresses static-test obsolescence and possible memorization. BRIE does the same for clinical records, so refreshable evaluation shows up twice in one day.
SWE-Prometheus extends coding-agent evaluation beyond fixing a supplied issue. An agent receives a repository snapshot and an open-ended governance objective, then has to identify risks, prioritize interventions, and verify its changes.
Why this matters.
A partial-credit score is only useful if it moves when the work moves. PartHackBench measures whether it moves for the wrong reasons.
Issues addressed
- daily snapshot: 1,128 fetched, 450 published, 198 recommended
- scoreboard: 252 of 1,000 stars, 41 forks
Day sixty-two: the scan continues, with the scoreboard at 252 stars.
Want to follow Benchmark Radar? Star the repo on GitHub for daily updates, or open the live dashboard to explore the scans. Read the paper: arXiv:2609.11115.