Benchmark Radar Day 94: The Right Answer Can Be No Plan at All

Day 94 of Benchmark Radar. The scan fetched 1,283 items, published 573, and marked 238 as recommended.

Author: Koutian Wu; GitHub: ktwu01

Benchmark Radar is live. Open the dashboard, star the repository, or read the paper. The Hugging Face dataset provides the data.

OpenGray evaluates whether radiotherapy-planning agents make the correct decision, not merely whether they produce a favorable dose plan. Its known-answer tasks include incompatible treatment goals, where an agent should recognize that no valid plan exists, and a feasible control with a stored plan satisfying every hard goal. The test rewards stopping or requesting review over optimizing an impossible specification. OpenGray: a benchmark for evaluating the decisions of radiotherapy planning agents.

EffectWorld represents each agent tool by what it reads, writes, and changes, including whether repeated calls are safe. An executable verifier can then check tool-use traces for unsafe effects without relying on human labels or another model as judge. Actions such as duplicate writes or deletion without confirmation can be caught even when the final answer looks correct. Small agents should know what their tools do.

Why this matters

Optimizing an impossible specification is a failure mode, not a success. Evaluation should check the effects of each tool call, so a correct-looking answer cannot hide an unsafe trace.

Scan coverage

This is a keyword-filtered discovery feed, not a representative survey of AI research. The archived briefing used 152 of 573 captured evidence records. These findings summarize its cited releases; discovery dates do not necessarily equal original publication dates.

Data and briefing: 2026-10-28 archived snapshot.