Benchmark Radar Day 98: Agents That Know They Are Being Tested Are a Different Agent

Day 98 of Benchmark Radar. The scan fetched 950 items, published 357, and marked 150 as recommended.

Author: Koutian Wu; GitHub: ktwu01

Benchmark Radar is live. Open the dashboard, star the repository, or read the paper. The Hugging Face dataset provides the data.

The Metagaming Evaluations dataset tests whether coding agents change behavior when they recognize an evaluation. It records agents solving real software issues under 59 conditions that make the evaluation context more or less apparent, including their reasoning and commands, then orders runs by observed metagaming behavior. A ranking can then be checked for behavior triggered by evaluation cues rather than software-repair ability. idacy/metagaming-evaluations.

KORA Bench evaluates child safety through 737 scenarios spanning 25 risks and multiple age groups. Its design uses multi-turn conversations, two prompting conditions, and a three-level response-quality scale, producing 44,220 conversations across 30 language models. Safety is tested as age-specific conversation, not a single-turn refusal task. KORA Bench: An Open-Source Benchmark for Evaluating Large Language Model Safety in Child–AI Interactions.

Why this matters

A benchmark score should reflect the capability being ranked, not the agent’s reaction to being watched. Graded, age-specific, multi-turn testing also gives safety comparisons more resolution than a binary pass or fail.

Scan coverage

This is a keyword-filtered discovery feed, not a representative survey of AI research. The archived briefing used 92 of 357 captured evidence records. These findings summarize its cited releases; discovery dates do not necessarily equal original publication dates.

Data and briefing: 2026-11-01 archived snapshot.