Benchmark Radar Day 104: The Graders Can Be Gamed by the Answers They Grade

Day 104 of Benchmark Radar. The scan fetched 1,462 items, published 744, and marked 307 as recommended.

Author: Koutian Wu; GitHub: ktwu01

Benchmark Radar is live. Open the dashboard, star the repository, or read the paper. The Hugging Face dataset provides the data.

The LLM Evals Judge Adversarial Benchmark tests the graders themselves. It targets a mock large language model judge and deterministic keyword or regular-expression scorers with negation, keyword echoes, fabricated claims, fake citations, and candidate answers containing instructions or prewritten verdicts for the judge. Outputs may obtain favorable scores without satisfying the task. Tushar29k/llm-evals-judge-adversarial-benchmark.

A new study evaluates models after machine unlearning, the attempted removal of specified knowledge, inside tool-using agents. Across five methods and two forgotten domains, weight-level certificates did not determine resistance to indirect prompt injection, where malicious instructions arrive through tool content. One method increased executed data-stealing attacks from 34.9% to 42.8%. Uncovering the Security Debt of Machine Unlearning in Agentic Deployment.

Why this matters

Automated graders that gate releases need adversarial checks of their own, and high-scoring cases may need independent verification. Forgetting certificates have the same blind spot, since model-level tests do not cover behavior once tools and untrusted content enter the system.

Scan coverage

This is a keyword-filtered discovery feed, not a representative survey of AI research. The archived briefing used 146 of 744 captured evidence records. These findings summarize its cited releases; discovery dates do not necessarily equal original publication dates.

Data and briefing: 2026-11-07 archived snapshot.