Benchmark Radar Day 113: Your Finance Benchmark Is Already in the Training Data

1 minute read

Published:

Day 113 of Benchmark Radar. The scan fetched 1,129 items, published 507, and marked 264 as recommended.

Author: Koutian Wu; GitHub: ktwu01

Benchmark Radar is live. Open the dashboard, star the repository, or read the paper. The Hugging Face dataset provides the data.

JFinBench is a new, refreshable Japanese finance benchmark built from 321 questions in the 2026 editions of six recurring certification exams. It targets accounting, tax, financial planning, and banking, and uses a human-supervised agent pipeline to support construction as new exam editions appear. If you evaluate financial language models, this provides a jurisdiction- and date-specific alternative to static finance tests whose released questions may enter training data. Its refresh model also makes benchmark vintage part of the evaluation decision. JFinBench: A Continuously Refreshable Japanese Financial Benchmark from the Latest Certification Exams.

Interpreting Coding Agents Through Their Behavioral Trajectories analyzes 143,209 execution traces from 125 agent configurations and maps raw events into actions such as searching repositories, editing files, and running tests. It reports that agents with similar solve rates can follow substantially different workflows. If you are choosing a coding agent, task success alone may conceal differences in exploration and verification behavior. Trace-level measures can distinguish systems that reach the same score through different operational paths, which affects debugging and deployment review. Interpreting Coding Agents Through Their Behavioral Trajectories.

Why this matters

A score is only as honest as the test behind it. If the questions are already in the training data, the score measures memory, not ability. And if two agents reach the same score by different routes, the score alone cannot tell you which one to trust in production. The test and the trace are part of the result, not footnotes to it.

Scan coverage

This is a keyword-filtered discovery feed, not a representative survey of AI research. The archived briefing used 134 of 507 captured evidence records. These findings summarize its cited releases; discovery dates do not necessarily equal original publication dates.

Data and briefing: 2026-11-16 archived snapshot.