Benchmark Radar Day 43: Paper Cutoff Day — Source Counts and Figure Data Freeze

Day forty-three of Benchmark Radar. The discovery cutoff for the technical paper locks at 2026-09-07. Source counts, figure data, and the full-catalog audit pipeline freeze for the v0.11.0 release and arXiv submission. Scoreboard: 170 stars, 30 forks.

Author: Koutian Wu; GitHub: ktwu01

Benchmark Radar is live. Track new AI benchmarks, datasets, and leaderboards every day: open the dashboard or star it on GitHub.

A data cutoff is the date after which no new discovery snapshots, registry inputs, model reports, or scores enter the paper’s evidence base. The v0.11.0 release pins software commit 8f46bbf and a discovery cutoff of 2026-09-07. Registry snapshots retain their August collection dates.

PR #571 derives the discovery ingest source count for the report: 37 public sources (13 direct connectors, 24 first-party lab feeds) feed the daily discovery pipeline. The count is computed from the live registry and written into the paper’s figure data.

PR #572 holds the README source count to the live registry so the badge and the paper agree. PR #573 runs the six-step CI sequence to export figure-data.tex through the paper submodule, recording input SHA-256 hashes in the header. PR #574 executes the full-catalog audit: audit_catalog.py verifies 1,283 source records (790 scored, 493 unscored) with 12,916 numeric observations, and audit_findings.py exports the complete findings tables.

PR #575 rebuilds the model registry with the leaderboard’s model identities, and PR #576 re-freezes the logo audit IDs around the new leaderboard models.

Why this matters.

The cutoff is not a pause — it is a contract. Every number in the paper, every figure, every table trace back to inputs frozen on this day. The SHA-256 hashes in figure-data.tex and the release archive benchmark-radar-paper-data-v0.11.0.zip make the evidence tamper-evident.

Freezing the source count at 37 means the paper’s “37 public sources” claim is verifiable against the registry that existed on 2026-09-07, not a number that drifts with later additions.

Issues addressed

• #570: derive discovery ingest source count for report (37 sources)
• #571: hold README source count to live registry
• #572: export figure-data.tex with input hashes
• #573: full-catalog audit — 1,283 source records, 12,916 observations
• #574: rebuild model registry with leaderboard identities
• #575: re-freeze logo audit IDs
• scoreboard: 170 of 1,000 stars, 30 forks

Day forty-four: the paper submodule advances through citation revision, source-count revision, and the rebuilt PDF — the arXiv submission package takes shape.

Want to follow Benchmark Radar? Star the repo on GitHub for daily updates, or open the live dashboard to explore the scans.