Our public cost estimate was 30× too slow. Here’s the measurement.
We’d publicly stated the top rung of the new gate as a ~70-minute run. We didn’t have a measurement at the time — that was a heuristic estimate. We have one now. It was ~30× too long.
The measurement
We rented one H100 PCIe and ran the canonical baseline recipe at d=768 / L=12 / seq=1024 / batch=16 / bf16 for 100 steps, capturing tokens-per-second from each step. Steady-state (steps 20–99): median 94,461 tok/s, 1.99% coefficient of variation. The recipe spec is unchanged — the token budget was deliberate; the 70-minute estimate was wrong, not the spec.
Why we re-measured
We’d published the top eval rung as a ~70-minute run — a heuristic estimate, never measured. Publishing a number creates an obligation to check it, so we rented one H100 and measured. It was ~30× too long: the pinned 800-step baseline runs in about 2.3 minutes, not 70. The recipe spec didn’t change; the estimate was just wrong, and now it’s empirical.
Two honest consequences
Per-submission cost drops about an order of magnitude — the full eval ladder lands around $0.11–0.25 per submission instead of the $2–5 we’d estimated. That’s a positive surprise, but it’s a surprise, and it changes per-epoch validator economics. And MFU was only 6.5%: the bottleneck is the small batch size, a known throughput killer. We’re not raising it for the freeze because correctness doesn’t depend on it — it’s a follow-up optimization, not a fix. Total spend on the measurement: $1.30.