We accepted 2,900 failures without reading them — and said so in the data
A gate everyone routes around isn't a gate. So we named the shortcut instead of pretending.
Publishing a panel run means clearing a gate: every failing probe reviewed by a person. On a sixteen-model run that is 2,902 probes.
Nobody reads 2,902 transcripts. We did not.
Expectation versus reality
The gate was designed assuming review would be per-probe and careful. That holds for a five-model pilot. It does not survive contact with a twenty-model panel.
What happens to an unrealistic gate is not that people stop publishing. It is that they satisfy it nominally and the record cannot tell.
Diagnosis
Careful per-probe review and blanket acceptance are both legitimate decisions in different situations. The system only modelled the first.
Why: When a system models one behaviour and people need two, the second one happens anyway and goes unrecorded.
An early triage tool was one commit away from auto-approving 'obvious' cases.
Why: That would have been the bypass the invariant exists to prevent, wearing a helpful name. It now re-checks each failure against the same code path the detector used and approves nothing.
What we do now
Two paths. The careful one, and a bulk path that writes its own name into every probe it touches. Anyone auditing a published loop can query which failures were individually inspected.
The last census: 2,902 probes, bulk-accepted, stamped. It is in the data and it is in this post.
The honest version of a shortcut is a shortcut that leaves a trace.
Which of your quality gates does everyone satisfy nominally — and would you know?
Every figure here describes something measured and committed. See the measurements · read the method