What a share card is allowed to say
The artefact that travels furthest from its caveats needs the harshest gate.
A page can show a rate next to its confidence interval, its tag split, and a link to the transcript behind it. A reader can judge. A share image is 1200 by 630 pixels, it travels alone, and nobody who reshares it reads the methodology.
Reasons a true number is still inadmissible
- Pseudo-replication — the difference is real at nominal n and gone at effective n
- Presence confound — the irrelevant control reproduces most of the claimed movement
- No control tag — a rate of zero could be perfect calibration or refusing everything
- Uninterpretable model — a staleness claim about a model that publishes no training cutoff
- Denominator drift — the arms did not lose the same number of probes, and dropout is never random
Of several hundred candidate comparisons in our corpus, a small fraction survive this gate. The rest are true and unquotable, which is an uncomfortable but honest place to be.
Every figure here describes something measured and committed. See the measurements · read the method
Related
Stop reporting a hallucination rate. Report what a wrong answer costs you.
5% is fine for a first draft and unacceptable for a dosage. The rate alone is not a risk statement.
The boring rows are load-bearing
Seven of the ten modes in our set are controls, and their flat lines are the finding.
We accepted 2,900 failures without reading them — and said so in the data
A gate everyone routes around isn't a gate. So we named the shortcut instead of pretending.