Delete the publish button from your eval dashboard
A button behind a password is a surface an attacker can reach and an operator can misclick.
By the end of this your published results will be diffable, revertible, attributable, and impossible for your pipeline to produce on its own.
What you need
- A git repository you already have
- A CLI you probably already have
- The willingness to delete an admin route
Stage 1 — move results out of the database
Write results as committed files. Have the site read those files at build rather than querying at request time.
Keeping the database as the source of truth and the files as an export.
Why: Then a row edited in the database changes what your site claims with no commit, no review, and no way to tell afterwards. We shipped that architecture and had to unwind it.
`git log` on your results directory should read as a publication history.
Stage 2 — make the runner unable to finish the job
Have runs write a status the runner itself cannot clear. Ours writes `review` and stops.
Try to publish without doing the human steps. It should be impossible, not merely discouraged.
Why: A pipeline that can mark its own output reviewed is a pipeline with no review gate wearing one as decoration.
Stage 3 — name the gates and make them cheap to satisfy honestly
Ours are: every failing probe reviewed by a person, and a residual written per card. The second is the one a machine genuinely cannot do — it is a claim about what still gets through, not a count.
Give the honest shortcut a name. Our bulk-accept path stamps `reviewed_by: bulk-accept` into every probe it touches, so the record says plainly that nobody read them individually.
Query a published run for which failures were individually inspected versus bulk-accepted. If you cannot, the shortcut is invisible and people will take it silently.
The accelerator
You do not need a new system. A directory of JSON, a validate command that exits non-zero, and a publish command that commits. That is the whole architecture.
Can your team publish an eval result without leaving a trace of who decided?
Every figure here describes something measured and committed. See the measurements · read the method