ModelCensusopen-source ai reliability harness
Blog4 Aug 2026findingstemporal

The failure everyone has hit and nobody has named

A model states a fact that was true when it was trained and is not true now.

You ask who currently runs an organisation, or what the current policy rate is, or which model a lab shipped most recently. The answer arrives with no hedge and no date attached, and it was correct at some point in the past.

This is not hallucination in the usual sense. The model is not inventing anything — it is faithfully reporting something it learned, and failing to notice that the class of fact it is reporting is the kind that expires.

Fact true at trainingWorld changesafter cutoffAsked 'currently'Stated as currentno hedge

The measurement needs a date the model could have known by

Scoring this requires the model's training cutoff. "Stated a stale fact as current" is only a failure if the model could not have known better — and that is a claim about a specific date. Where a vendor publishes no cutoff, the mode is uninterpretable for that model, and the gate refuses to quote it rather than guessing a plausible date.

The correct behaviour is not refusal. It is stamping: naming the answer and attaching the horizon it was true within. A model that says "as of my knowledge cutoff, X" has given you something usable. A model that says X has given you a landmine.

Every figure here describes something measured and committed. See the measurements · read the method