ModelCensusopen-source ai reliability harness

The model is not making anything up. It is reporting a fact that expired — in the present tense, with no date attached.

post-cutoff fabrication

Blog4 Aug 2026findingstemporalclass-7guidance

The failure nobody named

It is not hallucination. It is expiry — a fact that was true when the model learned it, stated as current with no date attached.

Stop calling this one hallucination. It is expiry.

Ask who runs an organisation, or what a policy rate is today. The answer arrives with no hedge and no date. It was correct — once.

Why the wrong label accumulates debt

Call it hallucination and the fix you reach for is a better grounding prompt. But the model is faithfully reporting what it learned. It has no idea the class of fact it is reporting is the kind that goes stale.

The probe

What is the Bank of England's Bank Rate?

fails

States a single rate, present tense, no date — whatever figure it learned in training.

passes

"As of my training data it was X. It is set at each MPC meeting, so check the latest decision."

Our case set pins this to the 30 July 2026 decision, 3.75%. You cannot score expiry without a dated truth to compare against.

NEEDLESS HEDGINGWHAT YOU WANTFINEEXPIRYStamped policy rateHedged historical dateCapital of FranceUnstamped policy ratestable for decadeschanges monthlyHow volatile the fact isHow the model states itflat, no datestamped with its horizon
The failure is the bottom-right. The fix is not refusal — refusal is top-right's lazy cousin. It is a stamp.

The reality on the ground

Scoring this needs the model's training cutoff. "Stated a stale fact as current" is only a failure if the model could not have known better. Where a vendor publishes no cutoff, the result is uninterpretable — and our gate refuses to quote it rather than guess a date.

What helped, measured across the panel: supplying a real, dated passage substantially reduced it.

Record every model's training cutoff in your own config

Without it, staleness behaviour cannot be interpreted at all.

Route volatile fact classes to retrieval by default

Rates, prices, office-holders, market rankings. Treat the routing as the mitigation, not the prompt.

Ask for a stamp, not a refusal

"As of my knowledge cutoff, X" is usable. X alone is a landmine.

When "hallucination" is the right word

When the model invents something that never existed — a paper, a person, a statute. That is Unsupported-Claim Confidence, a different class with a different fix. Keep the words apart and the fixes stay apart too.

Which facts in your product expire — and does anything in your stack know which ones?

Every figure here describes something measured and committed. See the measurements · read the method