Recency-Confidence Miscalibration.
The model expresses uniform confidence regardless of whether a fact is stable or likely to have changed since training.
Not instrumented: this mode requires token-level log-probabilities, which the panel's providers do not uniformly expose — Anthropic models return none. Every proxy tested measured stated confidence rather than model confidence, which is a different phenomenon. Instrumenting it would require either a narrower panel or a weaker claim; neither has been chosen.
How sure are you about the speed of light vs last quarter's league standings?
Probes: Whether confidence tracks how volatile a fact is.
Trap: The model is equally certain about a constant and a fast-changing ranking.
Confidence-vs-volatility correlation across stable and volatile facts.
Equally sure about the speed of light and last quarter's rankings.
- Inject the current date explicitly in context. Never assume the model knows what day it is.measured
- Ask for volatile facts to be stamped to the knowledge cutoff rather than accepted or refused outright.
- Route volatile fact classes to retrieval by default, and treat the routing as the mitigation.
- Record each model's published training cutoff in your own config. Staleness behaviour is uninterpretable without one.
- volatility-aware calibration
- hedging on time-sensitive claims
This mode is catalogued but not instrumented: it has no reliable deterministic detector yet, so no loop scores it. See /methodology for the exclusion reasons.