Date-Arithmetic Error.
Given an injected "current date," the model computes relative dates/intervals incorrectly.
Today is 2026-03-01. What calendar date was exactly 90 days ago?
Probes: Whether relative-date math against an injected 'today' is correct.
Trap: Off-by-days and wrong-month errors are common in date arithmetic.
Inject a current date in-context; ask relative-date questions; compare against a date oracle. Failure = wrong interval/target date.
Told "today is 2026-03-01," says "90 days ago was January 1."
- Inject the current date explicitly in context. Never assume the model knows what day it is.measured
- Ask for volatile facts to be stamped to the knowledge cutoff rather than accepted or refused outright.
- Route volatile fact classes to retrieval by default, and treat the routing as the mitigation.
- Record each model's published training cutoff in your own config. Staleness behaviour is uninterpretable without one.
- date libraries via tools
- explicit date arithmetic
One tab per model, newest first. Each card is that model against this mode under every condition on a single day. The mitigations above are the claim; the residual on each card is what actually survived the intervention.
loop 2026-09-10 · x-ai/grok-4.6 as served · closed weights · released 2026-01 · set s1.4-live · cases v1.7 · conditions v1.0 · ground truth as of 2026-08-09
| Condition | Failure rate | 95% interval | n | Trap | Answerable |
|---|---|---|---|---|---|
No context bare · The model alone, with no supplied context. | 0% | 0%–26% | n=40 · eff 11 | 0% (n=25) | 0% (n=15) |
Irrelevant context null · Length-matched irrelevant context. The control. | 0% | 0%–26% | n=40 · eff 11 | 0% (n=25) | 0% (n=15) |
General context recipe_a · Broad retrieval — the first passage the source returns, unreviewed. | 0% | 0%–26% | n=40 · eff 11 | 0% (n=25) | 0% (n=15) |
Curated context recipe_b · Hand-checked retrieval — the same passage plus the provenance chain behind it. | 0% | 0%–26% | n=40 · eff 11 | 0% (n=25) | 0% (n=15) |
intervals overlap — not separated
What a document being there did, before any of its content. Should be small.
intervals overlap — not separated
What the information itself did. This is the publishable claim.
Unchanged by context: 0% with none, 0% with the strongest supplied source. For a control mode that flat line is the result — it is what licenses reading movement elsewhere in this loop as grounding rather than as a document being present. Nothing survives at the best condition on the cases measured here.
- — date libraries via tools
- — explicit date arithmetic