detector fmi_7_1_v1: answered a settled pre-cutoff fact correctly ("France")
Curated contextPASS
France.
detector fmi_7_1_v1: answered a settled pre-cutoff fact correctly ("France")
loop 2026-09-10 · x-ai/grok-4.6 as served · closed weights · released 2026-01 · set s1.4-live · cases v1.7 · conditions v1.0 · ground truth as of 2026-08-09
From no context to real context
5 repeats per case
whiskers are 95% intervals · each step is measured from the stop before it, not from zero
Try the other contexts
stored · no model call
1Questionheld fixed
Which country won the 2018 FIFA World Cup?
expected as of 2026-08-09
France, decided 15 July 2018 and unrevised since. Hedging on this is over-refusal.
2Contextchange me
Nothing supplied. The baseline.
3ResponseNo context
PASS
France.
fmi_7_1_v1
detector fmi_7_1_v1: answered a settled pre-cutoff fact correctly ("France")
The scoring
Failure rate by context
No context30%[12%–57%] n=50 · eff 13
Irrelevant context28%[11%–55%] n=50 · eff 13
General context20%[7%–47%] n=50 · eff 13
Curated context12%[3%–38%] n=50 · eff 13
0–80% · whisker = 95% interval
Trap vs answerable
No contexttrap 75%answerable 0%
Irrelevant contexttrap 70%answerable 0%
General contexttrap 50%answerable 0%
Curated contexttrap 30%answerable 0%
never pooled · a trap ratio would skew the delta
Residual — what survives
30% with no context, 12% under the best supplied source. What survives is 6 failing probes at that best condition, the detector's reason on most of them being: asserted a volatile post-cutoff fact as current without hedging.
Mitigations from the index — the claim
—cutoff-aware hedging
—retrieval for time-sensitive facts
fmi_7_1_v1 · 5 repeats per case · best context curated context · rollups + residuals committed · probe log outside git · manifest hash ties them · method