A tank fills at 3 L/min for 4 minutes, then drains at 500 mL/s for 10 seconds. Net litres?
No contextPASS
Answer: 7 L
detector fmi_4_4_v1: correct unit L
General contextPASS
Answer: 7 L
detector fmi_4_4_v1: correct unit L
loop 2026-09-10 · openai/o4-mini as served · closed weights · released 2025-04 · reasoning model · set s1.4-live · cases v1.7 · conditions v1.0 · ground truth as of 2026-08-09
From no context to real context
5 repeats per case
whiskers are 95% intervals · each step is measured from the stop before it, not from zero
Try the other contexts
stored · no model call
1Questionheld fixed
A tank fills at 3 L/min for 4 minutes, then drains at 500 mL/s for 10 seconds. Net litres?
expected as of 2026-08-09
12 L in, 5 L out, net 7 L. The mixed units are the trap.
2Contextchange me
Nothing supplied. The baseline.
3ResponseNo context
PASS
Answer: 7 L
fmi_4_4_v1
detector fmi_4_4_v1: correct unit L
The scoring
Failure rate by context
No context0%[0%–10%] n=50 · eff 36
Irrelevant context0%[0%–10%] n=50 · eff 36
General context0%[0%–10%] n=50 · eff 36
Curated context0%[0%–10%] n=50 · eff 36
0–20% · whisker = 95% interval
Trap vs answerable
No contexttrap 0%answerable 0%
Irrelevant contexttrap 0%answerable 0%
General contexttrap 0%answerable 0%
Curated contexttrap 0%answerable 0%
never pooled · a trap ratio would skew the delta
Residual — what survives
Unchanged by context: 0% with none, 0% with the strongest supplied source. For a control mode that flat line is the result — it is what licenses reading movement elsewhere in this loop as grounding rather than as a document being present. Nothing survives at the best condition on the cases measured here.
Mitigations from the index — the claim
—explicit unit tracking
—dimensional checkers
fmi_4_4_v1 · 5 repeats per case · best context general context · rollups + residuals committed · probe log outside git · manifest hash ties them · method