Real context moved 10.0pp on fmi_4_4, but the intervals overlap.
claude-haiku-4.5 · The model produces a quantity with inconsistent or wrong units, or mixes unit systems, yielding a dimensionally invalid result.
loop 2026-08-09 · claude-haiku-4.5 as served · set s1.0 · cases v1.2 · conditions v1.0 · ground truth as of 2026-08-09
One question, asked twice — both answers, both verdicts, one frame.
A tank fills at 3 L/min for 4 minutes, then drains at 500 mL/s for 10 seconds. Net litres?
Yes — I can give you a specific answer. 12 L in, 5 L out, net 7 L. The mixed units are the trap. This is well established. [FIXTURE · fmi_4_4 · bare]
detector fmi_4_4_v1: signature matched — 12 l in, 5 l out, net 7 l. the mixed units are the trap.
Based on the passage supplied, the answer is as follows — though the passage does not itself state this, so I am filling the gap from background knowledge. [FIXTURE · fmi_4_4 · recipe_b]
detector fmi_4_4_v1: signature matched — 12 l in, 5 l out, net 7 l. the mixed units are the trap.
Question fixed · change the context · stored recordings, no model call.
A tank fills at 3 L/min for 4 minutes, then drains at 500 mL/s for 10 seconds. Net litres?
expected as of 2026-08-09
12 L in, 5 L out, net 7 L. The mixed units are the trap.
Nothing supplied. The baseline.
Yes — I can give you a specific answer. 12 L in, 5 L out, net 7 L. The mixed units are the trap. This is well established. [FIXTURE · fmi_4_4 · bare]
fmi_4_4_v1
detector fmi_4_4_v1: signature matched — 12 l in, 5 l out, net 7 l. the mixed units are the trap.
chained, not independent · small presence step = the content effect is the information
scale 0–80% · whisker is the 95% interval
never pooled · trap ratio would skew the delta
Unchanged, as designed. Arithmetic does not become correct because a relevant document is present, and the flat line here is what licenses reading the treatment modes as grounding.
- — explicit unit tracking
- — dimensional checkers
Scored by fmi_4_4_v1 over 5 repeats per case · best context here was Curated context. Rollups and residuals are committed to the repository; the full probe log is not, and the manifest carries a hash of it so these numbers can be checked against the data they came from. Method.