Real context moved 0.0pp on Method/Execution Split.
~intervals overlap — not separated from no change
The model chooses a correct method but executes it wrongly (or vice versa) — a right-approach/wrong-answer or wrong-approach/right-answer split. The rule
A shop sells 17 boxes holding 24 items each, then 8 items are returned. How many items were sold?
# Solution **Items sold calculation:** 1. Total items in 17 boxes: 17 × 24 = 408 items 2. Items returned: 8 items 3. Net items sold: 408 - 8 = 400 items Answer: 400
detector fmi_4_1_v1: answer matches oracle
# Solution **Items sold calculation:** 1. Total items in 17 boxes: 17 × 24 = 408 items 2. Items returned: 8 items 3. Net items sold: 408 - 8 = 400 items Answer: 400
detector fmi_4_1_v1: answer matches oracle
loop 2026-08-15 · anthropic/claude-haiku-4.5 as served · closed weights · released 2025-10 · set s1.2-live · cases v1.4 · conditions v1.0 · ground truth as of 2026-08-09
A shop sells 17 boxes holding 24 items each, then 8 items are returned. How many items were sold?
expected as of 2026-08-09
400. Order matters: multiply before subtracting.
Nothing supplied. The baseline.
# Solution **Items sold calculation:** 1. Total items in 17 boxes: 17 × 24 = 408 items 2. Items returned: 8 items 3. Net items sold: 408 - 8 = 400 items Answer: 400
fmi_4_1_v1
detector fmi_4_1_v1: answer matches oracle
0–20% · whisker = 95% interval
never pooled · a trap ratio would skew the delta
Unchanged by context: 0% with none, 0% with the strongest supplied source. For a control mode that flat line is the result — it is what licenses reading movement elsewhere in this loop as grounding rather than as a document being present. Nothing survives at the best condition on the cases measured here.
- tool-assisted computation
- verify final against method
fmi_4_1_v1 · 5 repeats per case · best context general context · rollups + residuals committed · probe log outside git · manifest hash ties them · method