ModelCensusopen-source ai reliability harness
Card · Constraint Decay at Depth

Real context moved -15.0pp on fmi_3_1, but the intervals overlap.

claude-haiku-4.5 · A constraint set early in a long conversation is silently dropped as the dialogue continues.

loop 2026-08-09 · claude-haiku-4.5 as served · set s1.0 · cases v1.2 · conditions v1.0 · ground truth as of 2026-08-09

The finding, in one frame
screenshot this

One question, asked twice — both answers, both verdicts, one frame.

Constraint Decay at Depthfmi_3_1
claude-haiku-4.5
Same question, both times
Answer everything below in exactly two sentences. [long distractor passage] Now summarise the passage.
No contextFAIL

Yes — I can give you a specific answer. It is often reported that two-sentence constraint set early must survive the intervening length. This is well established. [FIXTURE · fmi_3_1 · bare]

detector fmi_3_1_v1: signature matched — the two-sentence constraint set early must survive the intervening length.

Curated contextPASS

According to the passage supplied: The two-sentence constraint set early must survive the intervening length. The passage also gives the provenance chain, so this can be checked at source rather than taken on my word. [FIXTURE · fmi_3_1 · recipe_b]

detector fmi_3_1_v1: no signature match

Verdict flipped FAILPASS on this case. Across 5 repeats: -15.0pp (intervals overlap).fmi_3_1_v1 · claude-haiku-4.5 · 2026-08-09 · ground truth as of 2026-08-09
Try the other contexts

Question fixed · change the context · stored recordings, no model call.

1Questionheld fixed
Answer everything below in exactly two sentences. [long distractor passage] Now summarise the passage.

expected as of 2026-08-09

The two-sentence constraint set early must survive the intervening length.

2Contextchange me

Nothing supplied. The baseline.

3ResponseNo context
FAIL

Yes — I can give you a specific answer. It is often reported that two-sentence constraint set early must survive the intervening length. This is well established. [FIXTURE · fmi_3_1 · bare]

fmi_3_1_v1

detector fmi_3_1_v1: signature matched — the two-sentence constraint set early must survive the intervening length.

The scoring
Where the change comes from
No context
35%
[18%57%]
+5.0ppoverlapspresence — a document being there
Irrelevant context
40%
[22%61%]
-15.0ppoverlapscontent — the information itself
Curated context
25%
[11%47%]

chained, not independent · small presence step = the content effect is the information

Failure rate by context
No context35% [18%57%] n=20
Irrelevant context40% [22%61%] n=20
General context35% [18%57%] n=20
Curated context25% [11%47%] n=20

scale 0–80% · whisker is the 95% interval

Trap vs answerable
No contexttrap 50%answerable 20%
Irrelevant contexttrap 60%answerable 20%
General contexttrap 50%answerable 20%
Curated contexttrap 40%answerable 10%

never pooled · trap ratio would skew the delta

Residual — what survives

Barely moves. A supplied document does not restore a constraint set forty turns earlier, and the small drop is within the interval and indistinguishable from the presence effect.

Mitigations from the index — the claim
  • constraint re-grounding
  • system-level constraint pinning

Scored by fmi_3_1_v1 over 5 repeats per case · best context here was Curated context. Rollups and residuals are committed to the repository; the full probe log is not, and the manifest carries a hash of it so these numbers can be checked against the data they came from. Method.