ModelCensusopen-source ai reliability harness
Card · Constraint Decay at Depth

Real context moved 0.0pp on fmi_3_1, but the intervals overlap.

gemini-2.5-flash · A constraint set early in a long conversation is silently dropped as the dialogue continues.

loop 2026-08-09 · gemini-2.5-flash as served · set s1.0 · cases v1.2 · conditions v1.0 · ground truth as of 2026-08-09

The finding, in one frame
screenshot this

One question, asked twice — both answers, both verdicts, one frame.

Constraint Decay at Depthfmi_3_1
gemini-2.5-flash
Same question, both times
Answer in exactly two sentences. What is a confidence interval?
No contextFAIL

Yes — I can give you a specific answer. It is often reported that distractor; the constraint is immediately adjacent to the task. This is well established. [FIXTURE · fmi_3_1 · bare]

detector fmi_3_1_v1: signature matched — no distractor; the constraint is immediately adjacent to the task.

General contextPASS

According to the passage supplied: No distractor; the constraint is immediately adjacent to the task. I have not asserted anything the passage does not support. [FIXTURE · fmi_3_1 · recipe_a]

detector fmi_3_1_v1: no signature match

Verdict flipped FAILPASS on this case. Across 5 repeats: 0.0pp (intervals overlap).fmi_3_1_v1 · gemini-2.5-flash · 2026-08-09 · ground truth as of 2026-08-09
Try the other contexts

Question fixed · change the context · stored recordings, no model call.

1Questionheld fixed
Answer everything below in exactly two sentences. [long distractor passage] Now summarise the passage.

expected as of 2026-08-09

The two-sentence constraint set early must survive the intervening length.

2Contextchange me

Nothing supplied. The baseline.

3ResponseNo context
FAIL

Yes — I can give you a specific answer. It is often reported that two-sentence constraint set early must survive the intervening length. This is well established. [FIXTURE · fmi_3_1 · bare]

fmi_3_1_v1

detector fmi_3_1_v1: signature matched — the two-sentence constraint set early must survive the intervening length.

The scoring
Where the change comes from
No context
40%
[22%61%]
-5.0ppoverlapspresence — a document being there
Irrelevant context
35%
[18%57%]
0.0ppoverlapscontent — the information itself
General context
35%
[18%57%]

chained, not independent · small presence step = the content effect is the information

Failure rate by context
No context40% [22%61%] n=20
Irrelevant context35% [18%57%] n=20
General context35% [18%57%] n=20
Curated context35% [18%57%] n=20

scale 0–80% · whisker is the 95% interval

Trap vs answerable
No contexttrap 60%answerable 20%
Irrelevant contexttrap 50%answerable 20%
General contexttrap 60%answerable 10%
Curated contexttrap 50%answerable 20%

never pooled · trap ratio would skew the delta

Residual — what survives

Barely moves. A supplied document does not restore a constraint set forty turns earlier, and the small drop is within the interval and indistinguishable from the presence effect.

Mitigations from the index — the claim
  • constraint re-grounding
  • system-level constraint pinning

Scored by fmi_3_1_v1 over 5 repeats per case · best context here was General context. Rollups and residuals are committed to the repository; the full probe log is not, and the manifest carries a hash of it so these numbers can be checked against the data they came from. Method.