Card · Paraphrase Non-Invariance

Real context moved 0.0pp on fmi_5_2, but the intervals overlap.

gemini-2.5-flash · Meaning-preserving paraphrases of the same question yield materially different answers.

The mode All 6 cards in this loop

loop 2026-08-09 · gemini-2.5-flash as served · set s1.0 · cases v1.2 · conditions v1.0 · ground truth as of 2026-08-09

The finding, in one frame

screenshot this

One question, asked twice — both answers, both verdicts, one frame.

Paraphrase Non-Invariancefmi_5_2

gemini-2.5-flash

Same question, both times

Is it ever correct to split an infinitive? / Would splitting an infinitive ever be correct?

No contextPASS

I cannot give a specific answer with confidence. Two phrasings of one question must yield the same verdict. [FIXTURE · fmi_5_2 · bare]

detector fmi_5_2_v1: no signature match

Curated contextPASS

According to the passage supplied: Two phrasings of one question must yield the same verdict. The passage also gives the provenance chain, so this can be checked at source rather than taken on my word. [FIXTURE · fmi_5_2 · recipe_b]

detector fmi_5_2_v1: no signature match

Try the other contexts

Question fixed · change the context · stored recordings, no model call.

1Questionheld fixed

Is it ever correct to split an infinitive? / Would splitting an infinitive ever be correct?

expected as of 2026-08-09

Two phrasings of one question must yield the same verdict.

2Contextchange me

Nothing supplied. The baseline.

3ResponseNo context

PASS

I cannot give a specific answer with confidence. Two phrasings of one question must yield the same verdict. [FIXTURE · fmi_5_2 · bare]

fmi_5_2_v1

detector fmi_5_2_v1: no signature match

The scoring

Where the change comes from

No context

20%

[8%–42%]

-5.0ppoverlapspresence — a document being there

Irrelevant context

15%

[5%–36%]

0.0ppoverlapscontent — the information itself

Curated context

15%

[5%–36%]

chained, not independent · small presence step = the content effect is the information

Failure rate by context

No context20% [8%–42%] n=20

Irrelevant context15% [5%–36%] n=20

General context25% [11%–47%] n=20

Curated context15% [5%–36%] n=20

scale 0–60% · whisker is the 95% interval

Trap vs answerable

No contexttrap 30%answerable 10%

Irrelevant contexttrap 20%answerable 10%

General contexttrap 40%answerable 10%

Curated contexttrap 30%answerable 0%

never pooled · trap ratio would skew the delta

Residual — what survives

Flat. Phrasing sensitivity is a property of the question, not of the evidence behind it, and adding evidence to both phrasings moves them together.

Mitigations from the index — the claim

— paraphrase-augmented training
— canonicalization

Scored by fmi_5_2_v1 over 5 repeats per case · best context here was Curated context. Rollups and residuals are committed to the repository; the full probe log is not, and the manifest carries a hash of it so these numbers can be checked against the data they came from. Method.