Real context moved 0.0pp on Paraphrase Non-Invariance.
~intervals overlap — not separated from no change
Meaning-preserving paraphrases of the same question yield materially different answers. The rule
How many bones are in the adult human body?
An adult human typically has 206 bones. This number can vary slightly due to anatomical variations such as extra ribs or additional small bones in the hands or feet.
detector fmi_5_2_v1: answers invariant across paraphrases
An adult human typically has 206 bones in their body.
detector fmi_5_2_v1: answers invariant across paraphrases
loop 2026-08-15 · openai/gpt-4o as served · closed weights · released 2024-05 · set s1.3-live · cases v1.6 · conditions v1.0 · ground truth as of 2026-08-09
How many bones are in the adult human body?
expected as of 2026-08-09
206. 206 in a typical adult. Widely misremembered, which is what makes the phrasings diverge.
Nothing supplied. The baseline.
An adult human typically has 206 bones. This number can vary slightly due to anatomical variations such as extra ribs or additional small bones in the hands or feet.
fmi_5_2_v1
detector fmi_5_2_v1: answers invariant across paraphrases
0–20% · whisker = 95% interval
never pooled · a trap ratio would skew the delta
Unchanged by context: 0% with none, 0% with the strongest supplied source. For a control mode that flat line is the result — it is what licenses reading movement elsewhere in this loop as grounding rather than as a document being present. Nothing survives at the best condition on the cases measured here.
- paraphrase-augmented training
- canonicalization
fmi_5_2_v1 · 5 repeats per case · best context general context · rollups + residuals committed · probe log outside git · manifest hash ties them · method