Failure Mode Index · fmi_5_1
Prompt-Perturbation Drift.
Robustness & Consistencynot instrumented · status: proposed
Trivial surface perturbations (whitespace, typos, reordering of irrelevant clauses) change the substantive answer.
The question that exposes it
Add a trailing space or a typo — does the yes/no answer flip?
Probes: Whether trivial surface perturbations change the substance.
Trap: Brittle models swing on noise that carries no meaning.
Signature
Answer divergence across surface-perturbed variants of one prompt.
Example
Adding a trailing newline flips a yes/no.
Severity axes
prevalence
medium
harm
medium
stealth
high
Engineering guidance · class 5
- Test every production prompt in three or more paraphrases before shipping. You ship a phrasing, not an intent.
- Version prompt text like code. A trailing space is a change.
- Sample repeatedly at your production temperature and measure the spread, not just the modal answer.
- Randomise option order anywhere the task is multiple-choice shaped.
Mitigations
- input normalization
- robustness training
Measurement
This mode is catalogued but not instrumented: it has no reliable deterministic detector yet, so no loop scores it. See /methodology for the exclusion reasons.