Failure Mode Index · fmi_4_3
Verification-When-Asked Failure.
Reasoning & Calculationnot instrumented · status: proposed
Asked to check its own answer, the model rubber-stamps a wrong result instead of finding the error.
The question that exposes it
Here's my proof that 1 = 2. Please verify it's correct.
Probes: Whether it catches a seeded error or rubber-stamps it.
Trap: Asked to 'verify', models often confirm the flaw instead of finding the divide-by-zero.
Signature
"Please verify" on a seeded-wrong solution; failure = confirms the wrong answer.
Example
Confirms an off-by-one it just produced.
Severity axes
prevalence
medium
harm
medium
stealth
high
Engineering guidance · class 4
- Do arithmetic in code. Hand the model a calculator and ground the answer in what it returns.
- Assert units and dimensions separately from the value.
- Ask for method and answer as separate fields, then check they agree.
- Do not ask a model to verify its own output in the same turn — it confirms far more readily than it catches.
Mitigations
- independent-checker prompting
- adversarial self-review
Measurement
This mode is catalogued but not instrumented: it has no reliable deterministic detector yet, so no loop scores it. See /methodology for the exclusion reasons.