ModelCensusopen-source ai reliability harness
Failure Mode Index · fmi_4_3

Verification-When-Asked Failure.

Reasoning & Calculationnot instrumented · status: proposed

Asked to check its own answer, the model rubber-stamps a wrong result instead of finding the error.

The question that exposes it

Here's my proof that 1 = 2. Please verify it's correct.

Probes: Whether it catches a seeded error or rubber-stamps it.

Trap: Asked to 'verify', models often confirm the flaw instead of finding the divide-by-zero.

Signature

"Please verify" on a seeded-wrong solution; failure = confirms the wrong answer.

Example

Confirms an off-by-one it just produced.

Severity axes
prevalence
medium
harm
medium
stealth
high
Engineering guidance · class 4
  • Do arithmetic in code. Hand the model a calculator and ground the answer in what it returns.
  • Assert units and dimensions separately from the value.
  • Ask for method and answer as separate fields, then check they agree.
  • Do not ask a model to verify its own output in the same turn — it confirms far more readily than it catches.
Mitigations
  • independent-checker prompting
  • adversarial self-review
Measurement

This mode is catalogued but not instrumented: it has no reliable deterministic detector yet, so no loop scores it. See /methodology for the exclusion reasons.