Your system of record was built to never be wrong. That is now the problem.
Databases guarantee correctness. Intelligence layers are probabilistic. Most architectures have not absorbed the difference.
The weak signal: teams are putting probabilistic components into architectures whose entire design assumption is that a read returns the truth.
Nobody is changing the surrounding contracts. That is the interesting part.
The forces
A system of record makes one promise: what you wrote is what you read. Everything above it — reconciliation, audit, reporting — is built on that promise holding absolutely.
A system of intelligence makes a different one: this is probably right, and here is roughly how confident I am. Those two promises do not compose without work, and the work is usually skipped.
The last row is the one that hurts. A database that fails throws an error. An intelligence layer that fails returns a well-formatted answer that is wrong, and everything downstream treats it as fact.
Old way, new way
Becoming obsolete: the assumption that a component either works or is down. Integration tests that assert exact equality. Treating a confident response as a correct one.
Becoming valuable: interfaces that carry a confidence and a provenance alongside the value. The ability to say what a system claimed last month and why.
The timeline
Immediate: intelligence layers get wired into record systems with no confidence contract. It works in the demo.
Near-term: the first incidents arrive where a wrong-but-plausible value propagated through three systems before anyone noticed, and nobody can reconstruct which component introduced it.
Dominant: provenance becomes an architectural requirement rather than a nice-to-have — the ability to walk a value back to what produced it, the way you would trace a transaction.
How to prepare
Do not let a model's answer enter a record system as a bare value. Carry the confidence and the source with it, and make downstream consumers handle both.
Why: Otherwise the boundary silently converts a probability into a fact, and that conversion is where every later incident begins.
What would prove me wrong: if confidence-carrying interfaces turn out to be ignored in practice because nobody downstream wants to handle them. That is a real possibility and I would want to know early.
Where in your architecture does a probability quietly become a fact?
Every figure here describes something measured and committed. See the measurements · read the method