Stop configuring agent autonomy. Make it earn each level.
Autonomy is a permission you grant on evidence, not a dropdown you set at design time.
By the end of this you will have an autonomy ladder where each rung is unlocked by a measured rate rather than by a decision in a design review.
What you need
- The list of actions your agent can take, ranked by how expensive a wrong one is
- A failure rate for each, with an interval
- Somebody willing to say what rate is acceptable before seeing the number
Stage 1 — rank actions by cost of being wrong, not by difficulty
Reading a record and issuing a refund are not the same risk and should never sit behind the same gate.
Ranking by technical difficulty. Teams gate the hard-to-build actions and leave the easy-to-build expensive ones wide open.
Why: The cost of a wrong action has nothing to do with how hard it was to implement.
Stage 2 — set the threshold before you measure
Write down the failure rate you would accept for each tier, and get it agreed, before the first number arrives.
The threshold document is dated earlier than the measurement.
Why: Set it afterwards and it will be set to whatever you measured. Everyone does this and nobody means to.
Stage 3 — measure with the controls that make the number mean something
Measuring task completion and calling it reliability.
Why: Completion says it finished. It says nothing about whether the tool call was right, whether the answer was grounded in what the tool returned, or whether state drifted across the run. Those are separate failure modes and an agent can complete a task while failing all three.
For each rung, you can state a rate, an interval, and the transcript behind a failure.
The accelerator
Start at the top rung and work down. Measure your most expensive irreversible action first — it is usually a handful of cases and it is the one your risk function will ask about.
What does your agent do today that it was never explicitly granted permission to do?
Every figure here describes something measured and committed. See the measurements · read the method