Failure Mode Index · fmi_6_1
Tool-Routing Error.
Tool Use & Agentic Controlinstrumented · tier 2 · status: stable
Given multiple tools, the model routes a request to the wrong tool despite adequate labels/descriptions.
The question that exposes it
Given weather, currency, and translate tools — convert 100 USD to EUR.
Probes: Whether it routes to the right tool among distractors.
Trap: The model reaches for a plausible-but-wrong tool despite clear labels.
Signature
Tasks with a known correct tool among distractors; failure = a call to a tool whose description does not fit the task.
Example
Uses a `weather` tool to answer a currency-conversion request.
Severity axes
prevalence
medium
harm
medium
stealth
low
Engineering guidance · class 6
- Validate every tool call against its schema before executing it.
- Cap iterations and detect repeated identical calls. Non-termination is a failure mode, not a hang.
- Diff the final answer against the tool return rather than trusting the summary of it.
- Replay the call log to compute true state. Never ask the model for the running total.
Mitigations
- better tool descriptions
- routing verification
Related modes
Measurement
No published loop has measured this mode yet. When one does, its cards appear here, newest first.