Every year the word moves one layer further from the model. That is what model maturity looks like from the outside.
prompt → context → harness
Starting the census · Part 3 of 3
The word follows the bottleneck
Prompt, then context, now harness. Every year the industry's favourite word moves one layer further from the model — and that is what maturity looks like.
The word everyone is using this month is harness.
It is not a new word. Evaluation people have used it for years. What is new is that it stopped meaning the rig you test a model in, and started meaning everything around the model in production.
That shift is not a fashion. It is a signal.
The forces
Each renaming arrived shortly after models got good enough that the previous layer stopped being the limiting factor. When a model could barely follow an instruction, the instruction was everything. When it could follow one reliably, what you put in front of it mattered more. Once agents started running for hours unattended, the model stopped being the only thing that could go wrong.
Old way, new way
How this plays out
Read any harness component list backwards and you get a failure taxonomy. Every guard exists because something specific goes wrong without it.
| harness component | the failure mode it exists to catch |
|---|---|
| Schema validation on tool calls | Tool-Argument Validity |
| Iteration cap and repeat detection | Agentic Loop / Non-Termination |
| Citation checker | Citation Resolution Failure |
| Today's date injected in context | Date-Arithmetic Error, Post-Cutoff Fabrication |
| Re-read state before acting | Ledger Reconciliation Failure |
| Constraint restated every turn | Constraint Decay at Depth |
| Paraphrase tests in CI | Paraphrase Non-Invariance |
| Answer diffed against tool return | Tool-Return Groundedness |
A harness is a list of mitigations. A census is the list of what they are mitigating. You cannot build the first well without the second.
What I would do now
Name the modes your application cannot tolerate, then design guards against them — not the other way round.
Why: A harness built from a list of popular components guards against someone else's failures and misses yours.
If nobody can name the failure a component exists to catch, it is either redundant or untested.
Why: Guards accumulate. Without a named purpose, nobody ever feels safe removing one, and nobody knows which ones actually work.
Run the same cases with a guard on and off. If the failure rate does not move, the guard is decoration.
Why: This is the same controlled comparison you would demand of a model claim, pointed at your own engineering.
That is what this series is about: a census of how models fail, so that the harness around them is built against something real. Next, the measurements.
Pick one component of your harness. Which named failure is it there to catch — and when did you last check that it still does?
- EleutherAI lm-evaluation-harness (repository created August 2020)
- Mitchell Hashimoto — My AI adoption journey (5 February 2026)
- OpenAI — Harness engineering: leveraging Codex in an agent-first world
- InfoQ — OpenAI introduces harness engineering (21 February 2026)
- A survey of context engineering for large language models (July 2025)
Every figure here describes something measured and committed. See the measurements · read the method