Most of an Agent's Reliability Lives in the Harness
What makes a coding agent reliable is mostly the layer around the model: the context it is handed, its tools, the checks on its output and the code that orchestrates sessions. That layer is ordinary engineering.
When a coding agent gets something wrong, the reflex is to blame the model or wait for the next one. The argument behind "harness engineering" is that this looks in the wrong place. The same model behaves very differently depending on what is wrapped around it.
I find the argument persuasive, with reservations about the numbers attached to it. The part I care about most is a quieter claim inside it: the wrapper is made of text files, scripts and tests, and an engineer who has never trained a model can build it.
Four layers
The cleanest map I have is from Alan Walsh of The AI Automators, in a video about OpenAI's Symphony spec. He borrows the inner and outer vocabulary from Birgitta Bockeler's article on Martin Fowler's site.
| Layer | What it is for | Who builds it |
|---|---|---|
| Model | Inference. Keeps no state between calls. | The vendor |
| Inner harness | The agent's main loop, tool handling and compaction, plus the surfaces for rules, skills, hooks and permissions | The vendor ships it, you fill it in |
| Outer harness | Code around the agent that enforces a process: start a session, hand it state from disk, gate what comes out | You |
| Orchestrator | Schedules many harness runs against a work queue such as a ticket board | You, or a spec like Symphony |
One detail in Walsh's account is easy to get backwards. A rules file and a hook feel like things I own, because I write them. The surfaces themselves are inner harness: the vendor decides when a rules file loads and what a hook may do. The outer harness proper is code of my own that controls the agent's lifecycle from outside.
What the agent is handed
Cole Medin teaches the inner layer as six components. Rules carry conventions. Skills carry workflows. MCP servers add capabilities, and LSP integration gives the agent a way to navigate code. Hooks enforce guardrails. Subagents move work into a separate context window.
Bockeler's split, as Walsh presents it, is more useful to me for design. Guides shape the agent's first attempt. Sensors catch mistakes afterward and feed them back. Most of Cole's six are guides. Hooks are where sensors attach.
The mindset Cole pairs with this comes from Dex Horthy of HumanLayer, whom he quotes: "It's not a model problem. It's a configuration problem." If the agent broke a convention, the convention goes into the rules file, and every later session inherits the fix.
Checks that do not ask
Sensors come in two kinds. Computational ones involve no model: linters, type checkers, schema validators, tests. Inferential ones use a model as a judge or reviewer. Walsh's main argument, taken from the Fowler article, is that people building with agents reach for the model-as-judge first and underuse the plain tools, which are faster, cheaper and deterministic.
Cole's demo repository makes this concrete with three hooks. One runs before any tool call and refuses to read or write a real .env file or delete a directory recursively. One runs a linter or type check after every file edit. The third blocks the agent from stopping until lint and tests pass.
The third is the one I would copy first. Telling an agent to run the tests before finishing is a request, and it may or may not comply. A stop hook removes the choice. Walsh frames the whole field this way: a prompt framework asks the model to follow a process, and a harness controls it. I write embedded C++ for a living, and I trust a failing build more than a review comment for the same reason.
Orchestration
The smallest outer harness in both videos is the Ralph loop, credited to Geoffrey Huntley:
while true; do claude -p "$PROMPT"; done
Each pass starts a fresh agent. State lives in git and on disk, so there is no long context to degrade and a crash costs one iteration. Gates can sit between passes.
Walsh lists the costs: every iteration spends tokens, and with weak gates the loop can make steady progress toward the wrong goal for hours. Cole's video covers neither cost nor failure modes, and it does not say where human review belongs. Walsh places human gates one layer up, where runs are scheduled against tickets.
The numbers are reported, not mine
In an April video Cole says studies show pull request acceptance rising from 6.7 percent with a bare model to about 70 percent with a harness. My notes do not name the study. In the May video he cites a Terminal Bench 2.0 result, credited to Viv Trivedy, where an agent moved from the top 30 to the top 5 by changing only the harness. No figure appeared on screen; he was reading from an article by Addy Osmani. Walsh quotes OpenAI's claim of a 500 percent increase in landed pull requests and flags it himself as marketing copy.
I have measured none of this. Both channels also sell something: Cole built Archon and runs a course, and The AI Automators close with a course pitch. I trust the direction, because people with different products report the same pattern. I do not trust the magnitudes.
There is also a counter-position. Mario Zechner, who built the Pi agent, argues that current models are already trained to act as coding agents and that a heavy harness can confuse more than it helps. I read the two together as a rule about fit: add a layer when a specific failure calls for it.
Why I think this is buildable
A rules file is markdown. So is a skill. A hook is a short script that allows or denies an action. What it takes is someone who knows what correct looks like in their codebase and can write that down as a check.
My own setup is a partial example. At home I use Claude Code with routing files in each folder of my vault and a library of a few dozen skills. My research skill has a real sensor in it: it will not finish a note until every timestamp anchor verifies. When I mapped my setup against Cole's six components in late May, hooks were the missing one, and I had nothing at the multi-session layer. I had guides in quantity and few sensors.
Key Takeaways
- An agent is a model plus a harness, and part of the harness is yours to write.
- Guides improve the first attempt. Sensors catch what the guides missed. Deterministic sensors are underused.
- A hook or gate turns an instruction the agent might follow into one it cannot skip.
- The headline numbers come from people selling harness tools. The direction is consistent. The sizes are unverified.
- Every piece is a text file, a script or a test.
Sources
- Cole Medin, "Harness Engineering: What Separates Top Agentic Engineers Right Now"
- The AI Automators, "OpenAI Just Showed Us What Comes After the Harness. Here's The Layer Almost Everyone's Missing."
- Cole Medin, "The Next Evolution of AI Coding Is Harnesses - Here's How to Build Them"
- AI Engineer, "Building pi in a World of Slop -- Mario Zechner"
The item at the top of my list from this: a stop hook that will not let a session end while its checks are failing.