Six months ago, a ranking compared models. Today, the independent evaluator we cite compares "GPT-6 Astra in Codex" to "Fable 5.1 in Claude Code" to "Muse Spark in Muse Code". The model is no longer measured alone: it is measured with its shell. This change in method says something important about what actually produces the result.
We used this term in our Friday comparison without explaining it. Here is what it covers.
What a model does not do alone
A language model, taken in isolation, receives text and produces text. That's it. It does not read files, run code, launch tests, or remember what it did ten minutes ago.
For it to become an agent, it needs to be surrounded by software that provides these capabilities and organises its work. That software is what we call the harness.
What a harness contains
Tools. Reading and writing files, running commands, searching a codebase, calling services. Each tool is described to the model, which decides when to use it.
The loop. The harness runs the model in a cycle: observe, decide, act, observe the result, start again. It decides when to stop, when to ask for confirmation, when to give up.
Context management. An agent accumulates history. The harness decides what to keep, what to summarise, what to discard. This is a point we highlighted regarding the drop in cache usage: a harness that reuses context well costs far less than one that sends it back in full every turn.
Guardrails. What the agent is allowed to do without asking, what requires validation, what is forbidden. This is the layer we discussed regarding flaws in file-access tools.
The same model in two different harnesses produces different results, sometimes by several points in a ranking. A harness that presents its tools poorly, manages context badly, or stops too early degrades an excellent model. A well-crafted harness gets more out of an average model than it seems worth. This is exactly what we described in our article on invisible settings, applied to agents: part of the value is in the shell.
Why labs build their own
Two years ago, harnesses were built by third parties, and models were interchangeable. Today, each lab ships its own, optimised for its model.
The reason is strategic and ties into what we see everywhere. When models converge, differentiation shifts to what you put around them. A well-integrated harness creates a habit, and a habit retains better than a score. This is the logic we described regarding Ox Alpha buying its way into workflows.
There is a downside: a proprietary harness reintroduces a dependency that open models had reduced. You can switch models easily; switching harnesses means relearning habits.
What this means for choosing
Test the pair, not the model. A ranking of isolated models no longer says much about what you will get. What matters is the model in the tool you will use.
Look at token efficiency. This is where harnesses differ most, and it is what decides the bill. On Friday we reported that one model consumed a third of the tokens of another for a comparable result: part of that gap comes from the harness.
Favour those that speak open protocols. A harness compatible with the standards currently consolidating lets you switch models. A closed harness locks you in twice over.
What to remember
The shift in rankings towards model-harness pairs acknowledges a reality that isolated evaluations masked: an agent is a system, not a model.
This makes comparisons more honest and more complicated. And it moves part of the competition onto a less visible ground, that of tooling, where a good share of what keeps users engaged will likely play out in the years to come.