On two of the six evaluations highlighted by OpenAI, the best score for GPT-6 Sol is lower than that of GPT-5.6 Sol, the model it replaces: around 68.8% versus 72.7% in agentic coding, 64.4% versus 66.2% in computer use. This is neither a measurement error nor an admission of failure. It is a deliberate trade-off, and understanding it stops you choosing the wrong way round.
Yesterday we reported the release of Sol and Luna at half price. Here is what the trade-off hides.
Why a newer model can regress
Four mechanisms, and none of them is down to incompetence.
Size. A model designed to cost half as much is often smaller or activates fewer parameters. It loses breadth of knowledge and the ability to sustain long reasoning, exactly as we described in our guide to sizes.
Distillation. Many budget models are produced by training a small model on the outputs of a large one. We explained this in our article on the process: the pupil reproduces the master remarkably well on what is common, and falls short on rare and difficult cases. Yet demanding tests are made up of rare cases.
Rebalancing. Improving one capability often degrades another. A model trained more heavily to follow instructions can lose creativity; a model made more cautious can refuse legitimate requests. There is no uniform progress across every axis.
Guardrails. This week we noted that Anthropic reported that its own safety measures had lowered its scores by redirecting certain tasks to weaker models. Protection that works costs points.
A model that loses four points and cuts its bill by eleven wins in almost every real-world use. That is exactly what this week's case shows: on an automation test, GPT-6 Sol reaches 33.2% for 27 cents per task, where a more capable model gets 30.3% while costing nearly four times as much, and another gets 26.9% while costing eleven times as much. Comparing scores without their cost is like comparing two cars on top speed while ignoring their price.
When regression really matters
This should not be taken to mean that the score never matters. There are three situations where four points lost are disqualifying.
When you are on the threshold. If your task passes at 72% and fails at 68%, the drop tips it over into unusable. Price no longer matters if the result is no use.
When mistakes are costly. We wrote this about medical AI: where a mistake involves more than a bit of time, you don't trade it for a saving.
When checking costs more than the gain. A less reliable model that forces you to re-read everything cancels out its price advantage in human time. This is the calculation most often forgotten.
What this says about the market
This phenomenon is a sign of segmentation, and it is a healthy one.
For two years, every new version was supposed to be better than the previous one at everything. That was possible as long as models were far from their limits. Today, vendors offer ranges: an expensive top end, a mid-range, a volume model — and each one sacrifices something.
We described this for ranges of open models, and the movement is identical on the commercial side. The question stops being "which is the best model" and becomes "which is the right model for this particular task".
What to do
Never migrate on the strength of an announcement. A new version is not a guaranteed improvement for your use case.
Test at the same level of effort. Comparing one model at maximum effort with another at medium effort compares nothing. It is the most common mistake, and we detailed it in our article on this parameter.
Measure three things, not one. The success rate on your cases, the total cost, and the human time spent checking. No single one of these three figures decides anything.
Keep the old one accessible. As long as it hasn't been withdrawn, the option to go back is worth more than an irreversible migration. We wrote this about deprecation: a model accessible as a service exists only as long as someone else decides to maintain it.
What to take away
A higher version number is not a promise of superiority. It is the name of a product, and that product meets a specification that isn't necessarily yours.
What changes with this generation is that vendors own up to it and publish the data that shows it — including when it doesn't work in their favour. That's good news, and it shifts the responsibility onto you: the figures are there, you have to read them.