When you read that a model scored a certain figure, there is almost always a small-print mention: maximum effort, high effort, with fallback. This setting can shift the result by dozens of points, and the cost by a factor of five to ten. Two tables comparing models at different effort levels compare nothing at all. And the default setting on your access is almost never the one that produced the announced figures.
Yesterday we published a table comparing GPT-6 Astra with its rivals. Here is the hidden variable that decides what those figures are worth.
What this parameter controls
Recent models produce an internal draft before answering, as we explained in our article on reasoning. The effort level determines how much of that draft the model is allowed to produce.
At low effort, the model thinks briefly and answers. At maximum effort, it can explore several paths, backtrack over its steps, check, and start again. The same model, with the same parameters, produces very different results depending on this slider.
Providers generally offer four or five levels, with names that vary — low, medium, high, maximum, sometimes an extra level reserved for intensive use.
What this changes in practice
Quality. On a simple task, the gap is zero: thinking for ten minutes about a vocabulary question improves nothing. On a hard problem — a long chain of reasoning, complex debugging, a proof — the gap can be considerable.
Cost. The internal draft is made up of tokens, and those tokens are billed. Maximum effort can consume five to ten times more tokens than low effort for the same question. This is what we described with a model consuming twelve times more reasoning tokens.
Latency. More effort means more time before the first response. For a conversation, that is noticeable. For an agent running in the background, it is irrelevant.
Here is the most important practical point. For GPT-6 Astra, the default effort for the programmatic access is set to low. The announced scores were obtained at maximum effort. In other words, if you plug in the model without touching this parameter, you will not get what was presented, and you may conclude it is disappointing. This is not specific to this model: it is a common choice, because a frugal default protects users from a surprise bill. But it means that the model you are testing is not the one from the press release, unless you adjust it.
Why comparisons depend on it
This explains a large share of the discrepancies between a provider's tables and those of an independent evaluator.
A lab tests its model at the level that favours it, which is legitimate as long as it is stated. An independent evaluator has to choose a comparable level for all, which is difficult because the scales do not match from one provider to another. A "high" at one may look like a "medium" at another.
When we wrote yesterday that Fable 5.1 was the most expensive model per task in the independent ranking, that was at maximum effort. At medium effort, the cost ranking changes. That is why we keep stressing the need to read benchmarks carefully: a score without its effort level is a number without a unit.
How to set it
The rule is simple and rarely applied: match the effort to the difficulty, not to the model.
For classification, extraction, rewriting: low effort. The result will be identical and the bill divided by five.
For complex reasoning, non-trivial code, long analysis: high or maximum effort, accepting the cost and the latency.
For a serious architecture, this is routing: simple tasks go to low effort, hard ones to high effort, and escalation happens automatically when the low-effort result fails a check. That is the logic we have been recommending for months, applied to one more slider.
What to remember
This parameter is probably the most under-used of all. Most users leave the default and judge the model on what the default produces.
That has two consequences. Many pay too much for simple tasks. And many conclude a model is not worth what is said about it, when they have never run it at the level that produced what is said about it.
Before comparing two models, first compare the same model at two effort levels on your own task. You will often find that the gap between settings is larger than the gap between providers.