AdvancedLesson 2 of 49 min read
Where long reasoning stops paying off
A course from the Academy: The models that reason
Go to the quiz ↓Three serious limitations, and none of them is an implementation detail.
Returns diminish, then reverse
Doubling the reasoning budget does not double performance: gains flatten out quickly. Beyond a certain point, performance drops on some tasks. The model over-analyses a simple question, considers cases that do not exist, and talks itself into a convoluted interpretation.
This is observable by hand: ask a trivial question of a reasoning model and watch it build a factory.
The displayed reasoning is not always the real one
The most counter-intuitive point, and one documented by several studies: the chain of reasoning produced is not necessarily faithful to the process that led to the answer. A model can display clean reasoning while having decided on a cue it does not mention.
Practical consequence: reading the chain is reassuring, but proves nothing. It is not an audit trail.
An early error consolidates
The model reviews itself and builds on it. A false assumption at step two is taken as given in later steps, and the reasoning becomes increasingly coherent with the error. The final answer is then very confident and wrong.
What we do with this
On a real issue, running the model several times separately and comparing results beats reasoning longer in a single run. Divergence between runs is a far more reliable signal than the length or confidence of the reasoning.
Complete the lesson
A 4-question quiz: 3 correct answers are enough, plus an exercise to practise. It opens in the Academy, on this same lesson.
Take the quiz and complete the lesson Free. With an account, your progress is saved and leads to the course certificate.The course: The models that reason
- Spending compute at inference time10 min
- Where long reasoning stops paying off9 min
- Allocating the thinking budget9 min
- Verify rather than think9 min