TechnicalAI glossary
Scaling laws
The regularities observed between a model's size, the amount of data, the computing power and its performance: by increasing these ingredients, results improve in a fairly predictable way. They justified the race to "ever bigger", but each gain requires ever greater resources, and whether they are slowing down is hotly debated.
In other words
Watering more to harvest more: it works for a long time, but each extra basket needs more water than the last.
Alyriaz articles that mention it
- A model that writes a text in one go instead of word by word
Diffusion models applied to text generate an entire response in parallel rather than word by word. We explain the principle, its speed advantages and its limitations.
- Scaling laws: why "bigger" worked, and why it's slowing down
Scaling laws describe a predictable relationship between size, data, compute and performance. We explain what they actually say and why the industry is now looking for other levers.