TechnicalAI glossary
Training
The phase where the model "learns": you show it billions of examples and it adjusts its parameters to predict better and better. That costs millions in compute.
In other words
Years of scales for a musician, except it takes months and thousands of GPUs.
Alyriaz articles that mention it
- Will students' essays train AI?
Turnitin wanted to be able to use student papers for its AI tools. Cambridge refused at first, and the software backed down. What that tells us.
- Mid-training, final fine-tuning: what happens after pre-training
Between a base model and a usable model, there are several distinct training phases. We explain what each one does and why that is where the value now lies.
- Cloudflare blocks mixed-use AI crawlers by default
Cloudflare's policy of blocking AI crawlers by default, which conflates indexing and training, has come into force. We explain the mechanism and what it changes in the balance of power with publishers.
- Sony and Warner sue Anthropic over song lyrics, targeting founders personally
Sony Music Publishing and Warner Chappell have filed a lawsuit against Anthropic and its founders over the use of song lyrics in training. We analyse the strategy and what the books precedent changes.
- Scaling laws: why "bigger" worked, and why it's slowing down
Scaling laws describe a predictable relationship between size, data, compute and performance. We explain what they actually say and why the industry is now looking for other levers.
- What is distillation? Fitting a large model into a small one
Distillation transfers the capabilities of a large model to a much smaller one. We explain the principle, why it works better than direct training, and its limitations.