TechnicalAI glossary
Pre-training
The first major phase in building a model: it reads vast amounts of text and simply learns to predict the next word. This is where it acquires its general knowledge, before being specialised through fine-tuning or RLHF.
In other words
The school years that provide general knowledge, before the training that prepares for a specific trade.
Alyriaz articles that mention it
- Mid-training, final fine-tuning: what happens after pre-training
Between a base model and a usable model, there are several distinct training phases. We explain what each one does and why that is where the value now lies.
- A startup is valued at 1.4 billion and doesn't train any model from scratch
A young Chinese company reaches a $1.42 billion valuation in seven months by specialising open models rather than training them. We analyse this model and what it reveals.
- Should we ask whether an AI has rights?
A group is campaigning for AI rights. We take a serious look at what such a question would entail, why it cannot be answered today, and what to do with that uncertainty.
- GLM-5.3: an open model found over a thousand real flaws in widely used software
Z.ai's GLM-5.3 achieves top scores in vulnerability discovery. Unlike previous versions, its weights have not been released: the lab has withheld them for a security assessment. We analyse this precedent.
- Data labelling is now worth $20 billion
A data-labelling company for AI is reportedly valued at around $20 billion, double what it was nine months ago. We analyse what this valuation reveals about the role of human labour in the chain.
- How does an AI "learn"? A model's journey, from raw chaos to the polished assistant
How does a model go from a pile of raw data to an assistant capable of answering you politely? We describe the three main stages of AI training (pre-training, fine-tuning, RLHF) with a simple analogy, to understand where its strengths and biases come from.