Training an AI is a bit like training a human from birth to their first job. First comes childhood, where you absorb the world indiscriminately, without any specific goal (pre-training). Then school and vocational training, where you learn to carry out specific tasks (fine-tuning). Finally, on-the-job experience, where a mentor corrects you so you become genuinely useful and pleasant to work with (RLHF). An LLM goes through these three stages, but in a few months and on servers.
You often hear that an AI has been "trained", as if it were a single mysterious operation. In reality, turning a pile of raw data into an assistant capable of answering you politely and usefully involves several clearly distinct steps. Understanding them means grasping where both the strengths and the flaws of the AIs you use come from. Let's take the journey.
Step 1: pre-training, the wild childhood
The first phase is called pre-training. By far the longest and most expensive. The model is exposed to a gigantic quantity of text — thousands of billions of words from books, websites, and code — and made to do one single thing, repeated endlessly: predict the next word, as we explained in our article on LLMs.
At the end of this step, the model has absorbed a staggering amount of knowledge and language structure. But it is still raw, wild, unusable as is. If you write it a question, it might just as easily answer it as complete it with ten more questions, because it has only learned to continue text, not to hold a conversation. It's a vast brain, but with no manners or intention.
Step 2: fine-tuning, vocational school
Next comes fine-tuning. You take the raw model and specialise it by showing it thousands of examples of the task you want it to perform. For a conversational assistant, you show it examples of good questions followed by good answers. The model then learns the expected format: receive a question, answer it directly and usefully.
This is also the step where you can specialise a model for a specific domain. A model fine-tuned on medical code will become excellent at medical terminology. A model fine-tuned on legal data will shine in law. Fine-tuning turns the wild generalist into a professional geared towards a specific use. This is what explains how, from a single base model, you can create several specialised versions.
The quality of fine-tuning often makes more of a difference than the raw size of the model. A mediocre model that is very well fine-tuned for a specific task can outperform a giant, poorly adjusted one. This is one of the reasons you should never judge a model solely by its parameter count: the way it has been educated matters as much as the size of its brain.
Step 3: RLHF, the mentor who refines character
The final step goes by an acronym you often come across: RLHF, for Reinforcement Learning from Human Feedback. This is the step that gives the AI its character, its politeness, its sense of what is useful or dangerous.
The principle: the model is made to generate several possible answers to the same question, and human evaluators rank these answers from best to worst. The model gradually learns to produce the kind of answers humans prefer. It's the mentor who, on the ground, says "that answer is better, clearer, more honest", and gradually shapes the AI's behaviour.
This step is powerful but delicate. If the evaluators reward flattering answers too much, the model can learn to say what people want to hear rather than the truth — a flaw known as sycophancy, which we touched on in our article on alignment. RLHF is therefore both the tool that makes AI pleasant and the one that can introduce subtle biases depending on how it is applied.
What to remember
A modern AI is thus the product of a three-stage journey: it absorbs the world indiscriminately (pre-training), learns to carry out tasks (fine-tuning), then refines its character through contact with human preferences (RLHF). Each step leaves its mark, its strengths and its biases, in the final behaviour you observe.
Understanding this sheds light on a lot of things. Why an AI knows facts up to a certain date (its cutoff date, set at the end of pre-training). Why two models built from the same data can have such different personalities (their fine-tuning and RLHF differ). And why the way of educating an AI has become an art as strategic as building the model itself. Behind every fluent answer you receive lies this long learning journey.