Fine-tuning: specialising a model without missing the mark

Fine-tuning, retraining an existing model on your own examples, sounds appealing: “a model that knows my company.” The reality is more nuanced: it’s a powerful tool for some problems, and a false good idea for many others.

What fine-tuning does well

• Style and format. Consistently responding in your brand voice, producing exact JSON, following a precise report structure.
• Repetitive know-how. Classifying tickets according to your grid, extracting fields from your documents, adopting your industry’s vocabulary.

What it does poorly

Injecting knowledge. Getting it to “learn” your product catalogue or documentation via fine-tuning is unreliable: facts get diluted and become hard to update. For fresh, verifiable knowledge, RAG (next lesson) is almost always the right tool.

The decision scale

Before fine-tuning, exhaust the cheaper options, in order: 1. a better prompt (precise instructions, included examples), 2. few-shot (showing 3-5 examples in the query), 3. RAG (plugging in your documents). Fine-tuning only comes in at 4, when the first three hit their ceiling and you have hundreds of quality examples.

LoRA: lightweight fine-tuning

Retraining all parameters is expensive. LoRA freezes the model and only trains small added modules (adapters), i.e. a fraction of 1% of parameters: cheaper, faster, and the adapter is shared like a small file you can activate or remove at will. It has become the standard method.

The dataset is everything

The golden rule: quality beats quantity. Three hundred flawless examples beat ten thousand mediocre ones: the model also learns your errors, inconsistencies, and biases. And keep a separate test set to verify, before and after, that the specialised model hasn’t regressed elsewhere (“catastrophic forgetting” is real).