How a machine learns: training

How can a program "learn"? No magic: just maths, data, and a lot of repetition.

The neural network

At the heart of modern AI lies the artificial neural network: millions of small computing units connected together, very loosely inspired by the brain. Each connection has a "weight", a simple number that says how much it matters. These numbers are called parameters; when we say a model has 70 billion parameters, we mean 70 billion tiny adjustments.

Training, in three steps

• 1. Guess. We show the network an example (a photo, a sentence to complete). It produces a response, at first, anything at all.
• 2. Measure the error. We compare its response to the correct one. The gap between the two is calculated precisely.
• 3. Correct. Each parameter is very slightly adjusted in the direction that would have reduced the error. Then we start over, billions of times.

Repeated on the scale of the entire internet, this cycle turns a network that answers at random into a system that picks up the grammar, facts, styles, and reasoning present in its data.

Data quality makes all the difference

An AI only learns what it is shown. Biased data → biased AI. False data → an AI that repeats mistakes with confidence. That's why selecting training data has become a major strategic issue, as much as computing power.