Complete beginnerLesson 1 of 56 min read
Why she makes things up, and why she sounds so sure of herself
A course from the Academy: Doubt, verify, cross-check.
Go to the quiz ↓There's one thing you need to understand, and it explains everything else: an AI doesn't seek the truth, it seeks the most plausible continuation.
What it actually does
It has read enormous amounts of text and learned to extend a sentence in a plausible way. When you ask "who wrote such-and-such a book," it doesn't consult a catalogue: it produces the name most likely to follow your question.
Most of the time that's the right one, because the plausible and the true coincide. Sometimes they don't, and nothing in its answer signals that.
Confidence is the real trap
A human who doesn't know hesitates: they say "I think," they change their tone. It doesn't. It gets things wrong in exactly the same tone as when it's right. There's no signal to spot in the wording, and that's what makes the error dangerous.
This is called a "hallucination." The word is poorly chosen: it's not delusional, it fills a gap with whatever looks most like an answer.
What to take away from this
This isn't a bug that will be fixed; it's a direct consequence of how it works. It improves, but it won't disappear. The defence, then, isn't to wait for a better version, it's to know what to check.
Key concepts
Complete the lesson
A 4-question quiz: 3 correct answers are enough, plus an exercise to practise. It opens in the Academy, on this same lesson.
Take the quiz and complete the lesson Free. With an account, your progress is saved and leads to the course certificate.The course: Doubt, verify, cross-check.
- Why she makes things up, and why she sounds so sure of herself6 min
- The three areas where you should always check6 min
- Check it in two minutes, without being an expert.6 min
- Recognising a fabricated image or voice6 min
- What we never entrust to it5 min