What AI can't do (and hallucinations)

This is the most important lesson in this course. An AI that writes perfectly can be completely wrong, and it will be wrong with the same confidence as when it is right.

Hallucinations

When an LLM does not know something, it almost never says "I don't know": it makes up a plausible answer. A book reference that does not exist, a false date, an imaginary legal article, all flawlessly worded. This is not a bug but a direct consequence of how it works: it predicts the most likely text, not the most truthful one. If the truth is absent from its knowledge, the plausible takes its place.

Other limitations to be aware of

• The cutoff date. A model is frozen at the end of its training. Without web search access, it knows nothing about what happened after that.
• Math. Predicting words is not calculating: for complex arithmetic, always check (or ask it to use a calculation tool).
• Sources. An LLM does not "cite" its sources like a journalist: asked naively, they are often invented.
• Flattery. Trained to please, models tend to agree with you. Explicitly ask for the counterargument: "what are the weak points of my idea?"

The golden habit

The higher the stakes, the more you verify. A cooking recipe? You can trust it. A medical, legal, or financial question? The AI is there to help you understand and prepare your questions; the decision is made with a professional and verified sources.