UseAI glossary
Data contamination (benchmarks)
A situation where benchmark questions, sometimes along with their answers, end up in a model's training data: at test time, it recites rather than reasons, and its score is artificially inflated. This is one reason to treat announced records with caution.
In other words
A student who got the exam paper in advance: their grade proves they have a good memory, not that they understood.
Alyriaz articles that mention it
- The model rankings as of 23 September 2026
State of the model landscape as of 23 September 2026: who leads on what, at what price, and how to read leaderboards that have become hard to compare.
- An open 744-billion-parameter agent beats the best closed model on web navigation
Shanghai AI Lab has quietly released a 744-billion-parameter agent that beats GPT-5.6 Sol on a web navigation benchmark. We look at what that result actually means.
- Why benchmarks age so quickly
Benchmarks lose their discriminating power within a few months. We explain the three mechanisms that wear them down, and how to read a score accordingly.
- Sony and Warner sue Anthropic over song lyrics, targeting founders personally
Sony Music Publishing and Warner Chappell have filed a lawsuit against Anthropic and its founders over the use of song lyrics in training. We analyse the strategy and what the books precedent changes.
- Can an AI have an idea that no one has had before?
A benchmark published in August 2026 measures models' ability to generate a scientific hypothesis by stripping away anything they might have memorised. The results suggest a real ceiling. We explain the protocol and what it reveals.
- What if AI ends up poisoning itself?
Model collapse describes what happens when an AI trains on content generated by other AIs: a gradual impoverishment, like a photocopy of a photocopy. A recent study revives the concern. We explore the phenomenon and the philosophical question it raises: can an intelligence that feeds only on itself still learn?