TechnicalAI glossary
Sparse attention
A family of techniques where each word no longer "looks at" all the other words in the text, but only a selection: its close neighbours or the passages deemed important. Goal: break the quadratic cost of standard attention, which explodes with text length, and make very long contexts affordable.
In other words
In a meeting of a thousand people, listen only to your neighbours and the key speakers rather than trying to follow everyone at once.
Alyriaz articles that mention it
- It starts today: weekly quotas come into force
The weekly usage limits announced for an agentic coding tool take effect today. We explain the mechanism and what to check.
- SubQ: the startup that may have killed the transformer
Since 2017 and the publication of the paper "Attention is All You Need", every major AI model has been based on the transformer architecture. SubQ, launched on 5 May 2026, is the first commercial model to break with this paradigm. Here is what changes technically, and why the AI community is deeply divided over the implications.
- DeepSeek V4: China rolls out its open-source weapon against OpenAI and Anthropic
On 24 April, DeepSeek released V4-Pro and V4-Flash, two open-source models that rival Claude Opus 4.7 and GPT-5.5 for a fraction of the price. And they run on Chinese chips. Here is what that really changes.