TechnicalAI glossary
Dense model
A model that uses all of its parameters for every word it produces, as opposed to a mixture of experts (MoE), which only activates a fraction of them. Simpler to train and run, it is common among small models; but at large sizes it becomes costly, because every answer calls on the entire network.
In other words
A meeting where the whole team is summoned for every question, rather than calling in only the relevant specialists.
Alyriaz articles that mention it
- What is an exaflop? Measuring computing power, and why comparisons mislead
Computing power announcements are multiplying in exaflops. We explain this unit, why the same hardware can show very different figures, and how to read these announcements.
- What model size for which use case?
Most tasks don't require the most powerful model. We explain which size suits which use case, and why choosing too large costs more without adding anything.
- OpenAI publishes the first results from its in-house chip
OpenAI has published the first results from its in-house inference accelerator, touted as delivering strong speed and efficiency. We analyse what this move reveals about the balance of power in infrastructure.
- Meta bets on cheaper rather than better
Meta is churning out code-focused models, betting on price rather than cutting-edge performance. We analyse this positioning and what it reveals about real-world usage.
- Dense model or mixture of experts: two philosophies, two use cases
A dense model uses all its parameters for every request, while a mixture of experts only activates a fraction of them. We explain why this architectural choice determines the possible use cases, particularly on-device.
- Alibaba releases Qwen 3.8-Max, 2.4 trillion parameters, and promises to publish its weights
Alibaba launched Qwen 3.8-Max on 3 August 2026, a 2.4-trillion-parameter model aimed at code and collaborative work, with a weights release announced for the following week. We analyse what this release means in the race for openness.