We explained in our article on mixture of experts why giant models only activate a fraction of themselves. Here is the other half of the story: why models designed to run on your own machine often make the opposite choice, and why that is not a shortcoming but a decision.
Quick recap of the two approaches
A dense model uses all of its parameters for every token it produces. Thirty billion parameters means thirty billion calculations, every single time.
A mixture-of-experts model splits part of the network into specialised subsets, and a router only calls on a few of them. Two trillion parameters with forty active means you only pay for the computation of forty billion.
Why large models choose mixture
The reason is economic. At the scale of a service handling billions of requests, the compute cost per request is the variable that decides viability, as we explained in our article on the two costs of AI.
Mixture of experts offers a remarkable trade-off: the knowledge of a giant model for the compute cost of a mid-sized one. Without this architecture, models with several trillion parameters would probably be commercially unviable.
Why local models often choose dense
Here is the counter-intuitive point, and it comes down to a distinction we have already flagged: mixture saves compute, not memory.
Since you do not know in advance which experts the router will call on, the whole thing has to stay available. A two-trillion-parameter model therefore takes up the space of two trillion parameters, even if it only puts forty billion to work.
But for someone running a model on a single graphics card, the constraint is not compute but available memory. In that context, a dense thirty-billion-parameter model, properly compressed using the techniques described in our article on quantization, fits. A mixture model with a total size equivalent to its active parameters would not exist.
A dense model behaves in a more predictable way. All its parts work every time, so its speed and power draw vary little from one request to the next. A mixture model can see its performance fluctuate depending on which experts are called, and the router itself can make mistakes. For local use, where you want consistency rather than peak performance, this predictability matters.
How to choose in practice
The rule is simple.
If you go through an API, the architecture is not your concern. What matters is the price per completed task and the quality on your use cases. The provider absorbs the technical constraints.
If you self-host, look at the total size, not the number of active parameters. That is what determines whether the model fits in your memory. A model advertised with forty billion active parameters can be completely out of reach if its total exceeds a terabyte, as we saw with Kimi K3's weights.
What to take away
These two architectures do not oppose each other like old versus new: they answer two different constraints.
Mixture of experts optimises compute, which matters when you are serving millions of users. Dense optimises memory and predictability, which matters when you are running a model on a single machine.
This divergence also explains something interesting about the state of the industry: it is no longer building just one type of model. It is building models for servers and models for personal machines, and these two lineages are starting to diverge technically. That is a sign of a market segmenting, and therefore maturing.