On 27 July, Moonshot is due to release the weights of Kimi K3, potentially the most capable model ever made freely available. A lot of people will be wondering whether they can run it at home. The answer, as we'll see, is instructive: it sheds light on what open really means, and what is actually within your reach.
Using an AI without the internet, without a subscription, without any of your data leaving your machine, and without anyone ever being able to cut off your access: that is possible today, on a decent personal computer. Here is what that actually involves, without exaggerated promises.
The four good reasons to do it
Absolute privacy. This is the strongest argument. When a model runs on your machine, your data goes nowhere. No server sees it, no privacy policy applies, no leak is possible. For sensitive documents, personal notes, client or patient data, this is a difference in kind, not in degree.
Zero cost per use. Once the model is downloaded, every request is free. You can fire off ten thousand requests without seeing a single bill, which changes everything for experimentation, learning, or high-volume processing.
Independence. No one can modify the model under your feet, change its terms, raise its prices, or cut off your access on a political decision. We have seen this year that this risk is not theoretical.
Offline operation. On a train, a plane, or in a poorly covered area, your local AI keeps working exactly the same.
What you need, concretely
The main constraint is memory, and more specifically the memory of your graphics card, known as VRAM. A model has to fit in memory to run at a decent speed. Here are the orders of magnitude, keeping in mind that they depend on the level of compression used.
| Model size | Memory required (compressed) | Realistic use |
|---|---|---|
| 3 to 4 billion | 4 to 6 GB | Summaries, rewriting, simple tasks |
| 7 to 9 billion | 6 to 10 GB | The sweet spot for most people |
| 30 to 35 billion | 20 to 26 GB | Clearly better, but demands a big card |
| 70 billion and up | 40 GB and well beyond | Out of reach for a standard consumer machine |
A raw model stores each of its parameters with high precision, which takes up a huge amount of space. Quantisation involves reducing that precision, a bit like going from a lossless audio file to an MP3. The model then takes up two to four times less memory, with a loss in quality that is often barely noticeable for everyday uses. It is thanks to this compression that models once reserved for servers now run on a desktop computer.
Where to start
Three tools dominate, from simplest to most technical. Ollama installs in a few minutes and works with a single command to download and run a model: it is the recommended entry point. LM Studio offers a full graphical interface, with a model catalogue and built-in chat, ideal if the command line puts you off. llama.cpp is aimed at those who want to control every parameter and fine-tune performance.
Practical advice: start with a 7 to 9 billion parameter model. That is the balance point where quality becomes genuinely useful while remaining comfortable on a decent desktop machine.
The limits, stated honestly
Let's be clear about what you should expect. A local 8 billion parameter model is not Claude Fable 5 or GPT-5.6. It will be good at summarising a document, rewriting text, answering general questions, and producing simple code. It will be noticeably weaker on complex reasoning, long multi-step tasks, and it will hallucinate more on precise facts.
The right way to look at it is to think in terms of use cases rather than fidelity. For anything sensitive, repetitive, or high-volume, local is excellent and free. For tasks that demand the best reasoning available, cutting-edge online models keep a real lead. The two complement each other very well.
The Kimi K3 lesson: open does not mean accessible
Let's return to our starting point, because it is highly instructive. Kimi K3 has 2,800 billion parameters. Even heavily compressed, such a model remains completely out of reach for a personal computer: its own creator recommends serving it on clusters of at least 64 accelerators linked at very high bandwidth.
This illustrates an essential nuance of the concept of an open model. Open means that anyone has the right to download, modify, and deploy the model. It does not mean that anyone has the material means to do so. Legal freedom is total; practical freedom depends on your infrastructure. For individuals, it is the mid-sized open models, between 3 and 30 billion parameters, that constitute the real democratisation.
Installing an AI on your own machine remains an experience worth trying, if only once. Seeing a model respond while the wifi is cut off changes something in how you perceive these tools: they stop being a service you subscribe to and become software you own again.