Read a model name

On open model catalogues, the names look like codes: Family-3-30B-A3B-Instruct-Q4_K_M.gguf. Each part means something, and knowing how to read them saves you from downloading twenty gigabytes for nothing.

Decoding, piece by piece

• The family and version: the model name and its generation. A more recent version of the same family is generally better at the same size.
• 30B: 30 billion parameters in total (B for billion). This is what determines the memory required.
• A3B: 3 billion active parameters per token. A sign of a mixture of experts: fast like a small model, but you still have to load all 30 billion.
• Instruct (sometimes it or chat): a version tuned to converse and follow instructions. A Base version is just a text completer: it does not answer like an assistant.
• Thinking or Reasoning: a version that thinks before answering, better on problems, slower and more demanding.
• Q4_K_M: the quantization level, here 4 bits with a common method that preserves quality well. Q8 is more faithful and twice as heavy; Q2 or Q3 are lighter but degrade noticeably.
• .gguf: the file format used by most mainstream local tools.

The other labels

Coder indicates a specialisation in programming; VL or Vision, a model that accepts images. Distill indicates a small model trained to imitate a large one: often excellent for its size, but not the equivalent of the large one.

The licence, which everyone forgets

"Open" does not mean "free for any use". Some models are published under very permissive licences (Apache 2.0, MIT), others under custom licences with conditions: user thresholds, prohibited uses, obligation to credit the model. For personal use, this is rarely a blocker; for a product or commercial use, read it before building on it.

Key points

• 30B: total parameters, so the memory. A3B: active parameters, so the speed.
• Instruct for conversing; Base is just a text completer.
• Q4_K_M, the common compromise; Q8 more faithful and heavier, Q2 too degraded.
• Read the licence before any commercial use.