Skip to content

TechnicalAI glossary

Token

The smallest unit of text a model processes, often a piece of a word. "Incroyable" can be 2-3 tokens. API prices are counted in tokens.

In other words

The LEGO of language: the model assembles bricks, not whole words.

Alyriaz articles that mention it

Go further at the Academy

See also

More in this family: Technical

Find it in the full glossary →All the terms