Embeddings and RAG: connecting AI to your data

How can an AI answer from your documents, your company's internal policies, your notes, articles from a site, without being retrained? Answer: embeddings + RAG, the duo behind most "enterprise AIs"... and behind Mnémosyne, this site's assistant.

Embeddings: meaning turned into numbers

An embedding converts a text into a long list of numbers (a vector) that captures its meaning. Magic property: two texts close in meaning have vectors close in geometry. "How do I cancel my subscription?" and "I want to terminate my contract" share almost no words, yet their vectors are neighbours. Search stops being about keywords and becomes about meaning.

RAG: retrieve, then answer

RAG (Retrieval-Augmented Generation) chains three steps:

• 1. Index: your documents are split into passages, each converted into an embedding, all stored in a vector database.
• 2. Retrieve: for each question, the passages closest in meaning are pulled up.
• 3. Generate: those passages are slipped into the LLM's context with the instruction: "answer based on these excerpts".

Why it's everywhere

RAG reduces hallucinations (the model relies on real sources), lets you cite those sources, and stays up to date by simply swapping documents, without retraining anything. Its limits: if retrieval surfaces the wrong passages, the answer will be poor; the quality of the chunking and the index makes all the difference.