Term · AI
RAG (retrieval-augmented generation)
RAG stands for retrieval-augmented generation. The AI system first searches your own documents and then answers, with the source included.
In short
RAG (retrieval-augmented generation) is a technique in which an AI system first looks up relevant passages in a collection of documents and passes them to the language model. The model bases its answer on those passages and can name the source. That way AI works with your information, without you retraining the model.
What is RAG?
RAG is a way to have a language model answer on the basis of your own documents. First the system looks up the passages of text that match the question. Then the language model writes an answer using those passages.
The name explains it. Retrieval is fetching, augmented is supplemented and generation is producing. The term comes from a scientific paper from 2020.
How does RAG work?
- Preparing. Your documents are divided into small pieces and made searchable. Think of manuals, terms and conditions or product sheets.
- Searching. When someone asks a question, the system looks for the pieces that fit best.
- Answering. The language model gets the question and those pieces. It writes an answer and says where it comes from.
The model itself does not change. If there is a new version of a document, you replace the document. That is different from training or fine-tuning, where you adjust the model itself.
Example in an SME
A made-up example
A wholesaler of fasteners has a large folder of product sheets. An inside sales employee asks the assistant: "Which plug suits aerated concrete?" The system looks up the relevant product sheets. The language model gives a short answer and puts the links to those sheets underneath.
What should you look out for?
- The quality of your documents. The answer is as good as the information that goes in. Old or contradictory versions give confusing answers.
- Fewer mistakes, not zero. RAG reduces the chance of a hallucination, but does not rule it out. Have the system show the source, so that you can check.
- Who may see what? Make sure that someone only gets information via the assistant that they would also be allowed to access without the assistant.
- Personal data. Do the documents contain data of customers or employees? Then the GDPR applies.
RAG is the technique behind an AI assistant on your own documents. Read more in What is an AI assistant on your own documents? I build such an assistant as custom work. Read more about AI in your business.
Frequently asked questions
What is the difference between RAG and training a model?
With RAG the model stays the same. It gets the relevant passages with each question. With training or fine-tuning you adjust the model itself. RAG is easier to keep up to date if your documents change often.
Do well-known AI tools work this way too?
Many business AI tools work in a similar way. The business version of Microsoft Copilot, for example, searches your email and files and can name the sources in its answer.
Sources: Lewis et al. (2020): Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, Dutch Data Protection Authority (Autoriteit Persoonsgegevens): Guidance on generative AI and the GDPR (in Dutch) and Microsoft Learn: Data, Privacy, and Security for Microsoft Copilot. Consulted on 7 October 2026.
Written by Bart van Engelenhoven Founder of Uniekaiconsultancy in Ede, the Netherlands. Bart analyses how SMEs work and builds software, integrations and AI applications himself. Last updated: 7 October 2026
Read more
Term
LLM (large language model)
An LLM (large language model) is the language model behind services like ChatGPT. It keeps predicting the next word. What that means for your business.
Read moreTerm
AI assistant
An AI assistant helps you with text, questions and small tasks, often in software you already use. What it is, how it works and what to watch for.
Read moreTerm
AI hallucination
An AI hallucination is an answer that sounds convincing but is wrong. Why language models do this and how to check an AI answer.
Read moreNext step
Curious what could work for you?
- The first conversation is free and without obligation
- By phone or at your premises
- You speak directly with me
Bart van Engelenhoven You tell me what takes up your time, I tell you honestly whether I can help.