RAG (Retrieval-Augmented Generation) is an architecture in which a language model first finds relevant information in the company's own sources and then uses it to answer. The model answers from your procedures, contracts and documents rather than its general knowledge. Most enterprise AI assistants are built this way.
Why RAG?
- Freshness: answers update as soon as documents do, without retraining the model.
- Accuracy: because answers rest on real sources, the risk of hallucination drops.
- Traceability: the assistant can show which part of which document an answer came from.
- Cost: existing models are used instead of training new ones.
How does RAG work?
- Preparation: documents are converted to text, split into meaningful chunks, and numeric vectors (embeddings) representing each chunk's meaning are produced.
- Indexing: chunks are added to a searchable index with their vectors and metadata (document name, date, access rights).
- Retrieval: when a user asks a question, it is also turned into a vector and the semantically closest chunks are found. Combining this with keyword search (hybrid search) usually works better.
- Generation: the retrieved chunks are given to the model with an instruction to "answer only from these sources".
- Citations: the documents used are listed under the answer.
Decisions that determine quality
- Chunking strategy: chunks that are too small lose context; too large ones add noise. Chunking by heading structure works well for most corporate documents.
- Document hygiene: old versions, conflicting procedures and scanned PDFs that cannot be read directly lower answer quality.
- Access control: users must only get answers from documents they are allowed to see, and this filter must be applied at retrieval time.
- Saying "I don't know": when no relevant source is found, the assistant should say so instead of guessing.
How is quality measured?
Build a test set from real user questions, with the correct source and expected answer for each. Measure how often the right source is retrieved, how faithful the answer is to the source, and user feedback. It takes a discipline similar to measuring accuracy in document processing.
Using it with ERP
RAG is not limited to documents. Process guides can come through RAG, while current order or stock data is fetched from the ERP in real time via an authorised API call, all combined in one assistant. For the data-protection side of such a design, see data protection in AI projects.
RAG is one of the most practical ways to start with enterprise AI. Its success depends more on data preparation and measurement than on the model. In AI solution architecture work, the first job is deciding which documents are truly reliable sources.
