RAG stands for Retrieval-Augmented Generation. Before answering, the system searches approved documents for relevant passages and gives them to the language model as context. Employees can query contracts, manuals, wikis, or technical documentation in natural language and inspect the sources used in the answer.
RAG is not one database feature. Quality comes from a chain of document preparation, retrieval, permission checks, context assembly, model response, and evaluation.
The RAG pipeline in detail
- Ingest and parse: PDF, Word, Markdown, Confluence, or file shares are imported. Tables, headings, page numbers, and document IDs are preserved as structure where possible. Scans require OCR first.
- Segment: content is not blindly cut at one fixed character count. Chunk size and overlap follow the document type, paragraphs, tables, and likely questions.
- Embed and index: an embedding model converts passages into vectors. Qdrant or Postgres with pgvector stores those vectors alongside source, version, and permission metadata.
- Retrieve: semantic vector search can be combined with keyword search and metadata filters. This helps with domain terms, product numbers, or exact clauses.
- Rerank: a reranker can reorder candidates for the specific question before only the strongest passages enter the context window.
- Answer and cite: the model receives the question, selected passages, and explicit response rules. The interface links to the document, page, or section and shows when no reliable source was found.
Permissions are part of retrieval
A RAG system must not wait until after generation to check whether a document was visible. User or group permissions need to constrain retrieval candidates first. I inherit existing permissions where source systems expose them reliably, or define an explicit access and indexing model. Separate tenants, HR files, or confidential contract areas do not enter one uncontrolled shared index.
Stack options
- Qdrant for a dedicated vector search service with filtering and hybrid retrieval options
- Postgres + pgvector where relational data, permissions, and vectors sensibly belong in an existing data model
- LlamaIndex, LangChain, or custom Python/TypeScript for ingestion, retrieval, and orchestration
- hosted model APIs or local inference through Ollama/vLLM, depending on quality, data class, load, and operations
- an existing chat client, small web frontend, or integration with Slack, Teams, a portal, or a line-of-business application
How quality is evaluated
- real questions with expected sources and professionally acceptable answers
- retrieval metrics: is the correct passage found and ranked highly enough?
- answer checks: is each material claim supported by the supplied context?
- tests for missing sources, contradictory documents, and stale versions
- latency and cost measurement at realistic document volume and concurrency
- regression tests when chunking, embeddings, reranker, prompt, or model changes
What’s included
- document, format, volume, and permission analysis
- ingestion, parsing, chunking, and reindexing pipeline
- embedding, database, and retrieval configuration
- source display, authentication, and permission filters
- evaluation set, quality baseline, and operational monitoring
- written architecture and operations documentation plus user onboarding
Realistic limits
RAG does not eliminate hallucinations. A model can miss a source, misinterpret it, or generalise beyond the evidence. The system therefore needs grounded answers, a visible “no reliable source” behaviour, and human review for critical results. Document cleanup, complex OCR, or professional classification of a large legacy archive is scoped separately after inspection.
After discovery, the usual planning range is 4–8 weeks. Document volume, access controls, integrations, and the agreed evaluation depth determine the actual schedule.