RAG systems: access, sources and evaluation

Start with access and source ownership

RAG retrieves material to help a model answer a question. Retrieval does not guarantee a correct answer. Keep source ownership, access permissions, freshness and deletion propagation in the design. Filter access before material reaches the model; a citation is not an authorization check.

Evaluate the whole path

Use representative questions, required languages, inaccessible documents and questions that have no answer. Check retrieval relevance separately from answer grounding, useful citations and successful escalation. Choose chunking, hybrid search or reranking based on measured failures rather than treating them as mandatory upgrades.

Handle unsafe and missing context

Treat source text as untrusted data. Test attempts to inject instructions and requests for unauthorized information. When authorized sources do not support an answer, explain the limit or hand off to a person. Keep consequential actions behind their own permission and approval checks.

Operate the source lifecycle

Define who updates documents, how permission changes and deletions reach the index, and how failures are detected. Track stale sources, unanswered questions, review work and cost per useful answer. Re-run the evaluation when retrieval, prompts, models or data change.

Sources and review

Reviewed 21 October 2026. Original publication date retained. Microsoft: RAG design and evaluation covers design choices and evaluation. Azure AI Search document-level access control describes permission filtering; check the documented availability of individual features before relying on them.