Enterprise RAG Design: AI That Cites Sources and Respects Access Boundaries

Imagine an employee who has to open dozens of documents to find a procedure. RAG can make that search easier by retrieving relevant content and providing it as context for a language model’s answer. But a fluent answer alone does not prove that the right document was found, or that the user is authorized to see that document.

Make visible which document the answer is based on

In the starting design we recommend, every important claim is shown together with the document name and the relevant section. The user should be able to open the source and check it. Simply placing three links under the text is not enough: the links must actually support the claim next to them.

For example, when a user asks “Whose approval do I need for remote work?”, the answer should show the approval step of the current procedure. If an exception that does not exist in the document is asked about, the system should say so — not invent a rule from a similar document.

Enforce document permissions at the retrieval stage

Microsoft’s documentation on document-level access control describes different approaches for restricting search results based on the user’s identity. Indexing permission metadata and enforcing it at query time ensures that the access boundary is not left to the model that generates the answer. Some built-in features are in preview; the methods supported by your infrastructure should be checked separately. Microsoft: document-level access control.

As a design check, try the same question with two different user roles. Does the title, summary or excerpt of a document one of them can see leak into the other’s answer? Shared caches and conversation history must respect the same access boundary.

Set up a document lifecycle for freshness

• Add a content owner, version, validity date and source address to every document.
• Distinguish old versions from the version in force during retrieval.
• Define when a deleted document, or one whose access has been revoked, is removed from the index and the related caches.
• Make indexing errors visible not only to the technical team but also to the content owner.

This checklist is an implementation recommendation. It is not a retention period or legal compliance decision; it should be considered together with the organization’s existing data policies.

Include five distinct cases in the first test set

1. Direct answer: The information is stated clearly in a single document.
2. Synthesis: The answer requires information from two permitted documents.
3. Conflict: The old and new procedures say different things.
4. Unanswerable question: The information does not exist in the organization’s sources.
5. Access boundary: The relevant document exists, but the user cannot access it.

Evaluate separately whether retrieval brought back the right document, how faithful the answer is to the source, and how accurate the citations are. That way, when the problem lies in the retrieval layer, you won’t waste time just changing the prompt. Microsoft also treats retrieval and response quality as separate components in RAG evaluation. RAG evaluation components.

Start with a narrow scope

For the first pilot, choose a document set with a clear owner that is updated regularly. Judge success not only by the number of answers, but by whether users reach the right source and whether unanswerable cases are handled correctly. This preparation is shared work for the search, identity and content management teams in an enterprise product development process.