Giving an AI system documents to search can improve access to current evidence, but it does not transform generated sentences into verified facts.
Separate retrieval from interpretation
A retrieval system first selects likely relevant passages and then a generator composes an answer. Failure can occur at either stage. The right page may never enter the retrieval set, an old policy may outrank a new one, or the model may misunderstand a qualification. A quoted passage helps auditing only when it is actually the passage used to support the claim.
Sources are also an attack surface
Documents, web pages and email messages are lower-trust data. They may contain instructions such as “ignore the user” or “send this file elsewhere.” Those sentences are content to examine, not authorized instructions to execute. OWASP treats prompt injection and vector/embedding weaknesses as important application risks. A system with retrieval but no privilege isolation may combine factual error with a security breach.
Make evidence visible
Return precise passages or section labels, effective dates and the approved document title. If the retriever surfaces conflicting documents, explain the disagreement instead of blending them. Prefer versioned, controlled sources for HR, finance and legal policies. Evaluate whether an answer is supported by the cited passages, not merely whether the topic is related.
Practical mitigation sequence
Control which sources may enter the index, label their owners and versions, quarantine unknown attachments, test retrieval recall, and set a refusal or escalation path for insufficient evidence. Separate instruction channels from retrieved content. Keep tool permissions outside the language model, and require approvals for irreversible actions. Ownership of indexing and orchestration details belongs with the agents team; these are cross-cutting safety criteria.
A simple test
Ask a policy assistant about a reimbursement cap changed last month. Include both old and new versions in a controlled test set. Check whether the system picks the governing revision, flags the superseded policy and cites the exact date. Add a malicious paragraph in a test document and verify that no forbidden operation occurs.
Limits
Even a passed test does not prove that future documents are safe. Reassess after changes to embedding models, document permissions, source corpus, prompt templates or retrieval ranking.
Sources & Further Reading
Guidance varies by jurisdiction and application. This is general education, not legal, medical or other professional advice.