Prompt caching and retrieval caches reduce repeated computation, but they are not substitutes for authorization or data lifecycle rules. A tenant-scoped tool result should never be served to a different tenant merely because their queries have the same words.
How it works
Microsoft’s Azure OpenAI prompt-caching documentation describes reused prefix computations and distinguishes caching mechanics from application answer generation. Historical documentation notes that cache behavior depends on supported models and deployment. At application level, a separate document/result cache may hold full retrieved passages. These two caches have different trust and deletion implications; do not assume a model provider’s compute cache behaves like your application’s semantic-answer cache.
A practical, bounded workflow
Design cache keys with tenant and authorization context when the stored value contains application data: a keyed entry might include tenant ID, authorized corpus version, user-role fingerprint, document revision and extraction version. For purely public shared model prefixes, avoid injecting private content into globally reusable instructions. Apply TTL and invalidation on document deletion, permission change, policy update and source-revision replacement.
Failure modes and limits
The dangerous case is a technically correct cache hit on content that the current user may no longer see. A revoked customer document can remain in a model-generated summary cache after it disappears from the search index. Prefix-caching cost savings do not imply that intermediate context can be disclosed across users. Keep tool outputs and prompts from acting as authority to modify caching policy.
Deployment controls
Filter retrieved records before model context assembly, not after generation. Each excerpt needs source provenance, data owner, corpus version and revocation status. A retrieved instruction is evidence to interpret, never a permission to change tool policy. Measure leakage and stale-corpus errors independently from linguistic fluency.
When to choose this approach
Use provider caching for stable prompt scaffolding when its stated privacy conditions fit the workload; use application response caching only when you can model entitlements and invalidation correctly. Exact rates, retention times and supported endpoint IDs are dated product details and must be checked separately.
What to evaluate before production
Seed two tenants with the same document title and distinct private bodies. Warm all caches with tenant A, query tenant B and verify zero cross-tenant text or metadata leakage. Change the access control list, delete the source, rotate a role and rerun. Record the cache-layer invalidation path, not just the final language-model reply.
Version, market and tier caution (25 October 2026): This is a technical and operational guide, not a live entitlement, pricing or geographic-availability certification. Numeric commercial limits and undocumented release dates are intentionally omitted. Reconfirm the exact account, deployment region, access policy and vendor release before procurement or publication.