Chunking determines what a retrieval system can see; reranking determines which retrieved snippets are likely to reach the agent. Both choices can change answer quality even when the same underlying language model is used. Treat them as versioned parts of a retrieval experiment, not invisible preprocessing.
How it works
Microsoft’s RAG documentation describes splitting large documents into smaller indexed pieces and using hybrid retrieval and optional semantic ranking at query time. A chunk is not automatically a complete fact: separating a policy exception from the rule can invert meaning. Preserve document ID, section path, effective date and exact location for each chunk. Reranking operates only on candidates returned by first-stage retrieval, so it cannot rescue a missing document that was never retrieved.
A practical, bounded workflow
For a technical manual, chunk at headings and semantic boundaries rather than blindly slicing every fixed number of characters. Attach the parent section to each piece so a troubleshooting step can be interpreted alongside its warnings. Retrieve a candidate set using identifiers and semantic similarity, then rerank only within the requested collection and access scope. Return page and section pointers to the agent instead of letting it fabricate references.
Failure modes and limits
Small chunks improve precision but can lose the context needed to understand exceptions. Large chunks preserve context but cost more to retrieve and may displace other evidence. Aggressive reranking can add latency and make a seemingly high-confidence top result look definitive even when contradictory documents exist. A zero-result query must produce an honest ‘not found’, not an invented citation.
Deployment controls
Filter retrieved records before model context assembly, not after generation. Each excerpt needs source provenance, data owner, corpus version and revocation status. A retrieved instruction is evidence to interpret, never a permission to change tool policy. Measure leakage and stale-corpus errors independently from linguistic fluency.
When to choose this approach
A basic keyword index may beat an elaborate RAG chain for part numbers and exact IDs. Consider reranking when a representative evaluation set shows meaningful relevance gains, and avoid assuming a provider’s benchmark applies to your corpus. Pricing and proprietary model limits belong in the separate dated AI 10 and AI 02 archives.
What to evaluate before production
Build an evaluation set with exact codes, paraphrased questions, negative examples, overlapping versions and policy exceptions. Report Recall@k, grounded-answer rate, false citation rate, p50/p95 latency and retrieval expense separately. Freeze the corpus version and review any per-tenant filtering before comparing chunking experiments.
Version, market and tier caution (25 October 2026): This is a technical and operational guide, not a live entitlement, pricing or geographic-availability certification. Numeric commercial limits and undocumented release dates are intentionally omitted. Reconfirm the exact account, deployment region, access policy and vendor release before procurement or publication.