What you need to know
"Add RAG" is not a design. The data an assistant needs comes in different shapes, and questions come in different patterns. Each combination has a retrieval method that fits it and several that do not.
Match the method to the data shape
| Data shape | Examples | Retrieval method |
|---|---|---|
| Unstructured text | Manuals, policies, articles, notes | Chunked index: dense, sparse or hybrid (lesson 3.5) |
| Structured records | Orders, claims, accounts, transactions | A query or API call with filters; never embeddings over rows |
| Tables and metrics | Sales figures, KPIs | SQL against the warehouse (the model can draft the query; code runs it with read-only, scoped access) |
| Live state | Stock, status, balances | A tool call to the owning system at question time |
| Codebases and file trees | Repositories, shared drives | Agentic search: the model lists, greps and opens files step by step |
Many assistants need several of these, behind one router or exposed as several tools.
Match the method to the query pattern
| Query pattern | What helps |
|---|---|
| Paraphrased, conceptual ("why does my table wobble?") | Dense embeddings |
| Exact identifiers (codes, clause numbers, names) | Keyword or hybrid search |
| Scoped ("in the 2025 policy wording", "for this product") | Metadata filters applied before ranking |
| Multi-part or multi-hop ("compare the excess in the old and new wordings") | Query decomposition or an agent that retrieves in several steps |
| High stakes, needs evidence | Citations to the retrieved passages, and quote-first answers |
Improving precision
- Metadata filters (product, version, region, date, document type) remove whole classes of wrong answers before ranking.
- Contextual retrieval adds a short description of each chunk's place in its document before indexing, which reduces misses from context-poor chunks; combined with hybrid search it lowers retrieval failures substantially.
- Reranking takes the top few dozen candidates and reorders them with a stronger relevance model, keeping only the best handful. It improves quality and adds latency and cost per query, so it belongs where accuracy matters more than milliseconds.
- Query rewriting turns a conversational question into a better search query, using the conversation so far.
Agentic search versus a one-shot index
For codebases, file systems and very large or fast-changing collections, letting an agent search iteratively (list, search, open, follow references) can beat a precomputed index: it always sees the current files and can follow a trail. It is slower and costlier per question, so it suits investigation, not high-volume FAQ answering.
In the exam
These questions describe the data and how users ask, then offer several retrieval designs. Pick the method for each data shape, add filters when the question names a scope, use hybrid search when identifiers matter, add reranking where accuracy outweighs latency, and choose agentic search for codebases or investigative tasks. Distractors apply one method to everything, usually vector search.
Check your understanding
Analysts ask "what was our revenue by region last quarter?" against a sales data warehouse. Which retrieval approach fits?
Show answer
Answer: B. Aggregations over structured data are what queries are for. Similarity search over rows cannot compute a correct total.
An assistant keeps quoting terms from an outdated policy wording, although the question names the 2026 wording. What is the best fix?
Show answer
Answer: A. A scope named in the question should be a hard filter. Ranking and prompt instructions make the right version more likely; a filter makes the wrong one impossible.
Which TWO situations suit agentic, step-by-step search over a precomputed index?
Show answer
Answer: A, B. Agentic search shines when the path must be followed and the content changes. High-volume FAQs need a fast index, and a single order lookup is a direct tool call.
Apply it: Plan retrieval for Larkfield's adjusters
Larkfield wants an adjuster assistant that answers questions while a claim is being assessed. The sources are: policy wordings (structured PDFs with numbered clauses, three versions in force), the customer's policy record in Policy Admin, the customer's previous claims in ClaimsCore, and an internal guidance library of about 600 short articles on assessing different kinds of damage.
- Choose the retrieval method for each source, with the reason.
- Name the filters that must apply before ranking.
Why: quoting the wrong wording version would be a costly error. - Decide whether to rerank, given that adjusters are not in a real-time chat.
- Say what every answer must include so an adjuster can trust it.
Show model answer
1. Methods
| Source | Method | Reason |
|---|---|---|
| Policy wordings | Hybrid search over section-based chunks, with clause numbers kept | Adjusters ask both in plain words and by clause number |
| Customer's policy record | Policy Admin API call | Live, authoritative state |
| Previous claims | Structured query in ClaimsCore for this customer | Records, not documents |
| Guidance library | Dense or hybrid search over one chunk per article | Short, self-contained articles asked about in plain words |
2. Filters. The wording version recorded on the customer's policy (from Policy Admin), and the product line (home or motor). Guidance articles filtered by damage type when the claim type is known.
3. Reranking. Yes for policy wordings: adjusters can wait a second or two, and the cost of quoting the wrong clause is high. Not needed for the guidance library if recall is already good in evaluation.
4. Every answer includes the quoted clause with its number and wording version, links to any prior claims it relied on, and a reminder that the coverage decision is the adjuster's.