Matching retrieval to data and queries

Two questions: what shape is the data, and what shape is the query?

Data shape is about how the information is stored. Is it rows and columns with exact values? Records with a few fields and a text body, like tickets or emails? Long prose, like policies and contracts? Code and logs, where exact identifiers matter and files point to other files? Or is it a live system of record, where any copy starts going stale as soon as you make it?

Query pattern is about what the user is asking for. An exact lookup (“order A-1042”). A conceptual question (“what is our policy on remote work abroad?”). An aggregation (“how many refunds over £500 last quarter?”). A multi-hop or comparative question (“which of our suppliers’ contracts allow termination for convenience?”). An open-ended survey (“what are competitors saying about pricing?”). Each pattern breaks a different retrieval method.

Pre-computed retrieval versus just-in-time retrieval

Anthropic’s multi-agent research post contrasts traditional RAG’s “static retrieval” (fetch the chunks most similar to the input query, once) with multi-step search that adapts to what each result reveals. The context engineering post names the same idea just-in-time context. The agent keeps lightweight identifiers such as file paths, stored queries and links, and loads the data through tools only when it needs it. Claude Code works this way on a codebase: its built-in search tools find files by pattern and search content with regex, and it reads only the files it needs. The post also describes it using head and tail to inspect large data without loading all of it.

Neither wins everywhere. The context engineering post states the trade-off directly: runtime exploration is slower than retrieving pre-computed data. It suggests a hybrid often works best, as with Claude Code loading its CLAUDE.md up front and fetching everything else on demand. The research post shows the cost side too: agents typically use about 4× the tokens of a chat interaction, and multi-agent systems about 15×. A high-volume FAQ bot should not pay agentic prices for questions a single retrieval answers. An analyst assistant answering open-ended, multi-source questions often should.

Designing the retrieval tools themselves

Once Claude decides how to retrieve, the tools you expose shape what comes back. Anthropic’s tool-writing guidance says a tool that returns every record (its example is list_contacts) wastes context. Build a search_contacts that jumps to what matters instead. The same guidance favours tools that return only relevant log lines, response_format options for concise or detailed output, and pagination, filtering and truncation with sensible defaults. Consolidating which tools to expose at all is covered in 3.1.

tools = [
    {   # Prose: policies and procedures -> hybrid search, returns search_result blocks
        "name": "search_policies",
        "description": "Search HR and finance policy documents by meaning or exact "
                       "term. Use for 'what is our policy on...' questions.",
        "input_schema": {"type": "object", "properties": {
            "query": {"type": "string"},
            "effective_on": {"type": "string", "description": "ISO date; defaults to today"},
        }, "required": ["query"]},
    },
    {   # Rows: expense claims -> the database aggregates, Claude never sees raw rows
        "name": "expense_stats",
        "description": "Count or total expense claims with filters. Use for "
                       "'how many' or 'how much' questions. Returns numbers, not claims.",
        "input_schema": {"type": "object", "properties": {
            "metric": {"type": "string", "enum": ["count", "sum_gbp", "avg_gbp"]},
            "department": {"type": "string"},
            "from_date": {"type": "string"}, "to_date": {"type": "string"},
            "min_amount_gbp": {"type": "number"},
        }, "required": ["metric"]},
    },
]

Notice what the second tool does not do: it does not let the model write free-form SQL against production, and it does not return raw rows for Claude to add up. A narrow, parameterised tool keeps the arithmetic in the database, keeps the context small, and gives you a clear permission boundary. Some teams do expose read-only SQL to an analyst agent. That is a separate decision about access and guardrails, and 3.2 covers it.

Adapting to harder query patterns

Multi-hop and comparative questions need more than one retrieval. “Which suppliers can terminate for convenience with under 60 days’ notice?” needs the list of supplier contracts, then the termination clause in each, then a comparison. A single top-K search returns whichever termination clauses happen to be closest in meaning, and misses the rest. Give Claude a search tool and let it loop: search, read, refine the query, search again. The Building effective agents post describes this as the augmented LLM, where the model writes its own search queries.

For broad surveys, the research post’s guidance is to start with short, broad queries, see what is there, then narrow. It warns against long, over-specific queries that return little. When a question splits into independent parts, such as the same fact for twenty companies, parallel subagents each search one part and return a condensed result. That is where a multi-agent design earns its token cost. It is a poor fit when every part depends on shared context. And when query types are distinct and predictable, a routing step that classifies the question and sends it to the right retrieval path is simpler than a fully agentic search.

Traps the wrong answers are built from

You should now be able to

• Classify a data source by shape: structured, semi-structured, prose, code, or live.
• Classify a question by pattern: lookup, conceptual, aggregation, multi-hop, survey, or freshness-critical.
• Match each pairing to a retrieval method: hybrid search, structured query, agentic search, live tool call, or parallel subagents.
• Weigh pre-computed retrieval against just-in-time retrieval on latency, cost, freshness and question complexity.
• Design narrow, parameterised retrieval tools that return only what the model needs.