CCAR-P 3.8 — Progressive discovery vs. monolithic context

Two strategies for the same agent

Monolithic

Every tool schema, manual and rule sent on each request
Nothing to discover, so nothing is missed
One stable prefix that caches well
Cost, latency and distraction grow with every addition

Progressive discovery

Small always-on core plus names and descriptions
Full content loaded when the task needs it
Scales to thousands of tools or documents
Depends on good descriptions; adds lookup steps

Monolithic context puts everything in front of the model on every request. Progressive discovery keeps a small core plus an index, and loads the detail only when the task calls for it.

Why “just load everything” stops working

Anthropic’s context-engineering post treats context as a finite attention budget. It names the effect context rot: as the number of tokens grows, the model’s ability to recall any one piece of information from them falls. Every token attends to every other token, so relationships grow with the square of the context length, and attention is spread thinner. The post’s guiding principle is to find the smallest set of high-signal tokens that makes the outcome you want most likely.

Tools show the effect most sharply. The tool search documentation says Claude’s tool-selection accuracy degrades once more than about 30–50 tools are available, and that a typical multi-server setup costs around 55,000 tokens in definitions before any work starts. Instructions behave the same way. Claude Code’s docs warn that a bloated CLAUDE.md makes Claude ignore instructions, and suggest keeping it under 200 lines. A bigger context window raises the ceiling. It doesn’t change the fact that more material means more noise around the part that matters.

When monolithic context is the right call

Monolithic is not a mistake in itself. It is the right strategy when the material is small, stable and needed on almost every turn. It is also right when the task needs the whole of something at once, such as checking a 40-page agreement for internal consistency, where any piece left unloaded is a piece the model can’t compare. And it is right when latency matters more than tokens: every discovery step is an extra model decision and often an extra round trip.

Caching strengthens the case. A long, unchanging prefix is exactly what prompt caching rewards: cache reads are billed at a tenth of the base input price, with a five-minute default lifetime. Caching is strict about order, though. It runs tools → system → messages, and changing tool definitions invalidates everything after them. A design that swaps tool sets in and out of the tools array on each request can throw away its cache every time. The tool search docs note that deferred tools are kept out of the cached prefix, so discovery and caching can work together.

Signal in the scenario	Leans monolithic	Leans progressive
Size of the material	A few thousand tokens	Tens of thousands and growing
How often each part is used	Nearly every request	A long tail, rarely used
Kind of reasoning	Needs the whole at once	Needs one relevant piece
Rate of change	Stable, cacheable	Frequent additions and edits
Latency budget	Tight; no room for lookups	Tolerates an extra step
Must it always apply?	Yes: safety, compliance rules	No: reference, how-tos

How progressive discovery works in Claude’s tools

Progressive discovery has the same shape everywhere: a thin, always-loaded index (names and descriptions) and full content that loads only when chosen. The context-engineering post describes it as agents keeping lightweight identifiers such as file paths, queries and links, and loading data at runtime. Each step reveals context that informs the next decision.

Levels of disclosure, from always-on to never-loaded

Always-on coresystem prompt, CLAUDE.md, hard rules
Indexskill descriptions, tool names, file paths
On-demand bodySKILL.md body, deferred tool schema
Deep referencelinked files read only in specific cases
Executed, not readscripts run; only output returns

Each level down costs nothing until the task reaches it. The top level is the only one the model is guaranteed to read.

Agent Skills are the clearest example. Anthropic’s Skills post describes the levels: a skill’s name and description sit in the system prompt from the start; the SKILL.md body loads when Claude judges the skill relevant; further linked files load only when a specific case needs them (the PDF skill keeps form-filling guidance in a separate forms.md); and bundled scripts run without their code entering context. The post concludes that the amount of context a skill can bundle is effectively unbounded.

Tool search does the same for tools. You send the full catalogue on every request, but mark most tools defer_loading: true. Only the non-deferred tools and the search tool enter Claude’s context. When Claude needs something else it searches, and the API expands matches into full definitions. The docs recommend it once you pass about ten tools or 10,000 tokens of definitions, and advise keeping the three to five most-used tools loaded. They advise against it when every tool is used on every request. Deciding which tools belong in the catalogue at all is 3.1.

A large catalogue with a small loaded corepython

tools = [
    # The search tool itself is never deferred.
    {"type": "tool_search_tool_bm25_20251119", "name": "tool_search_tool_bm25"},

    # Hot path: used on most requests, so always loaded.
    {"name": "get_account", "description": "...", "input_schema": {...}},
    {"name": "search_kb",   "description": "...", "input_schema": {...}},

    # Long tail: sent every time, but only loaded if Claude finds it.
    *[{**t, "defer_loading": True} for t in rare_tools],   # e.g. 140 tools
]

response = client.messages.create(
    model=MODEL, max_tokens=2048, tools=tools, messages=messages,
)
# Discovered tools arrive as tool_reference blocks the API expands itself.
# Pass the assistant content back unchanged on the next request.

Code execution pushes the idea furthest. Anthropic’s code-execution-with-MCP post presents MCP servers to the agent as files of code. The agent lists and reads only the tool files it needs, then filters large results in code before anything returns to the model. Its example drops from about 150,000 tokens to about 2,000. The price is a secure sandbox to run the code in.

Worked example · A telecom’s field-engineering assistant

A telecom gives engineers an assistant with 160 tools across eight MCP servers and 70 equipment manuals pasted into the system prompt. Every request is slow and expensive, and engineers report it often picks a tool from the wrong vendor’s system.

Measure the always-on load. Tool definitions and manuals together take most of each request before the engineer’s question appears. Tool choice is being made among 160 look-alike options.
Keep a small core. The safety rules for live-network work (lock-out steps, change-window checks) stay in the system prompt. They must apply on every turn, so they can’t depend on a lookup.
Defer the tool long tail. The five tools used on most jobs stay loaded; the rest are marked defer_loading: true behind tool search. Descriptions are rewritten to name the vendor and equipment family, because search matches on names and descriptions.
Turn manuals into skills. Each equipment family becomes a skill with a precise description (“Nokia 7750 router: alarms, card swaps, config backups”). Its body loads when a job involves that equipment, and deep procedures sit in linked files.
Re-evaluate. Compare tool-selection accuracy, tokens per request and latency against the old build on the same set of real job tickets.

Takeaway: Progressive discovery fixed cost and tool confusion together. The one thing deliberately left monolithic was the set of rules that must never be skipped.

Discovery is only as good as the index

Moving content behind a lookup creates a new failure mode: the model never looks. Claude Code’s docs say it plainly for skills: Claude matches the task against skill descriptions, and vague or overlapping descriptions lead it to load the wrong skill or miss one that would help. Tool search matches on tool names, descriptions and argument names, so a deferred tool called proc_v2 with a one-line description is effectively invisible. The context-engineering post adds two more costs. Runtime exploration is slower than retrieving pre-computed data, and without good tools and heuristics an agent can waste context chasing dead ends.

Placing one piece of context

Where should this piece of context live?

Must apply on every turn
Always-on coreor enforce with a hook
Small and used on most turns
Load up frontand cache the prefix
Large, needed occasionally
Discover on demandskill, deferred tool, file
Bulky data to process
Execute, don’t readcode filters it first

Hybrid is the usual answer

In practice the choice is rarely all one or all the other. The context-engineering post uses Claude Code as its model of a hybrid: CLAUDE.md files go into context up front, while tools such as glob and grep fetch files just in time. It suggests the hybrid suits work with less dynamic content, naming legal and finance. Claude Code’s own feature table shows the same layering, and it is a useful template for any agent you design.

Claude Code feature	Loads	Context cost
CLAUDE.md	Session start, in full	Every request
`.claude/rules/` with `paths`	When matching files are opened	Only when relevant
Skills	Descriptions at start, body when used	Low until used
MCP servers	Tool names at start, schemas on demand	Low until a tool is used
Subagents	Fresh context when spawned	Isolated; only a summary returns
Hooks	On their event, outside the model	Zero unless they return output

Worked example · An accounting firm keeps its style guide up front

An accounting firm’s drafting assistant writes client audit letters. It uses a 6,000-token house style and disclosure checklist on every letter, and 60 technical standards, each tens of thousands of tokens, that any one letter touches two or three of. A consultant proposes making everything on-demand “for consistency”.

Check usage. The style and checklist apply to every letter, every time. The standards are a long tail.
Keep the checklist monolithic. It is small, stable and always needed. It goes in the system prompt behind a cache breakpoint, so repeat requests read it at a tenth of the input price.
Make the standards discoverable. Each standard becomes a skill or retrievable document with a description naming its scope. The assistant loads the two or three a letter needs.
Reject “everything on demand”. Hiding the checklist behind a lookup would save little context, lose the cache benefit, and risk letters drafted without it.

Takeaway: The strategy is chosen per piece of context, not per system. Small and always needed goes up front; large and occasional is discovered.

Traps the wrong answers are built from

Tempting but wrong	Do this instead
Loading every tool, manual and policy because the context window can hold them	Keep a small always-on core and discover the long tail through skills, tool search or file access.
Putting must-follow rules behind a skill or lookup to save tokens	Keep them in the always-on core, and enforce critical ones outside the model with hooks or permissions.
Deferring tools or skills that are used on nearly every request	Keep the few hot-path tools loaded; defer only the rarely used ones.
Writing vague names and descriptions for deferred tools and skills	Write descriptions that say what, when and for which system, since discovery matches on them.
Swapping tool definitions in and out of the tools array on every request	Use deferred loading so the cached prefix stays stable.

You should now be able to

Explain context rot and why a larger window does not remove the cost of loading everything.
Identify when monolithic context is the right choice: small, stable, always needed, whole-document reasoning, tight latency.
Apply progressive discovery with skills, tool search, file references and code execution.
Keep must-follow rules in always-on context or enforce them outside the model.
Design descriptions and metadata so discoverable content is actually found.
Place each piece of context in a hybrid design, and account for prompt caching.

Progressive discovery vs. monolithic context