Context Management & Reliability

Question 1 · 5.1

An insurance claims assistant handles conversations that often exceed 80 turns. Late in long conversations it sometimes quotes the wrong policy number or claim amount, even though both were stated clearly near the start. The team uses automatic summarisation when context gets large.

Which change most directly fixes the errors?

• AMove to a model with a larger context window so summarisation happens less often.
• BAdd “always remember policy numbers and amounts” to the system prompt.
• CExtract key facts into a structured block sent with every request, outside summarised history.
• DSummarise more often so each summary covers fewer turns and is more accurate.

Question 2 · 5.1

A research agent calls a search tool and a document-fetch tool dozens of times per task. Each fetch returns an entire web page. By mid-task, answers are slower and the agent starts contradicting findings it made earlier.

Which two changes best address this? (Select 2.)

• ATrim fetched pages to the relevant passages, with their source, before appending them.
• BEnable tool result clearing so older fetch results are replaced once a threshold is passed.
• CRaise max_tokens so the agent has room to reconcile the earlier findings.
• DMove the full search results into the system prompt so they are never cleared.
• EInstruct the agent to reread all earlier results before each answer.

Question 3 · 5.1

You set instructions on the server-side compaction edit to “Keep it short.” What is the likely effect?

• AThe default summary prompt still runs, and your text is added to it as an extra hint.
• BYour text replaces the default prompt, so the summary may drop state and next steps.
• CThe API rejects the request because instructions must be at least 50,000 tokens.
• DNothing changes until you also set pause_after_compaction to true.

Question 4 · 5.1

A team keeps a rule “never edit generated files under src/gen/” as a path-scoped rule in Claude Code. After long sessions compact, Claude occasionally edits those files.

What is the best fix?

• ARepeat the rule in each prompt the engineers type.
• BDisable automatic compaction so the rule is never summarised away.
• CMove the rule to the project-root CLAUDE.md, which is reloaded after compaction.
• DRun /clear whenever the session gets long so the rule reloads.

Question 5 · 5.2

A telecom support agent escalates to humans whenever a sentiment classifier scores the customer as “negative”. Human agents report that most escalated cases are simple billing corrections the bot could have fixed, while several complex contract disputes from polite customers were never escalated.

What is the best redesign?

• ALower the sentiment threshold so fewer conversations count as negative.
• BEscalate on explicit requests, authority limits and policy gaps, with examples in the prompt.
• CHave the model rate its confidence from 1 to 10 and escalate below 7.
• DEscalate every conversation that mentions a contract or a bill.

Question 6 · 5.2

A bank’s support agent looks up a customer by name and date of birth and receives two matching accounts. The customer wants to cancel a standing order.

What should the agent do next?

• AChoose the account with the most recent activity, since it is the likelier one.
• BEscalate to a human because the lookup was ambiguous.
• CCancel the standing order on both accounts to be safe.
• DAsk for an additional identifier, such as the last digits of the account number.

Question 7 · 5.2

A research subagent, spawned by a coordinator in the Agent SDK, finds that the user’s request “compare our Q3 numbers” could refer to two different fiscal calendars. The team wants the user to choose.

What is the most appropriate design?

• AThe subagent calls AskUserQuestion directly so the user can pick the calendar.
• BThe subagent reports the ambiguity to the coordinator, which asks the user.
• CThe subagent picks the calendar used in the most recent report and continues.
• DThe subagent stops and returns an empty result so the coordinator retries it.

Question 8 · 5.3

In a multi-agent research system, the web-search subagent wraps its search call in a try/except and returns an empty list of findings on any exception. Users report that final reports sometimes skip entire subtopics without explanation.

What is the most effective change?

• AHave the coordinator re-run any subagent that returns an empty list.
• BRaise the exception so the whole research task stops and the user is alerted.
• CReturn a failure status with error type, attempts and partial results instead of [].
• DAdd “always find at least one result” to the subagent’s prompt.

Question 9 · 5.3

Which two errors should a subagent generally propagate to its coordinator immediately, rather than retrying itself? (Select 2.)

• AThe search API returns HTTP 429, rate limit exceeded.
• BThe document store denies access to the folder the subagent was told to read.
• CA date parameter was sent as “31/12/2026” but the tool expects ISO 8601.
• DA refund tool rejects a $5,000 refund because it exceeds the policy limit.
• EThe network connection to the database resets once mid-query.

Question 10 · 5.3

A coordinator receives a partial result from one of five subagents: a regulatory database was unavailable, so one jurisdiction is not covered. The other four jurisdictions are complete, and the report is due within the hour.

What should the coordinator do?

• ADeliver the report with the gap stated, and reroute if another source can cover it in time.
• BDiscard all results and start the whole research task again.
• CDeliver the report covering four jurisdictions without mentioning the fifth.
• DHave the synthesis agent infer the fifth from the other four.

Question 11 · 5.4

An engineer spends two hours in one Claude Code session exploring an unfamiliar logistics codebase. Early on Claude identified the exact classes that calculate shipping rates. Now its answers describe “how rate engines typically work” and contradict details it found earlier.

What is the most effective way to continue?

• AAsk Claude to reread the whole shipping/ directory to refresh its memory.
• BSave key findings to a notes file, then /compact or start fresh from it.
• CSwitch to a model with a larger context window and keep going.
• DAdd “do not generalise; use specific class names” to CLAUDE.md.

Question 12 · 5.4

Before changing a database column, a team needs to know every service in a 20-service monorepo that reads or writes it. The main Claude Code session will then plan the migration.

Which approach best manages context?

• ARead each service’s data-access layer in the main session, one after another.
• BPaste the full schema and all model files into the first prompt.
• CAsk Claude to recall from general knowledge which services usually touch such columns.
• DDelegate the search to subagents that grep for the column and return paths and usage.

Question 13 · 5.4

Which statement about Claude Code compaction is accurate?

• AEverything read during the session is kept verbatim after compaction.
• BCompaction deletes CLAUDE.md, so project rules must be repeated afterwards.
• CThe project-root CLAUDE.md reloads, but individual findings exist only in the summary.
• DCompaction only runs when you type /compact; it never happens automatically.

Question 14 · 5.5

An insurer’s claims-extraction pipeline shows 97% field-level accuracy on last month’s labelled sample. The operations lead proposes cutting human review from 100% of claims to a 2% random spot-check.

What should be done before approving the change?

• AApprove it, because 97% exceeds the 95% target the team agreed.
• BCheck accuracy by claim type and field; keep review where it falls short.
• CAsk the model to report confidence per claim and review anything below 0.8.
• DRaise the random spot-check to 5% to add a margin of safety.

Question 15 · 5.5

A document pipeline auto-approves fields when the model’s stated confidence is at least 0.9. An audit finds that 20% of auto-approved dates are wrong, even though the model reported 0.92–0.98 confidence on them.

What is the best next step?

• ARaise the auto-approve threshold to 0.99 for every field type.
• BAdd “be honest and careful about your confidence” to the prompt.
• CMeasure accuracy by score band per field, and set a date-specific threshold.
• DRemove automation for all fields until the model improves.

Question 16 · 5.5

Which two signals are generally more reliable for routing extractions to human review than a self-reported confidence score on its own? (Select 2.)

• AThe extracted line items fail to sum to the extracted total.
• BThe model wrote a longer explanation than usual.
• CTwo independent extraction passes return different values for the field.
• DThe document arrived outside business hours.
• EThe model used hedging words such as “likely” in its reasoning.

Question 17 · 5.5

A team samples 3% of auto-approved outputs at random each week. Handwritten forms are 4% of volume, and a new scanner vendor has recently started producing blurrier images of them.

What change to the audit makes a new failure on handwritten forms most likely to be caught quickly?

• ADouble the random sample rate from 3% to 6% of all items.
• BStratify the sample so every document type gets enough reviews.
• COnly audit items with confidence just above the threshold.
• DStop auditing and rely on complaints from customers instead.

Question 18 · 5.6

A multi-agent research system’s search subagents return findings as paragraphs of prose. The synthesis agent’s reports read well, but reviewers often cannot tell which source supports a given sentence, and some sentences blend claims from two sources.

What is the most effective fix?

• ATell the synthesis agent to add a references list at the end of each report.
• BHave subagents return claim records with source, location, quote and date for each finding.
• CIncrease the synthesis agent’s context so it can read the full source documents again.
• DAdd a final agent that guesses the most likely source for each sentence.

Question 19 · 5.6

Two reputable sources give different figures for the same market: one says 12% growth, the other 8%. The synthesis agent currently reports “about 10% growth”.

How should the synthesis handle this?

• AKeep the average, since it balances the two reputable sources fairly.
• BUse the more recently published figure and drop the other one.
• COmit the growth figure entirely because the sources disagree.
• DReport both figures with their sources, dates and definitions, and flag the conflict.

Question 20 · 5.6

A legal research tool passes contract PDFs to Claude and needs every statement in the answer to point to the exact page it came from. The team also wants the answer returned as strict JSON matching a schema.

Which design meets both needs?

• AEnable Citations on the PDFs and structured outputs on the same request.
• BPut page, quote and document fields in the JSON schema, then verify each quote.
• CReturn free text and ask a second model to convert it to JSON with page numbers.
• DDrop the page requirement, since the document name alone is enough provenance.