Model Selection and Optimization

Question 1 · 5.1

A document-review tool is tested with temperature set to 0.0 so results can be compared between builds. The test suite asserts that a given contract always produces one exact summary string. It passes locally and fails intermittently in CI.

What is the underlying problem?

• AThe CI environment is sending a different system prompt to the model.
• BOutput is never fully deterministic, even at temperature 0.0.
• Ctemperature must be paired with top_p set to 0.0 for determinism.
• DThe request needs a fixed random seed so generation can be repeated.

Question 2 · 5.1

A chat assistant works well for short sessions. After about forty turns, requests begin failing with a 400 error whose message says the prompt is too long. The user's messages are short throughout.

Which two changes address the cause? (Select 2.)

• ASummarise or drop older turns before sending the request.
• BAttach only the tool definitions the session actually needs.
• CLower max_tokens so the response fits alongside the history.
• DRetry the request with exponential backoff.
• EEnable prompt caching for the system prompt and tool definitions.
• FSwitch to a lower effort level to shorten the reasoning.

Question 3 · 5.1

A team migrates a summarisation service to a model that supports adaptive thinking. Their code reads response.content[0].thinking before formatting the answer. After the migration, some requests raise an attribute error.

What explains the failures?

• AAdaptive thinking returns its reasoning in the usage object instead of a content block.
• BThe budget_tokens value is too low, so the thinking block is dropped.
• CUnder adaptive thinking Claude may skip thinking entirely, so no thinking block is returned.
• DThinking blocks from previous turns were stripped, removing the current one.

Question 4 · 5.1

A retailer classifies product reviews into five sentiment-and-topic buckets. The prompt names the five buckets and describes each in a sentence. Output quality is inconsistent and new bucket names keep appearing.

What is the most effective next step?

• AAdd three to five diverse worked examples in an <examples> block.
• BAdd one carefully chosen example covering the most common review type.
• CRewrite each bucket description at greater length and detail.
• DLower the temperature so the model picks the likeliest label.

Question 5 · 5.2

A service already uses the official Python SDK with default settings. To “make it more resilient”, a developer adds a decorator that retries any exception up to three times with a fixed one-second delay. During a brief rate-limit event the service saturates its own worker pool and the incident lasts far longer than the limit did.

What is the best correction?

• AKeep the decorator but raise the delay between attempts to ten seconds.
• BRemove the decorator and set max_retries on the client instead.
• CDisable the SDK's own retries and keep the custom decorator.
• DCatch only RateLimitError in the decorator and retry that.

Question 6 · 5.2

A chat UI streams responses. When Claude calls a tool, the front end sometimes throws a JSON parse error and drops the turn. The logs show it parsing the contents of each content_block_delta as it arrives.

What is the correct handling?

• AAccumulate input_json_delta fragments until content_block_stop, then parse once.
• BRequest the tool call without streaming, then stream only the final text.
• CWrap each parse in a try/except and ignore the failures.
• DUse text_delta events to reconstruct the tool arguments.

Question 7 · 5.2

An architecture review asks how a new voice assistant should connect. The browser holds a live microphone session; the backend calls Claude for each utterance.

Which description is accurate?

• AThe backend should open a websocket to the Claude API for lower latency.
• BA websocket between browser and backend is normal; the backend calls Claude over HTTP and may stream with SSE.
• CServer-sent events are two-way, so one SSE connection can carry the audio up and the reply down.
• DStreaming should be avoided because partial output cannot be validated.

Question 8 · 5.3

A live-chat widget must acknowledge a customer message within one second. The task is classifying the message into one of eight intents. A trial on the most capable available model is accurate but takes three to four seconds per reply.

What is the best change?

• AKeep the model and stream the response so text appears sooner.
• BMove the classification to the fastest model and validate it against an eval set.
• CCache the model's answers so repeat messages return instantly.
• DRaise effort so the model reaches its answer in fewer tokens.

Question 9 · 5.3

After upgrading to a newer model, a service that extracts fields from invoices starts returning empty values for every document. The API returns HTTP 200, the token usage looks normal, and the logs show no errors. The extraction code reads response.content[0].text.

What is the cause?

• AThe new model returns structured output that must be read from usage.
• Bmax_tokens is now consumed by thinking, so no text is generated.
• CThe first content block is now a thinking block, so position zero holds no text.
• DThe model id was pinned to a version that no longer exists.

Question 10 · 5.3

A team is preparing to move a production agent to the next model release. They have an eval suite, prompts tuned over six months, and a hard-coded model id in three services.

Which two steps should come first? (Select 2.)

• AMove the model id into configuration in all three services.
• BRead the migration guide and remove parameters the new model rejects.
• CSwitch production to the new model and monitor error rates.
• DDelete the eval suite, since it was tuned against the old model.
• ERaise max_tokens everywhere to be safe.
• FRewrite every prompt before testing the new model.

Question 11 · 5.4

A documentation assistant sends a 30,000-token product manual in the system prompt, followed by a block containing the current date, which carries cache_control. The user's question is in messages. Cache creation tokens are high on every request and cache read tokens are always zero.

What explains this, and what fixes it?

• AThe manual is larger than the cache limit; split it across two breakpoints.
• BThe breakpoint sits after content that changes daily; move it onto the manual.
• CThe five-minute lifetime expires between requests; request the one-hour lifetime.
• DCaching requires the content to be in messages rather than system.

Question 12 · 5.4

After enabling prompt caching, a dashboard that plots input_tokens per request shows a dramatic drop, and the team reports a 95% cost saving to their manager. The monthly invoice falls by rather less.

What is wrong with the measurement?

• ACached tokens are free, so the invoice should have fallen by the full amount.
• Binput_tokens excludes cached tokens, so the dashboard is not measuring total input.
• CThe invoice lags usage by several days, so the saving has not appeared yet.
• DOutput tokens rose to compensate for the shorter input.

Question 13 · 5.4

A finance team asks for a monthly report of spend per product team. Each team has its own API key, and all of them share one workspace. An engineer proposes summing the usage fields their services already log.

Which two approaches will answer the question? (Select 2.)

• AQuery the Usage API grouped by API key, using an Admin API key.
• BAggregate the logged usage fields per service and price them per model.
• CQuery the Cost API grouped by API key for money rather than tokens.
• DUse a workspace API key to call the Usage API for each team.
• ERead the per-request usage object from the Console after the fact.
• FPoll the Cost API every minute and attribute spend by arrival time.

Question 14 · 5.4

An agent holds long conversations with a stable system prompt, a fixed set of tools and a history that grows by roughly 800 tokens per turn. Today it sets a single breakpoint at the end of the system prompt.

What change most reduces cost over a twenty-turn conversation?

• AAdvance a second breakpoint to the end of the previous turn on each request.
• BRaise the cache lifetime to one hour on the system prompt breakpoint.
• CSummarise the conversation into the system prompt every few turns.
• DMove the tool definitions after the system prompt in the request.