Optimising tokens, latency and cost

Where the money goes in one request

A request is billed on four kinds of token, reported in the response's usage block: uncached input (input_tokens), tokens written to the cache (cache_creation_input_tokens), tokens read from the cache (cache_read_input_tokens) and output (output_tokens). The prompt-caching docs point out a trap: input_tokens counts only the tokens after the last cache breakpoint, so total input is the sum of all three input fields.

Two ratios drive most decisions. Output costs five times input on every model in the table, so a verbose answer is often the biggest line on the bill. And a cache read costs a tenth of a normal input token. Prices change — they're shown here to practise the arithmetic, and the pricing page is the source of truth. One more caution from that page: Claude 4.7 and later models use a newer tokenizer that produces roughly 30% more tokens for the same text, so compare models on measured cost per task, not on list price. The token-counting endpoint, which is free to call, counts a prompt under the tokenizer of the model you name.

Prompt caching: pay once for the part that repeats

Caching stores a processed prompt prefix so later requests that start with the same content read it back cheaply and faster. The prefix is built in a fixed order — tools, then system, then messages — and any change invalidates everything after it. The rules that matter for design:

• Stable content first, variable content last. Anything that changes per request — a timestamp, the user's name — must come after the breakpoint.
• Automatic or explicit. A single top-level cache_control lets the API place and advance the breakpoint as a conversation grows; explicit breakpoints (up to 4 per request) suit sections that change at different rates.
• Minimum length. Prompts shorter than the model's minimum aren't cached, and no error is returned — for example 1,024 tokens on Claude Sonnet 5 and 4,096 on Claude Haiku 4.5.
• Lifetime. The default cache lives 5 minutes; a 1-hour option costs more to write.

system:
  "Current time: 09:41:07.
   User: Priya (Gold tier).
   [5,800 tokens of policy,
    tone rules and examples]"
  cache_control ← here

messages: [question]

system:
  "[5,800 tokens of policy,
    tone rules and examples]"
  cache_control ← here

messages:
  "Current time: 09:41:07.
   User: Priya (Gold tier).
   [question]"

The break-even is quick. A 5-minute cache write costs 1.25× normal input and each read 0.1×, so the pricing page notes it pays off after one read; the 1-hour cache write costs 2× and pays off after two reads.

IN, OUT, READ = 2.00, 10.00, 0.20          # USD per million tokens

def cost(prefix, fresh, out, cached=False, batch=False):
    prefix_rate = READ if cached else IN       # warm cache assumed
    usd = (prefix * prefix_rate + fresh * IN + out * OUT) / 1e6
    return usd * (0.5 if batch else 1.0)       # batch halves every rate

# 6,000-token stable prefix, 300 fresh tokens, 400 output tokens
print(cost(6000, 300, 400))                # 0.0166  baseline
print(cost(6000, 300, 400, cached=True))   # 0.0058  -65%
print(cost(6000, 300, 150, cached=True))   # 0.0033  shorter answers too

Look at the second line. Once the prefix is cached, the 400 output tokens ($0.004) are about 70% of the remaining cost. The next lever isn't more caching; it is answer length.

The other levers, and what each one costs you

Effort. Effort is set with output_config.effort and defaults to high. It affects all output — text, tool calls and thinking — and lower levels make fewer, terser tool calls. It isn't available on every model (Claude Haiku 4.5 is not on the supported list). Anthropic's guide to cost and intelligence goes further: sweep effort on your current model before trying multi-model set-ups, because it beats most of them.

Batch. The Message Batches API halves the price of every token and takes up to 100,000 requests or 256 MB per batch. Results arrive when the whole batch ends or after 24 hours, whichever is first; unfinished requests expire and aren't billed. Discounts stack with caching, but cache hits inside a batch are best-effort, and the docs suggest the 1-hour cache for shared context because batches can outlast 5 minutes.

Output length. Ask for brevity in the prompt; the latency guide notes that sentence or paragraph limits work better than word counts, because models count tokens, not words. max_tokens is a hard cap, not a length request: a response that hits it is cut off mid-sentence. On models with adaptive thinking, max_tokens also covers thinking, and thinking tokens bill as output even when the thinking text isn't returned.

Model routing. Anthropic's guide describes two multi-model patterns: a cheaper executor that escalates hard decisions to a stronger advisor, and a frontier orchestrator that fans bulk work out to cheaper workers. Its measured examples cut both ways — an orchestrator set-up cost about half as much on one benchmark and lost on cost on a harder one — so routing is a hypothesis to test on your own evals, not a guaranteed saving.

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,                         # a cap, not a length request
    output_config={"effort": "low"},         # routine question
    system=[{
        "type": "text",
        "text": POLICY_AND_EXAMPLES,         # long, identical every call
        "cache_control": {"type": "ephemeral"},
    }],
    messages=[{"role": "user", "content": f"{now()} | {question}"}],
)
u = response.usage
print(u.cache_read_input_tokens, u.input_tokens, u.output_tokens)

Traps the wrong answers are built from

You should now be able to

• Compute per-request and daily cost from the usage fields and the price list, including cache reads and batch discounts.
• Structure prompts so caching works, and explain its minimum length, lifetime and break-even.
• Choose between caching, batching, model choice, effort, output length, input trimming and streaming for a given workload.
• Explain the latency effect of each lever, including time to first token and streaming.
• Recognise when a saving costs quality, and require eval evidence before accepting it.