Cost and token management

Token budgeting starts before the call

Budgeting means deciding what a request is allowed to cost before you send it, rather than discovering it afterwards. The API gives you a free token-counting endpoint for exactly this: client.messages.count_tokens(...) accepts the same model, system, tools and messages you were about to send and returns input_tokens. It is free, separately rate-limited from message creation, and includes system prompts, tools, images and PDFs. The documentation is careful to call it an estimate — close enough to budget with, not a billing oracle.

# Free, and rate-limited separately from messages.create.
count = client.messages.count_tokens(
    model=MODEL, system=SYSTEM, tools=TOOLS, messages=messages,
)

if count.input_tokens > INPUT_BUDGET:          # your own ceiling, not the API's
    messages = compact(messages)               # summarise or drop older turns

resp = client.messages.create(
    model=MODEL, max_tokens=2000,              # bounds output, and so the worst case
    system=SYSTEM, tools=TOOLS, messages=messages,
)

u = resp.usage
total_input = u.input_tokens + u.cache_read_input_tokens + u.cache_creation_input_tokens
log.info("spend", total_input=total_input, output=u.output_tokens)

max_tokens is the other half of the budget, and the more important one, because output costs several times what input costs on every model. It caps the worst case of a single call. Where thinking is enabled it is a shared ceiling — reasoning tokens are output tokens — so a generous thinking budget inside a tight max_tokens buys reasoning at the expense of the answer.

Usage tracking: per request, and across the organisation

Every response carries a usage object, and it is the only honest per-request record of what you spent. Four fields do the work: input_tokens, output_tokens, cache_creation_input_tokens and cache_read_input_tokens. One detail catches everyone out — once caching is in play, input_tokens counts only the portion after your last cache breakpoint. Total input is the sum of all three input fields, so a monitor that plots input_tokens alone will appear to show costs collapsing at the moment caching starts working.

The organisation-level endpoints are /v1/organizations/usage_report/messages and /v1/organizations/cost_report. Both need an Admin API key — a workspace key will not do — and they answer different questions. The usage report gives token counts at minute, hour or day granularity and can be grouped by model, workspace, API key or service tier; the cost report gives money, daily only. Data appears within a few minutes of a request completing, so these are for reporting and alerting, not for enforcing a per-request limit.

A practical consequence: separate API keys and workspaces per environment and per major workload are a cost-attribution tool as much as a security one. Grouping the usage report by key only tells you something if your keys mean something. Key and workspace hygiene itself is 7.4.

Prompt caching, and where the breakpoint goes

Prompt caching stores a prefix of your request so later requests that begin identically can re-read it instead of reprocessing it. You mark the end of the cacheable prefix with a breakpoint: cache_control with type ephemeral on the last block that stays the same from request to request. Up to four explicit breakpoints are allowed per request.

The order the API assembles a request in is fixed — tools, then system, then messages — and caching follows that order. A change at any level invalidates that level and everything after it: edit a tool definition and the system prompt and messages caches go with it. This is why the stable material belongs at the front and the volatile material at the back, which is a prompt-structuring decision more than a caching one.

Three numbers shape whether caching is worth it. There is a minimum cacheable length that varies by model — from a few hundred tokens on the largest models to a few thousand on the fastest ones — below which nothing is cached at all. A cache write costs slightly more than ordinary input, and more again for the longer time-to-live. A cache read costs a small fraction of base input price. So caching pays as soon as a prefix is reused even a couple of times, and costs you a little if it is never reused.

system=[
 {"text": POLICY_10K_TOKENS},
 {"text": f"Today is {today}",
  "cache_control":
    {"type": "ephemeral"}}
]
# Breakpoint is after a line
# that changes daily, so the
# prefix hash is new each day
# and the 10k policy is
# rewritten, never read.

system=[
 {"text": POLICY_10K_TOKENS,
  "cache_control":
    {"type": "ephemeral"}}
]
messages=[
 {"role": "user",
  "content": f"Today is {today}.
    {question}"}
]
# Stable prefix cached;
# the volatile line moved
# behind the breakpoint.

Cache checkpointing across a conversation

A conversation is the awkward case, because it grows. Each turn appends the assistant's reply and a new user message, so a prefix that ended at the last turn is no longer the whole of the stable part. The technique the objective calls cache check-pointing is to move the breakpoint forward each turn: cache up to the end of the previous turn, so the next request reads everything up to there and writes only what is new.

With four breakpoints available, a typical agent uses them as a hierarchy rather than all at once: one after the tool definitions, one after the system instructions, one after any long retrieved context, and one moving forward through the conversation. They then invalidate independently and at the rate each one actually changes.

Traps the wrong answers are built from

You should now be able to

• Budget a request with the token-counting endpoint and an explicit max_tokens.
• Read every usage field correctly, including what input_tokens excludes when caching is on.
• Build a cost model from measured usage and current per-token prices.
• Place cache breakpoints on the last stable block, and order tools, system and messages to suit.
• Apply cache check-pointing across conversation turns and explain what invalidates a cache.
• Choose between per-request tracking and the organisation Usage and Cost APIs for a given question.