Tokens, requests and context

Language model APIs bill by the token: every request pays the input rate for all the tokens of the context it sends and the output rate for every token the model writes back. On the bill, cost therefore follows the length of each request in both directions, not the number of requests, and that length grows with history, retrieved documents and tool definitions while the question stays the same size.

How it works

A token is a fragment of text produced by the model's tokenizer: a word, part of a word, a punctuation mark or a leading space. The providers publish rough ratios for English. OpenAI gives about four characters, or three quarters of a word, per token; Google gives about four characters for Gemini, so 100 tokens are roughly 60 to 80 English words. Anthropic states that the count depends on the model's tokenizer and on the content; none of the three gives a ratio for other languages. A German or French prompt cannot be estimated from the English ratio; Anthropic and Google offer token counting that measures a request's input for a given model before it is sent.

Billing follows a price list with rates per million tokens, split by kind:

• Input tokens: everything the request sends, at the base input rate.
• Output tokens: everything the model generates, at a multiple of the input rate on all three price lists.
• Cached input tokens: a repeated beginning of the request, the prefix, can be read from a cache at a fraction of the input rate. Anthropic charges a premium to write to the cache, OpenAI enables caching by default for supported models and lists cache writes as a column of their own, and Google adds a storage fee per million tokens per hour.
• Reasoning or thinking tokens: tokens a model generates while thinking before it answers, billed as output. OpenAI documents that they are not visible through the API but occupy the context window; Google's output price includes them, and Anthropic reports in each response how many billed output tokens were thinking.
• Batch: asynchronous processing without an immediate answer, at lower rates on all three price lists.

The context window is the ceiling on what request and response may hold together. A model keeps nothing between requests: the system prompt, tool definitions, the conversation so far and retrieved documents arrive again every time, and all of it is billed again as input. Anthropic adds that newer models keep earlier turns' thinking blocks in context and bill them as input. Length can change the rate itself: OpenAI prices input above 272,000 tokens as long context, Google charges more for Gemini Pro prompts above 200,000 tokens, and Anthropic for Claude Haiku 5.5 prompts above 100,000.

Text is not the only metered unit. Google converts an image of up to 384 pixels a side to 258 tokens, larger images to 258 tokens per 768 × 768 tile, and audio to 32 tokens per second. Anthropic adds a tool use system prompt to every request that carries tools, 286 tokens on Claude Opus 5.5, and bills server-side tools such as web search per use on top of the tokens their results add to the context.

What it means for your cost

The cost of a request is input tokens times the input rate plus output tokens times the output rate, with cache reads, cache writes and batch tokens each at their own rate. Every lever works on one term:

• Context length: each paragraph of system prompt, turn of history and retrieved document is paid for on every request.
• Output length: output is the expensive rate and includes reasoning nobody reads; an output cap (max_tokens, max_output_tokens) bounds it.
• Caching: a stable prefix shared by many requests is written once and read cheaply afterwards.
• Batching: work that needs no answer now costs less as a batch.
• Model choice: rates differ widely between one provider's models, and the tokenizer can change with the model.

The providers' usage data reports these terms as separate counters. OpenAI reports cached tokens and cache writes within input and reasoning tokens within output; Google reports input, output, thinking, cached and tool-use tokens separately. A cost report that counts requests explains little, since two features with the same request count can cost very different amounts and a feature can grow more expensive without one extra request. Unit cost becomes meaningful per token type and per request, as the unit economics of a feature. Where those numbers come from, per project and per key, is covered in AI provider billing.

Where it goes wrong

• Context grows unnoticed. Cost per conversation rises faster than the number of messages, because every request resends the whole history and retrieval puts the same long passages into the context for every question, needed or not.
• Output has no ceiling. A few requests cost many times the rest, from long answers or heavy reasoning without a cap. OpenAI documents the opposite trap too: a cap reached during reasoning bills input and reasoning tokens with no visible answer.
• Caching is expected where the prefix changes. The cached share stays low because a hit needs the entire prefix to match exactly. A timestamp or user data at the start makes every request a miss, as does a changed thinking setting at Anthropic, and where writes carry a premium a missed cache costs more than none.
• A model upgrade brings a new tokenizer or rate. Cost rises with unchanged prompts. Anthropic documents that its tokenizer from Claude 4.7 onwards produces about 30 percent more tokens for the same text and advises recounting prompts against the new model.
• Requests are counted instead of tokens. Forecasts built on requests times an average cost miss as soon as prompts, history or the model change, because a request is not a unit of cost.

Example

List prices per million tokens in USD, read on 2026-10-15. Claude Sonnet 5.5 on the Claude API with global routing (the default, without the US-only inference multiplier); Gemini 3.1 Pro Preview on the Gemini API paid tier, prompts up to 200,000 tokens.

Anthropic documents its cache rates as multiples of input: 1.25 times for a five-minute write, twice for a one-hour write and 0.05 times for cache hits on Claude Sonnet 5.5; both providers give batch as a 50 percent reduction. Above 200,000 prompt tokens, Gemini 3.1 Pro Preview costs USD 4.00 for input and USD 18.00 for output.

Sources

Read on 2026-10-15. OpenAI pricing, tokens and context window, prompt caching and reasoning models; Anthropic pricing, Token counting and extended thinking; Google Gemini API pricing and understand and count tokens.