Counting Tokens Before Inference: Three Provider APIs

Every LLM request draws down two budgets at once: the model’s context window and the provider’s tokens-per-minute quota. Amazon Bedrock, Anthropic and Google Vertex AI each expose a token-counting endpoint that measures the exact input a model will tokenize before you spend a single inference token. Treat that count as a pre-flight gate: it is what a router needs to choose between models, what a limiter needs to avoid throttling, and — on Bedrock — the input to a quota formula in which output tokens can consume many times their face value. The three endpoints differ in request shape, in what they accept, and in how literally you can trust the number.

Why Pre-Flight Counting Pays Off

Throttling on managed model APIs is denominated in tokens per minute, not requests. One oversized RAG prompt assembled from retrieved chunks can consume more of your TPM quota than dozens of short calls, and the failure mode is expensive: the request is rejected after you have already paid to build and serialize the context. Counting first converts that failure into a routing decision — trim the prompt, move the stable system prompt into a cached prefix, or downgrade to a cheaper model for prompts that fit its window. It sits in the same family of serving-cost levers as weight quantization: small input-side adjustments with multiplicative effects on spend and throughput. The practical rule is simple: never let the first measurement of a prompt’s size be the invoice.

The Three Provider Endpoints

Each provider exposes a count-only call that accepts essentially the same payload you would send for inference, so you can wire it directly into a gateway or client wrapper.

In prose: Google documents no charge for the CountTokens API and caps it at 3000 requests per minute, which is generous enough to run per-request in a gateway. AWS charges nothing and goes further — it promises the count will match the billed amount for the identical input. Anthropic’s endpoint is the most permissive on payload types (base64 images and PDFs count fine) but explicitly labels its result an estimate. All three are model-specific: the count is only meaningful for the exact model ID you pass, because tokenizers differ across and within provider families.

Bedrock Quota Burndown Math

Bedrock is the provider where counting changes capacity planning, because quota consumption follows a two-stage formula. At the start of each request, Bedrock deducts total input tokens plus max_tokens from your TPM and TPD quotas; when the response finishes, it adjusts the deduction to InputTokenCount + CacheWriteInputTokenCount + (OutputTokenCount x burndown rate) and replenishes the unused remainder. Cache reads are excluded from quota consumption entirely, which is the accounting reason stable prefixes should be cached on purpose rather than by accident.

The burndown rate is where surprises live. On the bedrock-runtime endpoint, one output token from an Anthropic Claude 4.8 model consumes 15 tokens from TPM and TPD quotas, while Claude Opus 5.5, Claude Sonnet 5, Claude Opus 5 and Claude Fable 5.1 burn 10 tokens each and Anthropic models at version 4.7 and below burn 5. Non-Anthropic models default to 1:1. AWS illustrates the interaction with max_tokens directly: a documented scenario with 3,000 input tokens, 1,000 cache-write tokens and max_tokens set to 32,000 deducts 36,000 tokens up front, versus 5,250 when max_tokens is tuned to 1,250 — same final 9,000-token adjustment in both cases. If you are hitting TPM limits earlier than your traffic justifies, your max_tokens value, not your traffic, is usually the culprit.

Estimate Drift Across Models

Pre-flight counts are planning numbers, not billing guarantees, except where a provider says otherwise. Anthropic states that Claude 4.7 and later models and Claude Mythos Preview use a newer tokenizer, and that the same input text produces approximately 30 percent more tokens than on earlier models — a large enough shift that counts measured against an older model will understate cost on the new one. Anthropic also notes its count is an estimate that may differ slightly from the tokens actually consumed, and that tokens added automatically for system optimizations are not billed. Google’s documentation likewise describes multimodal token counts as an estimate. The operational answer is to recount prompts against the target model whenever routing changes, and to log the usage fields in each inference response as the ground truth for reconciliation.

A Pre-Flight Checklist

1. Count with the provider endpoint for the exact model ID you will call; tokenization is model-specific and not portable across versions.
2. Recount whenever a router switches model versions — newer tokenizers can inflate the same text materially.
3. Set max_tokens near your 95th-percentile completion length, not a round maximum; Bedrock deducts input plus max_tokens up front.
4. Design stable prefixes for prompt caching and remember cache reads do not deplete Bedrock quota.
5. Treat response usage fields as billing ground truth, log them centrally, and reconcile pre-flight estimates against actuals weekly.
6. Wrap the counting call in the same retry and timeout policy as inference — it is a network hop, not a free local function.

Counting before inference costs one cheap API call and removes the two most common self-inflicted outages in LLM production: context-window overflows and quota throttling from padded max_tokens values. Wire the endpoint into your gateway once, and every routing, trimming and caching decision downstream gets a number to work with.

Sources

• Amazon Bedrock — Monitor your token usage by counting tokens before running inference
• Amazon Bedrock — How tokens are counted
• Anthropic — Token counting
• Google Cloud — Use the Count Tokens API