Last reviewed: 2026-07-31

Direct answer

Count the complete request that will reach the selected model before you approve its budget. That means using the exact model ID and including the system instruction, conversation history, retrieved context, tool definitions, files, images, and other inputs—not merely the latest user message. Keep a separate output allowance because a preflight input count does not predict how many tokens the model will generate.

For each workload and model route, calculate input cost and output cost with the current CometAPI price for that same model and billing mode. Then multiply by expected request volume and any calls created by agent steps, retries, or fallbacks. Run a bounded pilot and compare the estimate with the usage returned after successful requests. The budget should record both the estimate and the observed variance.

The basic text-token calculation is:

monthly_input_cost = requests * average_input_tokens * input_rate_per_million / 1_000_000
monthly_output_cost = requests * output_allowance * output_rate_per_million / 1_000_000
monthly_base_cost = monthly_input_cost + monthly_output_cost
approved_budget = monthly_base_cost * workflow_call_multiplier + documented_contingency

This is a planning model, not an invoice replica. The CometAPI prelaunch cost guide recommends combining request volume, average input, expected output, and current rates, then validating the result with pilot usage. Its rough character-based shortcut can help during very early discovery, but it is not a substitute for counting the real request against the intended model.

Who this is for

This workflow is for AI platform engineers, FinOps owners, product leads, and budget approvers who route workloads through CometAPI and need a defensible forecast before launch or model migration. It is particularly useful for chat products that resend history, retrieval-augmented generation systems, agents with tool schemas, and multimodal requests whose billable input is larger than the visible prompt.

It is less useful as a one-time arithmetic exercise. Model routing, request structure, output limits, and pricing can all change. Treat token counting as a release control tied to a specific workload version, model ID, interface, and pricing snapshot.

Key takeaways

  • Count the exact request shape against the exact destination model. Counts from a different model or tokenizer are not interchangeable.
  • Separate preflight input counts from output allowances. Provider count operations generally tell you about input; actual response usage is needed for reconciliation.
  • Include hidden-to-the-reader structure such as system instructions, tool declarations, conversation history, retrieved passages, files, and multimodal content.
  • Preserve an evidence row for each model route instead of averaging unlike providers into one blended token count.
  • Treat local tokenization as an estimator with a named method and version, not as universal truth.
  • Fail closed when counting or pricing evidence is missing: hold approval, reduce the pilot, or use a documented conservative ceiling.
  • Compare estimated and actual usage before scaling. Repeated variance is a contract mismatch that needs investigation, not a reason to quietly enlarge the budget.

Sources checked

  • The CometAPI guide to estimating AI API costs supplies the planning framework: request volume, input and output tokens, model rates, output limits, workflow multipliers, sensitivity analysis, and pilot reconciliation.
  • OpenAI’s tiktoken counting example explains that models can use different encodings and that tokenization helps estimate context fit and token-priced usage. The page is explicitly archived, so use it as an illustration of model-aware local counting rather than as a current cross-provider contract.
  • Anthropic’s token-counting documentation says its count operation accepts the same structured inputs used to create a message, including system prompts, tools, images, and PDFs. It also warns that the result is an estimate and that the same content can tokenize differently across model generations.
  • Google’s Gemini token-counting documentation distinguishes the preflight input count from post-request usage. Its usage fields can separately report input, output, thought, cached-content, tool-use, and total tokens, and its documentation confirms that system instructions, tools, and non-text modalities contribute to tokenization.

Contract details to verify

1. Pin the model and request interface

Record the CometAPI model ID, endpoint family, workload version, and pricing-snapshot date. Do not label a row only as flagship, fast, or mini. A budget needs the identifier actually selected by routing logic. If the application can fall back to another model, create a separate row for that route with its own count and price.

Confirm the output-control field supported by the selected interface. The CometAPI source notes that the applicable output limit can be endpoint-specific. An ignored or misspelled limit turns a bounded forecast into an open-ended assumption.

2. Count the serialized workload, not a hand-edited prompt

Build representative samples from sanitized workload shapes. Include short, typical, and large requests. Preserve the number and size of retrieved passages, history turns, tool definitions, and media items while removing user content from the evidence packet.

Anthropic documents that system prompts, tools, images, and PDFs can be submitted to its count operation. Google likewise states that text and non-text inputs are tokenized and that system instructions and tools contribute to input usage. Those details are a warning against counting only the text box a user can see.

For an OpenAI-compatible route, a local tokenizer can be useful when it is mapped to the intended model. However, the supplied OpenAI recipe is archived and lists historical model mappings. If the selected current model or message wrapper is not covered by a verified mapping, label the local result provisional and rely on a small pilot for calibration.

3. Build three budget cases

Use at least three cases instead of one average:

  • Typical: representative input count and expected response length.
  • High: a large but legitimate context plus the configured output ceiling.
  • Workflow: the high case multiplied by expected planning, tool, verification, retry, and fallback calls.

Keep input and output rates separate. Do not apply a cached-input or batch discount unless the workload and selected route meet the relevant contract and the pilot demonstrates the expected behavior. Snapshot the source and review date rather than copying a price into a permanent spreadsheet with no provenance.

4. Happy-path operator workflow

  1. Freeze the workload version, model route, and output limit.
  2. Select sanitized short, typical, and large request shapes.
  3. Submit the complete input to the provider-aware count method supported for that route, or use a verified local tokenizer when appropriate.
  4. Reject any sample that exceeds the selected model’s input or combined-context constraints.
  5. Calculate typical, high, and workflow cost cases with a current CometAPI pricing snapshot.
  6. Run a small, capped pilot.
  7. Capture actual usage returned by successful responses without retaining prompt content.
  8. Compare estimated input with actual input and compare the output allowance with observed output.
  9. Investigate material variance, correct the request contract, and repeat the pilot before approval.
  10. Store the approved assumptions beside the usage review. The CometAPI cost-and-usage tracing guide provides the next control for reconciling live evidence.

5. Error-path operator workflow

If token counting times out, is rate-limited, rejects the request shape, or has no verified mapping for the model, do not silently substitute a count from another model.

  1. Mark the sample as uncounted and record the error class.
  2. Check the model ID, request schema, media support, and input size.
  3. Retry only under the normal bounded retry policy.
  4. If the method remains unavailable, either pause approval or run a tightly capped pilot using a conservative input ceiling and output limit.
  5. Keep the fallback estimate visibly provisional.
  6. Require actual usage evidence before increasing traffic.

Anthropic notes that token counting and message creation have separate rate limits, so a count failure does not by itself prove that generation is unavailable. Operationally, it still means the preflight evidence is incomplete.

6. Keep a sanitized evidence log

A useful record contains dimensions and counts, not customer text or credentials. For example:

{
  "request_id": "req-0421",
  "workload": "support-summary",
  "workload_version": "v3",
  "model_id": "selected-model",
  "count_method": "provider",
  "count_status": "success",
  "input_estimate": 1840,
  "output_cap": 300,
  "actual_input": 1852,
  "actual_output": 164,
  "workflow_calls": 1,
  "price_snapshot_date": "2026-07-31",
  "content_logged": false,
  "payload_excerpt": "[REDACTED]",
  "error_class": null
}

For a failed count, retain only what is needed to diagnose the control:

{
  "request_id": "req-0422",
  "workload": "support-summary",
  "model_id": "selected-model",
  "count_status": "error",
  "http_status": 429,
  "error_class": "rate_limit",
  "fallback_action": "hold_budget",
  "content_logged": false,
  "payload_excerpt": "[REDACTED]"
}

Aggregate these records by workload version and model route. Do not use a global average that hides a high-context route or a costly fallback.

Failure modes

Reusing a count from another model

Different models can use different tokenization. Anthropic specifically documents a tokenizer change for newer model generations and instructs readers to recount against the model they intend to use. A migration budget built from the old count can therefore understate usage even when the visible prompt is unchanged.

Counting only the latest user message

System instructions, history, retrieval context, tool schemas, images, audio, video, and documents may all contribute to input. The result can be a neat spreadsheet that describes only a fraction of the submitted request.

Treating a character shortcut as an exact count

A characters-per-token shortcut is suitable for rough discovery, especially for plain English text. It does not preserve model encoding, structured message overhead, language mix, tools, or multimodal inputs. Do not use it as sole approval evidence.

Treating preflight input as total cost

Anthropic’s count result is an estimated input total. Google’s preflight method also returns input tokens, while response usage supplies additional categories. Output, thinking, tool use, cached content, and workflow fan-out can change the final cost. The budget needs an output allowance and a post-call reconciliation step.

Ignoring loops, retries, and fallbacks

A request may trigger planning, execution, verification, and another model route. Counting one call correctly still understates a workflow that makes several calls. Record a call multiplier based on the application design and validate it during the pilot. If retry behavior is material, pair this workflow with the retry-inflation review .

Using stale or mismatched pricing

Even a perfect token count produces a bad forecast when multiplied by a price from another model, billing mode, or review date. Attach a dated pricing snapshot to the approval and use the pricing-snapshot control checklist when the model catalog changes.

Failing open when counting breaks

Substituting zero, dropping tool schemas, or copying yesterday’s unrelated count makes the approval look complete while removing its evidence. A failed count should create a visible hold, conservative pilot, or exception—not an invisible default.

Logging the payload to prove the count

Raw prompts and files are unnecessary for routine cost evidence. Store workload dimensions, model route, status, counts, usage, and a redaction marker. Keep sensitive content out of the cost ledger.

FAQ

Is dividing characters by four good enough?

Only for an early rough range on suitable text. The CometAPI source uses that approximation in an illustrative estimator, while the provider documents show why exact request structure and model-aware tokenization matter. Use the selected model’s count method or a verified tokenizer before approval.

Which token count should I trust for a multi-model route?

Keep one count per destination model. Prefer a current provider-aware count of the exact serialized request. If you use a local tokenizer, record its mapping and treat unsupported models as provisional. Never average counts first and then apply several unrelated prices.

Can I know output tokens before making the request?

Not exactly. Set an enforceable output ceiling where the endpoint supports one, model a typical output and a high case, then record actual output usage during a pilot. Approval should show both expected and bounded exposure.

Does a preflight count equal billable usage?

No. Anthropic explicitly describes its count as an estimate that may differ slightly from message usage. Google separates preflight input counting from richer post-response usage. Use preflight for control and post-call usage for reconciliation.

What if the count endpoint is rate-limited?

Do not infer a zero count or switch to another model’s tokenizer without evidence. Record the error, use bounded retries, and either pause approval or run a small capped pilot with a conservative ceiling. Anthropic documents separate limits for counting and message creation, so track those operations separately.

How often should the count be refreshed?

Refresh it when the model ID, provider route, system instruction, tool schema, retrieval policy, media settings, conversation-history policy, or tokenizer mapping changes. Recounting should also be part of a model migration, because unchanged text can produce a different count on another model.

Reader next step

Choose one high-volume workload and freeze its exact request contract. Count three sanitized request shapes against every model route, attach current pricing, set a real output ceiling, and run a capped pilot. Reconcile the results before approving scale, then turn the high-case exposure into actionable thresholds with the budget-alert input guide .

When you are ready to test the workflow across available models, Start with CometAPI . Preserve the model ID, count method, pricing snapshot, and actual usage together so the next reviewer can reproduce the decision.