Last reviewed: 2026-08-05

Direct answer

Treat every streaming request as financially open until you have both a terminal event and a usable token record. Text arriving at a client proves that generation started; it does not prove the final input, output, or total token count. For a CometAPI route, keep three separate states: complete and observed, partial and observed, and incomplete with usage pending. A missing usage event is an unknown value, not zero.

The evidence arrives differently across providers. The OpenAI streaming cookbook describes enabling usage reporting so an extra final chunk carries statistics for the entire request. That page is marked archived, so treat its mechanism as a compatibility reference and confirm the current behavior of the selected CometAPI route. Anthropic’s streaming messages documentation puts input usage in message_start and exposes cumulative output usage in message_delta before the final message_stop event. The Gemini API reference defines streamGenerateContent and a UsageMetadata object with promptTokenCount, candidatesTokenCount, totalTokenCount, and thoughtsTokenCount. The Amazon Bedrock ConverseStream reference separates messageStop from a metadata usage object containing inputTokens, outputTokens, and totalTokens.

These differences lead to one operational rule: normalize event evidence at the gateway boundary, but retain the original provider event names and counters. Close a row as settled only when the route’s documented terminal signal and usage fields have arrived. If the connection closes first, preserve what was observed, label the remainder pending, and reconcile it later from an approved provider or gateway record. Do not quietly bill an estimate as fact.

Happy-path workflow

  1. Create a fresh stream_id and attempt number for each request. A retry is a new attempt, even when the user sees one answer.
  2. Record only routing metadata at start: provider family, CometAPI route, model identifier, request class, and a monotonic start time. Keep prompt and completion content out of the cost ledger.
  3. Enable the route’s documented usage reporting option when one exists. For OpenAI-compatible routes, verify whether the option is forwarded; do not assume that an OpenAI parameter name works on every model.
  4. Persist a small event envelope for each chunk: sequence number, event type, receive time, byte count, and whether a usage object was present. Store cumulative counters as snapshots, not as additive deltas unless the provider explicitly defines them that way.
  5. On the terminal event, map input, output, reasoning, tool, cached, and total counters into your ledger’s fields. Mark usage_confidence as observed and record the provider field names in a mapping note.
  6. Reconcile the arithmetic before closing the row. If a provider supplies total tokens, compare it with the component fields. If the numbers disagree, keep the row open for review instead of choosing the larger number.

A sanitized ledger record can look like this:

{
  "stream_id": "[REDACTED]",
  "attempt": 1,
  "provider": "openai-compatible",
  "route": "cometapi",
  "model": "[REDACTED]",
  "event_state": "complete",
  "terminal_event": "usage_and_stop",
  "stop_reason": "end_turn",
  "input_tokens": 842,
  "output_tokens": 317,
  "total_tokens": 1159,
  "usage_confidence": "observed",
  "error_class": null,
  "started_at": "[REDACTED]",
  "ended_at": "[REDACTED]"
}

Error-path workflow

When a client cancels, a proxy times out, or a provider emits an error, finalize the transport record but not necessarily the financial record. Save the last sequence number, the last observed usage snapshot, the terminal event that was expected, and the reason the stream ended. Use a status such as partial_observed when counters are available, or usage_pending when they are not. A partial answer must be treated as potentially billable; absence of a final chunk cannot be interpreted as a free request.

If an error event arrives before message_stop, retain the error type and do not manufacture a stop reason. If a gateway reconnects, do not append the second connection to the first attempt unless the provider contract explicitly guarantees resumability. Start a new attempt and link it with a short internal parent reference. This prevents one user action from hiding two billable calls.

For pending rows, set a reconciliation deadline and an owner. Check the route’s usage export, invoice detail, or request log when those records are available. If only an estimate is possible, store the formula, inputs, and confidence beside the estimate and keep the settled amount separate. An estimate is useful for an alert; it is not evidence that the final amount has been charged.

Who this is for

This workflow is for FinOps owners who need complete spend attribution, platform engineers maintaining a streaming gateway, and SREs investigating disconnects or timeouts. It also helps product teams that display partial answers and need to know whether a cancellation changed cost.

It is especially useful when one CometAPI integration fans out to several model providers. A single tokens column hides important differences: one provider may send a final aggregate, another may send cumulative updates, and another may place usage in a separate metadata event. The operator’s job is to preserve those distinctions while presenting a common ledger to finance.

Key takeaways

  • Count a stream as settled only when usage and the route’s terminal signal agree.
  • A missing terminal usage event is unknown or pending, never zero.
  • Store cumulative provider counters as snapshots; adding every snapshot double counts.
  • Give every retry its own attempt identity and cost row.
  • Keep event metadata and sanitized identifiers, not prompts, completions, or credentials.
  • Use provider-specific adapters behind one normalized CometAPI ledger.
  • Treat an estimate as an alerting value until an observed record replaces it.
  • Test clean completion, client cancellation, gateway timeout, provider error, and retry separately.

Sources checked

The public references below were refetched for this article. They document event order and usage fields, not a universal CometAPI billing promise, so the route contract still needs verification.

Contract details to verify

Before shipping an adapter, write down the route contract in plain language:

  • Terminal evidence: which event means the provider considers the message finished, and which event carries final usage?
  • Counter semantics: are output counters cumulative, incremental, or only present once?
  • Scope: do totals include tool calls, reasoning tokens, cached content, and all generated candidates?
  • Cancellation: what does the route return when the client closes the connection after receiving partial text?
  • Error behavior: can an error event arrive after some usage, and is a later retry charged independently?
  • Gateway forwarding: does CometAPI preserve provider event names and usage objects, or transform them?
  • Join key: which sanitized request identifier connects the stream record to a usage export or invoice line?
  • Retention: how long are event envelopes and pending estimates kept?

Use the existing Trace CometAPI Cost and Usage for Token Budgets article to align the normalized ledger with broader spend records. Pair it with Review CometAPI Error and Cost Signals Before Token Budget Decisions when defining error classes and approval thresholds. A third useful reference is the CometAPI Cost Ledger Source Pack checklist , which can hold the route-specific field mapping and evidence owner.

Run a controlled test for each model family: one normal stream, one client cancellation after visible output, one gateway timeout, and one provider error. Compare the event envelope with the eventual usage record. A passing test has an explicit state transition and a reproducible explanation for every token count. A failed test stays pending; it does not get fixed by copying a neighboring request’s total.

Failure modes

The final usage chunk never arrives. This is common when the transport is interrupted. Keep the last observed snapshot, set usage_pending, and alert on age. Do not write zero or reuse a prior total.

Cumulative counters are added together. Anthropic documents cumulative usage in message_delta. Adding 10, then 20, then 30 output tokens records 60 instead of 30. Store each snapshot and select the latest valid one.

A stop event arrives without usage. A clean-looking messageStop or message_stop establishes transport state, not necessarily a complete cost record. Mark the row complete_transport but pending_usage until the route provides counters.

An error arrives after partial content. Preserve the provider error type, sequence, and last usage snapshot. Classify the result as partial or failed according to the route contract. Do not infer that the model produced no billable work.

A client retry is merged into the original stream. Separate attempts prevent double counting and make latency analysis honest. Link attempts with an internal parent reference, but never sum a cumulative snapshot from one attempt with a total from another.

A proxy reconnects and duplicates chunks. Use sequence numbers and an event hash within one attempt. If the provider has no resumable-stream guarantee, treat a second connection as a new request.

Unknown event types are discarded. Anthropic’s documentation notes that new event types can appear. Preserve an unknown envelope and continue parsing safe fields; dropping it can erase the only terminal or usage signal.

Tool-use events are mistaken for final text. A stream can pause while a tool call is assembled. Keep content-block state separate from message completion, and do not close the ledger when a tool input merely finishes.

Provider totals and components disagree. Keep both values, record the discrepancy, and route the row to reconciliation. A deterministic choice is better than a silent guess, but it still needs an evidence trail.

FAQ

If the user saw text, was the request billed?

You cannot answer from display text alone. Text proves that some response work occurred, while billing and usage depend on the provider and route. Treat the row as observed partial only when a usage snapshot exists; otherwise keep it pending.

Can I estimate output tokens from the number of characters?

You can create an operational estimate, but label it clearly and keep it out of settled totals. Tokenization varies by provider, model, language, and modality. Replace the estimate with an observed usage record when one becomes available.

Should I add every usage event in a stream?

Only if the provider defines each event as an incremental delta. Anthropic’s message_delta usage is cumulative, so the safe default is to store snapshots and use the latest valid cumulative value. Record the adapter rule with the ledger row.

Does message_stop prove that the bill is final?

It proves that the provider sent its documented terminal message event. It does not prove that a gateway forwarded every usage field or that an external billing record has reconciled. Pair terminal state with usage evidence and route-specific contract checks.

What should happen when a stream is canceled by the user?

Keep the partial output state and any usage already observed. Mark the attempt canceled or partial, preserve the cancellation timestamp, and reconcile later. If the user tries again, create a new attempt rather than overwriting the first.

How do I handle a provider error after an observed input count?

Keep the input count, error type, and missing output count as separate fields. The request may have incurred input cost even if no final answer was delivered. Let the provider or gateway record settle the amount.

Can one normalized schema cover every provider?

It can cover common dimensions such as input, output, total, status, and confidence, but it should also retain a provider_fields map. Provider-specific counters such as thoughts, cached content, tool-use prompts, or candidate totals should not be discarded.

Reader next step

Choose one CometAPI route and make the ledger testable before changing production defaults:

  1. Capture a clean stream and confirm the terminal usage mapping.
  2. Cancel after visible output and verify the row becomes partial or pending, never zero.
  3. Force a timeout and a provider error, then confirm each attempt has its own identity.
  4. Compare normalized totals with the route’s available usage record.
  5. Set an owner and deadline for every pending row.

For the broader cost-control workflow, start with Trace CometAPI Cost and Usage for Token Budgets , then review the error evidence before approving a route. When you are ready to test a route through the gateway, Start with CometAPI .