Keep AI API Cost Records Independent of Trace Sampling

Last reviewed: November 6, 2026 (Asia/Shanghai).

Direct answer

Keep AI API cost records on a durable path that runs for every upstream attempt, independently of whether its diagnostic trace is retained. Capture the attempt identity before dispatch, preserve usage evidence or an explicit unresolved outcome afterward, and reconcile calculated costs with the applicable billing source. Trace retention must not determine which attempts enter the cost report.

OpenTelemetry’s sampling guide describes sampling as selecting which traces or spans are processed and exported. Head sampling decides early; tail sampling can select traces by errors, latency, or other attributes. A dashboard that sums token attributes from the retained spans therefore needs an explicit completeness assessment before anyone treats it as total usage.

The operating check is simple: with the same dispatched attempts and usage evidence, changing trace retention should leave the accounting result unchanged. Missing traces can reduce diagnostic detail. Missing cost evidence must remain visible as uncertainty.

Who this is for

This guide is for AI platform engineers and FinOps owners whose cost dashboards use observability data. It is especially useful before reducing trace retention, adding a tail sampler, or replacing request records with metrics derived from spans.

The architecture, numerical example, and acceptance checks below are recommendations. No authenticated model call, billing account, or production telemetry pipeline was tested for this article. The example uses hypothetical amounts, not supplier prices.

Key takeaways

• Record each upstream attempt even when no diagnostic span is retained.
• Give retries and fallback calls distinct attempt identities within their logical request.
• Verify where metrics are created; aggregation after trace sampling cannot recover discarded observations.
• Use monitoring counters to detect gaps, with durable records supporting accounting and reconciliation.
• Keep absent usage, observed zero usage, calculated cost, and billed cost distinguishable.
• Preserve cost rows when optional trace enrichment is unavailable.

Contract details to verify

Identify what each dataset can establish

Draw the path from model dispatch to usage capture, local persistence, exporters, collectors, transformations, and reports. Mark every point that can drop, sample, aggregate, retry, or overwrite observations. Check the deployed configuration rather than assuming a metric is complete because its name contains “total.”

Use separate evidence for different decisions:

The Prometheus overview explicitly says Prometheus is unsuitable when per-request billing requires 100% accuracy because collected data is likely insufficiently detailed and complete. Maintain accounting evidence alongside the monitoring measurements used for alerts and operational cost trends.

For a metric derived from spans, identify the input population. A transformation receiving only retained spans sees only those spans. Moving aggregation before a downstream sampler can avoid that particular loss, but cannot recover spans already omitted by an upstream sampler. A separate application measurement can avoid dependence on trace retention; it still needs its own delivery and coverage checks.

Define one attempt without relying on a trace

Assign an opaque attempt ID at the controlled dispatch boundary. Retain a separate logical-request ID to group retries, fallback attempts, or parallel candidates. A trace ID is optional correlation: one trace can contain several upstream calls, and a cost event must remain usable when its trace is unavailable.

Record the workload and owner scope, account reference, endpoint family, model identifier, dispatch time, outcome, usage evidence reference, and settlement state. Keep observed usage categories and their units separate from the price revision and currency used to calculate cost. Include an upstream request identifier when the response exposes a suitable one.

Exclude credentials, prompts, generated text, and raw customer identifiers from routine cost events. Controlled accounting storage can retain the necessary opaque references without making every identifier a monitoring label.

The Prometheus instrumentation guidance warns that each label set creates another time series with memory, CPU, storage, and network costs. Use bounded monitoring dimensions such as approved workload class, outcome, and model group. Keep per-attempt identifiers in the durable record instead of creating a time series for every request.

Operator workflow

Capture accounting evidence outside trace-selection conditions

Use this recommended sequence in the component that owns upstream dispatch:

1. Persist the attempt intent and acquire its dispatch ownership before sending. Prevent other workers from dispatching the same intent independently.
2. Dispatch the permitted attempt. Ensure hidden SDK retries are either disabled or represented in the accounting design with their own exposure and lineage.
3. Persist received usage through an idempotent state transition. If usage is absent, delayed, or lost after dispatch, retain a pending outcome and its evidence gap.
4. Update the appropriate monitoring measurements. Offer diagnostic attributes to tracing under the separate tracing policy.
5. Reconcile unresolved attempts and calculated costs with later usage or billing evidence.

Do not place step 3 inside a “span is recording” condition or a callback that runs only for exported spans. Inspect wrapper libraries and collector transformations as well as application code. A sampled diagnostic log carrying the only usage copy has the same dependency problem.

Keep critical admission state durable when a trace exporter fails. If the accounting store cannot accept new intents, stop or defer new dispatches unless an explicitly bounded, durable alternative has already been approved. A successful response followed by a lost usage write is still an accounting gap; preserve the preexisting attempt for recovery.

The site’s atomic budget reservation guide covers admission, dispatch ownership, and unresolved commitments. Here the additional requirement is that no trace-selection decision can bypass those records.

Preserve missing evidence through aggregation and joins

For each defined scope and cutoff, classify every potentially dispatched attempt into one mutually exclusive state: final usage captured, explicitly unresolved, or verified never sent. The last state requires evidence that dispatch did not occur and cannot occur later. A timeout alone does not establish it.

When a completion event repeats, return the existing result rather than adding its usage again. A later correction needs a versioned replacement or adjustment with its own identity. Retain raw usage evidence and the applied pricing revision so a replay cannot silently reprice history.

Build spend reports from the accounting population first. Add trace details through an optional, cardinality-controlled join or separate lookup. An inner join to retained traces can remove legitimate cost rows; a one-to-many join to spans can multiply them. Check both row counts and monetary totals before and after enrichment, and keep a visible “trace unavailable” state.

The FinOps Foundation’s Data Ingestion capability calls for appropriate granularity and completeness, quality checks, availability monitoring, and reconciliation among usage and billing sources. Apply those principles to this path with explicit indicators:

• attempts awaiting usage, their age, and their outstanding reserved exposure;
• failures or lag between usage capture and durable persistence;
• known usage without a valid price mapping;
• accounting-to-billing differences by a compatible account, period, currency, and cost basis.

A complete count of known attempts does not prove that every dispatch was recorded. Audit alternative clients, retry layers, background workers, and bypass routes. Provider-side evidence can expose activity that the application never captured.

Why a retained-trace percentage cannot settle the bill

Consider a hypothetical workload whose full attempt costs are known independently. A diagnostic policy retains 10% of short attempts and all long attempts. The table fixes the retained counts for illustration; it is not a measured sampling result.

Summing the retained costs reports $21.80 against the known $38.00. Dividing $21.80 by the overall 19% retained-attempt rate instead produces about $114.74. The expensive class has a different retention rate, so one multiplier cannot reconstruct this total.

A known probability sample can support a statistical estimate when its inclusion probabilities, weights, population, and uncertainty are handled correctly. Outcome-based policies, multiple sampling stages, and delivery loss can complicate those conditions. Keep any resulting estimate labeled as an estimate. It does not identify each missing charge or replace reconciliation.

Use the durable attempt records for calculated usage cost, and retain unresolved exposure separately. Provider charges can include adjustments or components outside that calculation. The site’s cost export ingestion guide explains how to ingest refreshed billing evidence without counting overlapping snapshots twice.

Verification before rollout

Use a fake model endpoint, synthetic usage, and disposable accounting storage. These are proposed acceptance checks, not reported test results.

Compare identical fixture inputs across trace policies. A live canary needs comparison by traffic scope and timing because real workloads vary. Approve a lower retention rate only after tracing changes leave accounting unchanged and collection failures remain observable.

Failure modes

A quiet dashboard is reported as savings. Trace retention fell while paid traffic continued. Record telemetry policy changes beside spend reports and check independent usage or billing evidence before claiming reduced consumption.

A monitoring counter becomes the invoice ledger. Aggregation is useful for alerts, but cannot necessarily reconstruct missing attempts, corrections, or account adjustments. Preserve the durable evidence needed for that decision.

Every successful request is treated as fully accounted for. The user received text, but the usage event failed to persist. Keep capture and settlement status separate from delivery status.

Zero usage and missing usage collapse into one value. Preserve null or explicit pending states until evidence establishes a value. Releasing reserved funds on missing data can create false budget headroom.

Reporting drops or multiplies costs during enrichment. Optional trace joins alter the financial population. Enforce join cardinality and compare totals before publishing the enriched view.

Sources checked

Four primary public pages returned HTTP 200 on November 6, 2026 UTC and Asia/Shanghai:

• OpenTelemetry: Sampling — retained versus unretained traces, early head decisions, and attribute-based tail selection. The page displays an October 16, 2025 modification date.
• Prometheus: Overview — time-series monitoring and the explicit limitation for exact per-request billing.
• Prometheus: Instrumentation — attempt and failure monitoring, label cardinality costs, counters, and missing metrics.
• FinOps Foundation: Data Ingestion — source granularity, completeness, quality, timeliness, and usage-to-billing reconciliation.

These are currently served official references for an evergreen accounting design. No page-specific update date was visible in the retrieved Prometheus or FinOps text. Retrieval establishes current availability; the design still needs verification against the deployed instrumentation and supplier billing contract.

FAQ

Must we retain every trace to account for AI costs?

No. Preserve the required accounting evidence independently, then select diagnostic traces under the observability policy. Reducing trace detail must leave usage capture and pending outcomes intact.

Are metrics emitted before sampling enough?

They can avoid loss caused by a later trace sampler. They still need coverage, persistence, restart, and delivery checks. An aggregate alone may not support per-attempt corrections or billing disputes.

Can we use a trace ID as the settlement key?

Use a dedicated attempt identity. Multiple calls can share a trace, and retries or fallbacks can each have financial consequences. Keep trace correlation optional.

Should incomplete cost data disable all application traffic?

Apply the documented control for the affected scope. Critical admission can defer new spending while already dispatched attempts remain visible. A previously approved durable local allowance may support bounded continuation; an unbounded silent bypass cannot establish a financial ceiling.

Reader next step

Choose one AI route and identify the earliest sampling decision in its telemetry path. Trace a synthetic usage event through persistence and reporting, then repeat with every diagnostic trace dropped. Require unchanged accounting results, visible pending outcomes, and preserved report rows before reducing production trace retention.