Bound Background Responses Before Asynchronous AI Spend Drifts

Last reviewed: November 4, 2026 (Asia/Shanghai); public references fetched November 4, 2026 UTC (last checked 2026-11-04T20:18:04Z).

Direct answer

Background model responses are a delivery mode, not a budget. A request that continues after the client disconnects still needs an owner, a maximum output allowance, a deadline, a polling plan, and a terminal-state policy. Before enabling it, calculate a conservative pre-call estimate from the complete input and output ceiling, record the response ID as the attempt identity, and reconcile actual usage after the response reaches a terminal state.

CometAPI’s current Responses reference documents background: true, says the initial call returns a response that can be retrieved later by ID, and exposes queued, in_progress, completed, failed, cancelled, and incomplete statuses. It also defines max_output_tokens as an upper bound that includes visible output and reasoning tokens, and provides input, cached, cache-write, output, reasoning, and total usage fields. The same page exposes response IDs and terminal status fields; the upstream cancellation contract is checked separately below. These are interface facts; they do not establish a guaranteed runtime, price, cancellation credit, or final invoice.

The safe design is a bounded state machine: admit once, persist the response ID, poll at a controlled cadence, stop when the business deadline or cost ceiling is reached, and settle from provider usage or an explicit unresolved state. A background flag must never become a reason to retry after an unknown submission or to poll forever.

Who this is for

This guide is for platform engineers, FinOps owners, and application teams running long model tasks such as reports, code review, document analysis, or tool-assisted research. It assumes the application can persist an opaque request record, change request limits, and keep sanitized usage evidence separate from prompts and outputs.

The examples describe CometAPI’s Responses surface and its documented OpenAI-compatible behavior. No authenticated request, deployment, billing experiment, or latency result was performed for this article. Numeric limits in the examples are policy illustrations, not provider defaults.

Key takeaways

• Treat the response ID as the attempt identity from the first accepted create call.
• Set max_output_tokens before dispatch; it covers reasoning and visible output in the documented Responses contract.
• Give every background task an application deadline shorter than the retention and operational recovery window.
• Poll one owner per response ID with backoff and a maximum poll budget; duplicate pollers add load and complicate state.
• A cancel request is a control signal, not proof that remote work stopped or that usage became zero.
• Reconcile input, cached, cache-write, output, reasoning, total usage, status, and price-snapshot evidence before closing cost.
• Keep failed, incomplete, cancelled, and unknown attempts visible in the ledger alongside successful work.

Contract details to verify

Freeze the request contract before admission

The CometAPI Responses schema accepts plain text or structured input, optional instructions, background, max_output_tokens, max_tool_calls, reasoning, store, prompt_cache_key, and tools. The page says max_tool_calls is a total across built-in tools in one response, rather than a separate allowance for each tool. If tools are enabled, include that ceiling in the workload policy; a token cap alone does not bound search, file, or code actions.

Keep a versioned record for the exact model, route, input shape, tool set, reasoning setting, output ceiling, service tier, prompt-cache configuration, and pricing snapshot. CometAPI’s estimate guide says to combine model pricing with billed units and output limits before the call, then compare actual usage and billing records afterward. Its example uses a rough input estimate and an explicit max_completion_tokens cap; treat that as a planning guard, not an invoice.

Use a preflight record like this, with no prompt or credential:

{
  "workload_id": "long-report",
  "model": "approved-model-id",
  "route": "/v1/responses",
  "estimated_input_tokens": 6800,
  "output_cap_tokens": 2400,
  "max_tool_calls": 2,
  "reasoning_effort": "low",
  "pricing_snapshot": "pricing-2026-11-05-a",
  "deadline_at": "2026-11-05T02:00:00Z",
  "decision": "allow"
}

The input estimate should include system or developer instructions, conversation state, tool definitions, attachments, and the new user input. If the selected tokenizer is only an approximation, retain an uncertainty allowance. Reserve enough budget for the output cap and any separately controlled tool or storage unit. If the price snapshot, model support, or usage mapping is missing, stop at review_required instead of silently substituting a different model.

Define lifecycle and state ownership

Create one local record before the network call with a generated logical-work ID, a submission fingerprint, and state creating. After a successful create response, atomically store the provider response ID, model, creation time, configured limits, and state queued or in_progress. A transport timeout after handoff is submission_unknown, not rejected; reconcile by querying the provider using the persisted identity before considering a new attempt.

Only one worker should own transitions for a response ID. A separate reader can serve status, but it should not create a second response or settle the same usage twice. State transitions should be monotonic for normal work:

creating -> submission_unknown | queued | in_progress
queued/in_progress -> completed | failed | cancelled | incomplete | deadline_exceeded
submission_unknown -> reconciled_existing | proven_absent

deadline_exceeded is an application outcome. It does not claim that the provider used that status. If the response remains active at the deadline, request cancellation where supported, record the cancellation time and response ID, and retain unresolved exposure until the final provider state or billing evidence arrives.

Poll with a bounded budget

OpenAI’s current Background mode guide describes the same Responses lifecycle: set background: true, poll GET /v1/responses/{id} while status is queued or in_progress; CometAPI exposes the same status vocabulary in its response schema, so verify the exact route before treating every other value as terminal. It also documents cancellation through POST /v1/responses/{id}/cancel. The guide warns that background data may be retained only temporarily for polling when store is omitted or false; retention rules vary by project policy. CometAPI’s reference exposes store and response IDs, so verify the selected route’s retention behavior before choosing a long recovery window.

Use one poll owner and an increasing delay with jitter. Stop when one of these conditions is met: a terminal response is observed, the application deadline arrives, the poll budget is exhausted, or the response record is no longer retrievable. A missing response is an evidence gap; it is not permission to create a replacement. Keep a separate poll ledger from the model-cost ledger so a high poll count cannot be mistaken for model usage.

A simple policy can begin with a two-second delay, grow to thirty seconds, and stop after a fixed number of polls or an overall deadline. Those values are application choices. The objective is to bound control-plane work while leaving enough observations to recover a terminal result before the retention window ends.

Settle success and incomplete work differently

When a response is completed, validate the output contract before marking the logical operation successful. A response can be complete while the product parser rejects it, a tool result is missing, or the answer fails a business rule. Record the model status and product acceptance separately.

For failed, cancelled, or incomplete, retain all usage fields that are present and classify the business outcome. The CometAPI schema lists incomplete_details.reason values such as max_output_tokens and content_filter; do not turn an incomplete result into ordinary success. If usage is absent, store unknown rather than zero and keep the attempt open for reconciliation.

A cancel request is not a billing reversal. The OpenAI guide explains how to cancel an in-flight background response, while CometAPI’s public pages do not promise that cancellation removes already consumed usage. Count any returned usage, preserve a pending commitment when it is unavailable, and ask the account owner or support path about account-specific treatment.

Operator workflow

Happy path

1. Define the logical unit, accepted output, deadline, model, route, tool allowance, and cost ceiling.
2. Snapshot current CometAPI request and pricing references; calculate the pre-call estimate with an explicit output cap and uncertainty margin.
3. Persist creating plus a unique fingerprint and reserve the maximum approved exposure.
4. Create one background response. On success, persist the response ID and all configured limits before acknowledging the job.
5. Poll through one owner with bounded backoff. Store only status, timestamps, response ID, and safe error classes.
6. On a terminal response, retrieve usage and output metadata, validate the product result, and settle the attempt against the pricing snapshot.
7. Release only the unused portion of a reservation after dispatch and reconciliation rules permit it. Keep unresolved exposure visible.
8. Close the logical unit with accepted, failed, incomplete, cancelled, or unknown outcome, plus the next action for any unresolved attempt.

Error path

If the create call times out, do not immediately repeat it. Mark submission_unknown, query by the stored attempt identity or provider inventory, and submit a replacement only after evidence proves the original was not accepted. If a poll returns an authorization, not-found, or retention error, preserve the last known state and escalate; a transient control-plane failure does not prove the model stopped.

If the response stays active near the deadline, stop accepting it as an on-time result, issue one cancellation request where supported, and record that cancellation may race with remote processing. If the response later completes, settle its usage and discard its late output for the original user action. If it is never retrievable, retain the unresolved amount under the original response ID and investigate through the documented support or billing channel.

If the configured max_output_tokens or max_tool_calls is rejected, fail closed. Repair the route contract or choose a reviewed supported combination; do not retry without the guardrail. If the product needs a second attempt, give it a new attempt ID and a separate reservation while keeping the first uncertain attempt visible.

Verification before rollout

Use a fake response store or provider sandbox for these acceptance tests; they are test criteria, not observed production results:

Record model, route, response ID, attempt, state, configured caps, status, usage source, pricing snapshot, cancellation request, and product acceptance. Exclude prompts, tool arguments, retrieved documents, generated text, authorization headers, and account secrets.

Failure modes

Background is treated as free. Asynchronous execution changes when the caller waits; it does not remove model, tool, or retry usage. Keep the same admission and settlement controls as synchronous work.

The client timeout is mistaken for provider cancellation. Closing a connection can leave the response running. Persist the ID, poll or reconcile it, and do not infer zero usage from a local timeout.

Every timeout starts another response. An unknown create can already have been accepted. Reconcile the original attempt before any replacement.

Polling is unbounded. A worker may poll forever after retention expires or after the business result is no longer useful. Use deadline, poll-count, and retention checks.

A token cap is the whole budget. Reasoning, tools, retries, input, and storage can add separate exposure. Keep route-specific and workflow-level limits.

A terminal status equals a successful product operation. Validate output and business rules separately from provider status.

Missing usage is recorded as zero. Preserve unknown exposure until provider or account evidence resolves it.

Retention rules are copied from another account. Background storage and polling behavior can depend on project policy. Verify the exact route, project, and store setting before relying on later retrieval.

FAQ

Does background: true reduce the model price?

No. It changes response delivery and retention behavior. Use the current price source and usage evidence for the selected model and route.

Is max_output_tokens only visible answer length?

No. CometAPI’s Responses reference says it includes reasoning and visible output tokens. The exact supported controls can vary by model, so retain the configured value and verify the returned usage.

Can a cancel call guarantee no further charge?

No guarantee is stated in the checked sources. Cancellation is a control request; remote processing and billing evidence must be reconciled separately.

How long should polling continue?

Until a terminal state, application deadline, poll budget, or verified retention boundary. Choose values from the business SLA and route contract, not from an assumed average latency.

Should background tasks use store: true?

Only when the selected project’s data policy and recovery window justify retained response data. OpenAI’s guide says retention differs by project policy and that background data may be temporary when storage is omitted or false. Verify CometAPI behavior before enabling retention.

What belongs in the cost ledger?

Store attempt ID, response ID, model, route, caps, status, usage categories, pricing snapshot, reservation, cancellation request, product outcome, and reconciliation state. Keep content and credentials outside routine cost records.

Reader next step

Choose one long-running task and write its deadline, accepted output, model, max_output_tokens, tool cap, pricing snapshot, poll owner, and unknown-attempt policy on one page. Run the fake-store acceptance matrix, then a bounded sandbox canary. Promote only when create, poll, cancel, terminal-state validation, and usage reconciliation all have an owner and a visible failure path.

For route-level request and usage fields, see CometAPI cost and usage evidence for token budgets . For adjacent retry controls, review CometAPI retry evidence for token-budget decisions . Keep the CometAPI pricing snapshot controls beside the approval record.

Sources checked

• CometAPI: Create a model response — fetched November 4, 2026 UTC. Supports background, response IDs, status values, max_output_tokens, max_tool_calls, store, cancellation-compatible response lifecycle, and usage fields.
• CometAPI: Estimate request cost before calling a model — fetched November 4, 2026 UTC. Supports pre-call estimates from model pricing and bounded input/output or task units, plus comparison with actual usage afterward.
• OpenAI: Background mode — fetched November 4, 2026 UTC. Supports asynchronous creation, polling queued/in-progress responses, terminal states, cancellation, and temporary retention caveats. This is the upstream contract referenced by CometAPI’s OpenAI-compatible Responses surface; verify CometAPI-specific behavior before relying on it.
• CometAPI: Query balance and usage — fetched November 4, 2026 UTC. Supports account/key usage and balance checks as reconciliation evidence; it does not prove a per-response bill without joining request identifiers.

These are current technical references for an evergreen operating method. They do not report a new release, account-specific price, completed background job, or measured saving. The deadlines, poll backoff, uncertainty treatment, reservation handling, and acceptance matrix are application recommendations derived from the documented contracts.