Last reviewed: 2026-08-08

Direct answer

An expired or cancelled batch is not a reason to replay the original input file. It is a reconciliation event. Freeze the original manifest, capture the provider’s final batch object, retrieve every available output and error artifact, and join those artifacts to the original rows using an immutable per-request identifier. Only after that join should you create a new manifest containing rows that are demonstrably unresolved or eligible for retry.

The safe unit of work is the request row, not the parent batch. OpenAI’s Batch API documentation describes a 24-hour completion window, unique custom_id values, output and error file identifiers, request counts, and states such as completed, expired, and cancelled. Anthropic’s Create a Message Batch reference likewise says a batch can take up to 24 hours and that unique custom IDs are needed because results can arrive out of order. Amazon Bedrock’s batch inference guide describes independent JSONL record processing and result retrieval from Amazon S3. These are different interfaces, but they support the same control principle: reconcile individual records before spending again.

Classify each original row as succeeded, provider_error, not_returned, cancelled_without_result, or validation_rejected. Treat a row as safe to exclude from a retry only when you have a matching successful result that passed your application checks, or a documented business decision that the result is intentionally not needed. Treat a missing or ambiguous row as unresolved and hold it for review. A new attempt should carry a new attempt identifier while retaining the original custom ID, so the ledger can show lineage without confusing a retry with a second business item.

Who this is for

This procedure is for platform and reliability engineers who run asynchronous evaluations, classifications, embeddings, or other large inference jobs; FinOps owners who need to explain a sudden second charge; and application teams that inherit a batch after an on-call handoff. It is especially useful when a job is close to its deadline, a cancellation is still propagating, or the provider returns results in a different order from the submitted file.

It assumes you can preserve the source JSONL or equivalent request manifest and can read the provider’s batch metadata and result artifacts. It does not require a particular SDK. The examples use harmless placeholders and omit access details. Adapt field names to your provider and gateway, but keep the evidence and decision boundaries intact.

Key takeaways

  • A parent state such as expired describes the batch lifecycle; it does not prove that every row failed or that no row was billed.
  • Generate a stable custom ID for every input row before submission. Never use line position as the only join key.
  • Preserve the original manifest, provider metadata, output file, error file, and usage ledger as an immutable evidence set.
  • Reconcile by row, then build a retry manifest from unresolved rows only. Do not copy the entire parent file into a new attempt.
  • Give each retry a new attempt ID and record the parent batch ID, original custom ID, and reason for retry.
  • Stop when the provider result, your application validation, or your cost ledger is ambiguous. A deliberate hold is cheaper than an untraceable duplicate run.

Sources checked

The OpenAI Batch API guide was checked for the documented 24-hour window, cost-discount description, unique request IDs, batch states, request counts, and output and error file fields. It also lists the asynchronous endpoints and explains that each input file targets one model.

The Anthropic Message Batches API reference was checked for the up-to-24-hour processing statement, the required unique custom_id, out-of-order result matching, and request-level parameters.

The Amazon Bedrock batch inference guide was checked for the JSONL-to-S3 workflow, independent record processing, job-state notifications, pricing and quota references, and the limitations on provisioned models, tool calling, and structured output. The links describe provider capabilities; the reconciliation controls in this article are an operator procedure built on those public details.

Contract details to verify

Before a resubmission, verify the following contract details against the exact provider, model, endpoint, and gateway configuration in use:

  1. Lifecycle window and terminal state. Record created, started, completed, expired, or cancellation timestamps when available. OpenAI documents a fixed 24h completion window and distinguishes cancelling from cancelled. Anthropic documents a maximum processing time of 24 hours. Do not invent a common deadline for another provider; use its current job contract.

  2. Row identity and cardinality. The source manifest must have one unique custom ID per row. Compare the number of submitted IDs with the number of distinct result IDs, error IDs, and unresolved IDs. A count that matches only after deduplication is a warning, not a pass. Preserve duplicate IDs as an input defect and correct the source rather than guessing which row a result belongs to.

  3. Artifact completeness. For OpenAI, capture the batch’s output and error file IDs and request counts before downloading content. For Bedrock, capture the input and output S3 locations and the job state. For Anthropic, capture the available batch result material and join its records by the documented unique custom ID. If an artifact is still being generated, keep the run in awaiting_evidence and do not submit a child attempt.

  4. Result acceptance. A provider response is not automatically a successful business result. Validate the expected schema, model version, content policy outcome, and downstream write status. Store a compact result digest or row fingerprint rather than sensitive prompt or response text. If your application rejected a syntactically valid response, classify it according to the retry policy and retain the rejection reason.

  5. Cost and usage treatment. The OpenAI guide advertises a 50% discount for Batch API requests, but that is not a universal promise for every provider or route. Reconcile the usage records and invoice or usage export available to your account. Mark cost as known, estimated, or pending; never mark an expired row as free merely because it has no output. A FinOps approver should be able to trace every cost line to a parent batch, row ID, and attempt.

  6. Feature compatibility. Bedrock batch inference processes records independently and does not support tool calling or response_format structured output. OpenAI lists supported endpoints and rejects some incompatible request settings. Check the current endpoint contract before transforming a retry file; changing the request shape can turn a retry into a different workload.

Failure modes

Resubmitting before the parent is final. A cancellation may still be in progress, or an output artifact may not yet be complete. The result is a race in which the original and retry both produce work. Use a state gate: resubmit_allowed is false until the parent is in a terminal state and all documented artifacts have been fetched.

Assuming expired means zero completed rows. A batch can reach a lifecycle deadline after some records have finished. If you replay every input, successful rows run twice. Join returned records to the frozen manifest and retry only the rows with no accepted result.

Joining by array position. Anthropic explicitly warns that results can be out of order, and file-based systems can reorder lines during processing. A positional join can attach one customer’s result to another customer’s request. Require a unique custom ID and reject any result whose ID is missing or duplicated.

Counting a provider error as a retry instruction. Some failures are permanent input or policy errors. Preserve the error category, apply a bounded retry policy, and route non-retriable rows to an exception queue. A retry manifest should contain a reason code, not just a Boolean flag.

Losing the original manifest. If the prompt, model, or request parameters change before reconciliation, you cannot prove what was processed. Store a content fingerprint, model and endpoint labels, row count, and creation timestamp with the manifest. Keep raw payloads in the provider’s approved storage boundary and keep operational logs sanitized.

Using unsupported features in a new batch. A transformation that adds tool calls or structured output may fail on a Bedrock batch even if the synchronous route accepted it. Compare the retry body with the documented batch contract and run a small validation sample before launching the full file.

Treating a cost estimate as an invoice. A missing usage field can reflect delayed reporting, not a zero charge. Mark the amount pending, record the evidence source, and set a review deadline. Close the exception only when the usage export or invoice agrees with the row-level reconciliation.

Use a log record that is sufficient for an audit without storing sensitive values or full content. For example:

{
  "run_id": "run_demo_2026_08_08",
  "provider": "provider_name",
  "parent_batch_id": "batch_demo_001",
  "attempt": 1,
  "custom_id": "item_00042",
  "provider_status": "expired",
  "row_outcome": "not_returned",
  "input_fingerprint": "sha256:[REDACTED]",
  "output_seen": false,
  "error_seen": false,
  "usage_input_tokens": 0,
  "usage_output_tokens": 0,
  "cost_state": "pending_reconciliation",
  "decision": "hold_for_review",
  "next_attempt_id": "attempt_2_pending"
}

The happy path is: snapshot the parent, fetch artifacts, produce a one-row-per-ID reconciliation report, have a second operator approve the unresolved set, generate a child JSONL, validate its IDs and model, submit it, and link the new batch ID back to the parent. The error path is: any missing artifact, duplicate ID, late state transition, or unexplained cost moves the run to hold_for_review; notify the owner, preserve evidence, and do not launch a broad replay. This distinction makes an exception visible instead of silently multiplying spend.

FAQ

Should I resend every row when a batch expires? No. Resend only rows that lack an accepted result and meet your retry policy. Successful rows should remain closed in the ledger.

Does an expired state mean the provider did not charge me? You cannot infer that from the state alone. Check the provider’s usage or billing evidence and label the amount pending until it is confirmed.

Can I reuse the original custom ID? Keep the original custom ID as the business-row key, but add a distinct attempt ID for the retry. This preserves lineage and prevents two attempts from being mistaken for one execution.

What if the output file is unavailable? Hold the run. Record the last observed state, artifact identifiers, and retrieval errors, then retry the artifact retrieval according to the provider’s documented availability. Do not guess that every row is unresolved.

How do I handle a provider error? Classify the error first. Retry transient or explicitly retriable categories within a bounded budget; correct invalid input or unsupported features before resubmitting; and escalate policy or safety errors instead of looping.

Where should the cost evidence live? Keep the immutable manifest and provider artifact references with the cost ledger. A related CometAPI cost and usage trace can help your team connect row outcomes to usage records without putting sensitive payloads in the article or log.

Reader next step

Take the next expired or cancelled run and create a reconciliation sheet before touching the retry button. Include the parent batch ID, frozen manifest fingerprint, one row for every custom ID, provider result or error reference, application acceptance decision, usage state, and retry reason. Have an approver sign off on the unresolved count. Then generate a child manifest containing only those approved rows and link its attempt ID to the parent. For the surrounding accounting workflow, use the CometAPI token-budget review cadence and keep the full UTM tuple on the link so the next review can be attributed.