Last reviewed: 2026-08-08

Direct answer

Structured output schema token costs should be treated as a measurable request contract, not estimated from schema characters or assumed to behave identically across providers. Freeze one representative request, route, model, and generation configuration. Run the current schema and proposed schema through the available provider counting interface, then send controlled requests and compare observed usage, validation results, and errors. Approve the change only when the evidence is explainable and remains within thresholds your team set before the test.

This approach is grounded in provider-side measurement. OpenAI’s token-counting guide says its counting endpoint accepts the same input shape as the Responses API, includes request-formatting tokens, and handles tools and schemas that are difficult to count locally. Google’s token guide says API cost is determined in part by input and output tokens, documents a preflight input count, and lists observed usage fields for input, output, thinking, cached content, and tool use. Google’s structured-output guide shows that a JSON Schema is supplied as part of structured-output configuration. Anthropic also exposes a public Count tokens in a Message API reference , which should be treated as its own provider contract rather than assumed to match another API.

Use two deltas, not one vague cost estimate:

input_token_delta = candidate_input_tokens - baseline_input_tokens
delta_percent = input_token_delta / baseline_input_tokens * 100

Keep token deltas separate from currency until the current price and billing unit for the exact model and route have been verified. A token regression can be measured from the supplied sources; a currency estimate needs a separate, current price record.

A concrete happy-path workflow is:

  1. Select a sanitized fixture that represents a common production request. Keep its prompt, system instruction, tools, files, and generation settings unchanged throughout the test.
  2. Assign immutable identifiers to the current and candidate schemas. Store the complete schemas in version control, but use a short version and hash in operational logs.
  3. Count the full current request with the provider’s supported counting method. Record whether the count is exact, estimated, input-only, or unavailable.
  4. Count the candidate request without changing any other field. Calculate the absolute and percentage input-token deltas.
  5. Send a bounded, low-volume test for each schema through the same model route. Capture returned usage fields, HTTP status, validation outcome, and retry count.
  6. Validate both JSON shape and required business rules. Structured output can make the shape predictable, but the application must still test whether values are usable.
  7. Pass the gate only when the candidate is accepted by the target route, produces valid output, stays within the predeclared token threshold, and has no unexplained usage difference.
  8. Attach the result to the normal budget change-control workflow rather than approving a schema from an isolated developer test.

The error path is equally important. If the provider counter cannot represent the schema-bearing request, mark the test observed_only; do not substitute a local character estimate and call it exact. If the request is rejected, usage is missing, or preflight and observed counts cannot be reconciled, mark the result inconclusive, stop automated promotion, and preserve a sanitized error record. If validation fails, fix the schema or application contract before retrying. An unbounded retry loop turns a schema defect into additional API traffic.

For a broader preflight pattern, pair this gate with the CometAPI token-count control .

Who this is for

This control is for AI platform engineers who maintain shared model routes, FinOps practitioners who need defensible token forecasts, and product or API owners who version JSON schemas for extraction, classification, or agent workflows. It is especially useful when the same logical operation can move among provider models behind a CometAPI-routed integration.

It also helps reviewers who do not write prompts but must approve a change. A compact evidence record lets them see which variables were frozen, how the schema changed, what the provider counted, what the completed request reported, and why the result passed or failed.

This is not a universal conversion formula for schema bytes, tokens, or currency. Provider APIs expose different counting and usage contracts. The purpose of the gate is to measure each supported route and retain enough evidence to notice a regression later.

Key takeaways

  • Test the complete request, because formatting, tools, schemas, system instructions, and other request structure can affect provider-reported tokens.
  • Compare schema versions on the same route and model; otherwise the result mixes schema change with model or tokenizer change.
  • Record preflight counts and observed usage separately. They answer related but different questions.
  • Gate on schema acceptance, token delta, output validation, and retry behavior rather than token count alone.
  • Set both absolute and percentage thresholds before running the test so the approval rule cannot be moved to fit the result.
  • Keep price evidence separate and current. Do not turn a valid token delta into a currency claim using an unverified rate.

Sources checked

These sources establish counting and request-contract mechanics. They do not establish one cross-provider price, one universal schema overhead multiplier, or a safe threshold for every workload. Those values belong in the operator’s tested contract.

Contract details to verify

Write the following details into the gate definition before collecting results:

  • Route and model: Record the exact provider route and model identifier used for both versions. A friendly model family name is not precise enough for a reproducible comparison.
  • API surface: Record whether the workload uses a response format, tool definition, or another schema-bearing field. Do not assume two surfaces serialize schema identically.
  • Fixture identity: Hash the sanitized fixture and record its revision. Prompts, system instructions, tools, files, and generation controls must remain fixed.
  • Schema identity: Record schema version, hash, nesting depth, property count, required-field count, and whether descriptions changed. These are diagnostic dimensions, not substitutes for provider counts.
  • Measurement semantics: Label a result as preflight_and_observed, preflight_only, or observed_only. Also record whether the preflight method accepts the complete request shape.
  • Usage mapping: Preserve the provider’s raw usage categories in a restricted record while mapping them to normalized input, output, cached, thinking, and tool-use fields where available. Do not invent zeroes for absent fields.
  • Validation contract: Test JSON parsing, schema conformance, required values, enum membership, and application-specific rules. Record pass or fail separately from token usage.
  • Failure policy: Define which errors stop promotion, whether one bounded retry is allowed, and who can approve an exception.
  • Price evidence: Store the price source, unit, currency, effective date, and reviewer separately from the token test. Recalculate currency when that evidence changes.

A sanitized event can look like this:

{
  "event": "schema_cost_gate",
  "request_id": "req-7f31",
  "route_id": "route-a",
  "provider": "provider-a",
  "model": "model-a",
  "schema_version": "v3",
  "schema_hash": "a91c72d4",
  "measurement_mode": "preflight_and_observed",
  "baseline_input_tokens": 780,
  "candidate_input_tokens": 842,
  "input_token_delta": 62,
  "delta_percent": 7.95,
  "output_tokens": 118,
  "tool_use_tokens": null,
  "http_status": 200,
  "schema_validation": "pass",
  "retry_count": 0,
  "prompt_body_logged": false
}

Do not log request headers, complete prompts, user content, complete generated output, or unrestricted schema descriptions. Those fields are unnecessary for a cost gate and may contain sensitive material. Keep the full fixture and schema in controlled storage; the operational event needs only stable identifiers and measurements. The token-usage evidence guide can help turn these fields into a reviewable record.

Failure modes

  • Only the schema file is counted. The production request also contains message boundaries, instructions, tools, and other structure. OpenAI explicitly notes that formatting tokens may not appear in locally tokenized text. Test the complete request whenever the counting interface permits it.
  • The model changes between tests. A different model can change tokenization or usage categories. A comparison with two moving variables cannot isolate schema overhead.
  • Preflight is treated as the invoice. A preflight count estimates or reports request size under a documented contract. Observed usage is the evidence returned for the completed operation. Store both and investigate differences.
  • Missing usage becomes zero. A missing field means unknown unless the provider contract says otherwise. Converting unknown to zero hides regressions and corrupts forecasts.
  • Schema portability is assumed. A JSON Schema accepted by one route may use a shape or configuration that another route handles differently. Run acceptance and validation tests on every approved route.
  • Descriptions grow without review. Long descriptions may be useful to a model, but they are still part of the schema-bearing request surface. Measure the candidate rather than assuming descriptive text is free.
  • Validation errors trigger automatic retries. Repeating the same invalid contract creates more requests without improving the outcome. Stop, record the validation class, and repair the schema or parser.
  • Output variability obscures the input regression. Compare input-token deltas separately from output-token results. Then examine output distribution with a bounded representative set.
  • The log captures sensitive payloads. Cost evidence does not require raw user content. Use hashes, versions, counts, statuses, and validation classes.
  • A percentage-only threshold distorts decisions. A small baseline can produce a large percentage from a minor absolute change, while a large baseline can hide a costly absolute increase. Require both tests.

FAQ

Do all structured-output schema tokens become billable input tokens?

The supplied sources do not establish one universal rule for every provider and API surface. OpenAI says tools and schemas can add tokens handled by its count endpoint, while Google says API cost is determined in part by token usage and exposes both preflight and observed measurements. Verify the exact route, count the supported request shape, and compare returned usage before making a billing claim.

Can a local tokenizer replace the provider count?

Not for this gate when a provider-side count is available. OpenAI documents limitations for local tokenizers, including request formatting, tools, schemas, images, files, and model-specific behavior. A local estimate can be retained as a diagnostic, but label it clearly and do not present it as provider-reported usage.

What if preflight and observed input usage disagree?

First verify that the same model, payload, schema version, and API surface were used. Then inspect whether the preflight interface covered the full request and whether observed usage separates cached, tool-use, thinking, or other categories. If the difference remains unexplained, fail the gate and escalate the contract review instead of choosing the lower number.

Should the gate use tokens or currency?

Use tokens as the primary regression signal because they can be compared directly from the test. Add currency only after joining the result to a current, reviewed price record for the exact route and unit. Keep both values so a later price change does not erase the original technical evidence.

Does valid structured output remove application validation?

No. A schema-shaped response can still contain an unusable identifier, an impossible date, or a classification that violates business rules. Validate the JSON contract and domain rules separately, and retain both outcomes in the gate result.

What threshold should a team choose?

There is no source-backed universal threshold. Choose an absolute token ceiling and a percentage ceiling from workload volume, unit economics, latency constraints, and error tolerance. Declare them before the test. Any exception should identify an owner, reason, expiry, and compensating control.

When should the gate run again?

Run it when the schema, model, route, API surface, system instruction, tool definition, or token-accounting contract changes. A scheduled spot check can also detect provider-side behavior changes, but it should use the same frozen fixture so results remain comparable.

Reader next step

Choose one high-volume structured-output operation and create a sanitized, versioned fixture. Measure the current schema, measure one proposed change, send a bounded test through the target route, and store the sanitized event alongside the change record. If any count is missing or unexplained, hold the change rather than filling the gap with an estimate.

Then apply the same fixture to each model route approved for that operation. This turns schema cost from a hidden prompt detail into a repeatable release gate with an owner, evidence, and an explicit failure path.

When you are ready to compare provider routes behind one access path, Start with CometAPI .