Pricing-doc incident review for CometAPI

Last reviewed: 2026-05-08

Who this is for: platform owners, FinOps analysts, and on-call engineers who need to investigate a CometAPI spend anomaly without turning the review into a generic API smoke test.

This draft focuses on one narrow failure mode: an incident where a team’s pricing assumptions may no longer match the pricing documentation, local configuration, or request mix. For broader cost-control navigation, start at /sites/ai-cost-controls/ or browse related operational notes in /sites/ai-cost-controls/posts/.

Key takeaways

• Treat the CometAPI pricing documentation as an incident artifact, not just a reference page.
• Capture the exact documentation URL, access timestamp, model identifiers, request sample, and internal cost formula used during the affected window.
• Do not assume endpoint paths, auth headers, request fields, response fields, rate limits, or billing mechanics from a pricing page unless the source explicitly states them.
• Reconcile three ledgers separately: documentation assumptions, application telemetry, and provider-side usage or billing exports.
• Numerical thresholds in this checklist are examples to tune for your workload; they are not universal guarantees.

Concise definition

A pricing-documentation incident is a cost-control incident where the suspected cause is a mismatch between what operators believe an API call should cost and the pricing or billing evidence available at review time. The mismatch may come from stale internal docs, a changed model route, incorrect token accounting, missing usage fields, currency or unit confusion, or undocumented assumptions in a budget calculator.

Evidence to pin before analysis

For this review, the public evidence URL is the CometAPI pricing documentation page: CometAPI About Pricing.

Before assigning a cause, save the following into the incident record:

1. Pricing documentation URL: https://apidoc.cometapi.com/about-pricing-873788m0
2. Access date: 2026-05-08
3. Screenshot or PDF capture of the visible pricing page, if your compliance process allows it.
4. Internal calculator version, spreadsheet version, or code commit used to estimate cost.
5. Affected model names as logged by the application, not as remembered by the team.
6. Request IDs, timestamps, tenant or project IDs, and environment names.
7. Token usage fields returned by the API, if present in responses.
8. Local retries, fallbacks, cache misses, batch jobs, or streaming behaviors during the window.
9. Any support ticket, dashboard export, or invoice export used to validate actual spend.

Incident-review checklist

1. Freeze the review window

Pick a bounded interval before querying data. A useful starting window is:

• incident_start: first alert, invoice anomaly, or deployment linked to the spend change
• incident_end: first time cost returned to expected range, or current time if still active
• comparison_window: same weekday/hour pattern from the prior week or prior deployment version

Treat these as working intervals. Adjust them if the evidence shows a delayed batch job, retry storm, or backfill.

2. Separate pricing-doc drift from traffic drift

A spend increase does not automatically mean pricing changed. Build two columns:

3. Rebuild the cost estimate from raw usage

Do not start with the invoice total. Rebuild from the lowest-level data you have:

1. Group requests by exact logged model.
2. Sum input and output tokens separately where usage fields are available.
3. Keep failed, timed-out, retried, and canceled requests in separate buckets.
4. Apply the pricing assumption your team used at the time of the incident.
5. Apply the pricing assumption visible in the checked CometAPI pricing documentation.
6. Compare both estimates with provider-side usage or billing export.

If you cannot retrieve token usage for a sample, mark the estimate as incomplete rather than filling values from averages.

4. Look for documentation-to-code translation errors

Pricing pages are often read by humans and translated into code, spreadsheets, alerts, or budget policies. Review these translation points:

• Unit mismatch: per token, per 1K tokens, per 1M tokens, request-based, or other units.
• Direction mismatch: input versus output pricing.
• Model-name mismatch: display name versus API model identifier.
• Currency mismatch: displayed currency versus internal ledger currency.
• Rounding mismatch: per request, per batch, daily aggregate, or invoice aggregate.
• Environment mismatch: staging keys accidentally using production budget rules.
• Feature mismatch: streaming, caching, batch mode, or tool calls handled differently than expected.

Only mark an item as verified if you can point to a source artifact: pricing documentation, API response, dashboard export, invoice export, or internal code.

Contract details to verify

This table intentionally distinguishes what the pricing documentation supports from what must be verified elsewhere. The source provided for this draft is the CometAPI pricing documentation page, so endpoint and transport details should not be inferred unless that page explicitly states them.

Sanitized pricing-check example

Use a small, redacted request sample to verify that telemetry captures the model and usage fields your cost calculator needs. Replace placeholders with your actual endpoint and model after checking the current API reference.

curl -sS "$COMETAPI_BASE_URL/<verify-endpoint-path>" \
  -H "Authorization: Bearer $COMETAPI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<logged-model-id-from-incident>",
    "messages": [
      {
        "role": "user",
        "content": "Return one sentence confirming this pricing telemetry check."
      }
    ],
    "max_tokens": 32,
    "stream": false
  }'

Validation goal:

• Confirm the application logs the exact model identifier sent.
• Confirm whether the response includes usage fields.
• Confirm whether your telemetry pipeline stores those fields.
• Confirm whether your pricing calculator maps that model identifier to the same unit assumptions documented in the reviewed pricing source.
• Do not use this example to infer rate limits, availability, or final billing behavior.

Practical validation steps

Step 1: Create an evidence ledger

Create a table in the incident document with these columns:

Step 2: Validate model mapping

For each affected model:

1. Pull the exact model string from request logs.
2. Pull the model string used in the cost calculator.
3. Pull the model label from the pricing documentation or billing export.
4. Mark the mapping as:
   • exact
   • alias
   • fallback
   • unknown
5. Escalate any alias, fallback, or unknown mapping before closing the incident.

This catches a common operational error: a budget rule that looks correct for one model family but is applied to a different routed model.

Step 3: Validate usage capture

Sample at least a small set of requests from the incident window and comparison window. For each request, record:

• request timestamp
• model identifier
• endpoint path
• response status
• input token count, if available
• output token count, if available
• retry count
• fallback route, if any
• tenant or project
• estimated cost under old assumption
• estimated cost under reviewed assumption

If usage fields are missing from logs, the incident finding should say that cost attribution is limited by observability gaps.

Step 4: Compare estimates against billing evidence

Use three totals:

Do not force these to match exactly unless the billing contract defines rounding, timing, and inclusion rules. Instead, classify the difference:

• expected rounding or timing gap
• missing telemetry
• model mapping error
• retry or fallback amplification
• stale pricing assumption
• unresolved provider-side discrepancy

Step 5: Record a corrective control

Close the review with one control that prevents recurrence. Examples:

• Add a pricing-documentation capture step to every monthly budget review.
• Require model IDs in logs before enabling a new route.
• Add an alert when output-token share changes materially.
• Add a calculator test that fails when a model has no pricing mapping.
• Add a finance review for aliases, fallbacks, or batch routes.
• Link this incident from the site editorial trail at /sites/ai-cost-controls/editorial/.

Incident notes template

Use this compact template when the issue is active:

What not to conclude from this review alone

Avoid these conclusions unless you have additional evidence:

• That current pricing is unchanged from a previous capture.
• That a model is available in every account or region.
• That a request failure is or is not billable.
• That a rate-limit response guarantees no billing event.
• That a pricing page defines all endpoint paths or request schemas.
• That a small test request predicts production invoice behavior.

The pricing documentation is necessary evidence, but it is not a substitute for contract terms, billing exports, application telemetry, or endpoint-specific API documentation.

FAQ

Is this a pricing reconciliation procedure?

Not exactly. Reconciliation usually starts from invoice totals and works backward. This review starts from documentation and operational assumptions, then checks whether the application, calculator, and billing evidence agree.

Should we archive the pricing page every time?

For active incidents and formal monthly reviews, yes. A timestamped capture helps reviewers understand which pricing source was used. Follow your organization’s legal and compliance rules for storing third-party documentation.

What if our logs do not include token usage?

Record that as a finding. You can still compare request volume and model mix, but token-based cost attribution will be incomplete. Add usage-field capture to the corrective actions if the API response provides the necessary fields.

Can we rely on a single test request?

No. A single sanitized request is useful for validating telemetry shape, model logging, and response fields. It should not be used as proof of production billing, rate limits, or final invoice behavior.

How often should pricing assumptions be reviewed?

Set a cadence that matches your spend risk. Monthly may be enough for low-volume systems; higher-spend or multi-model systems may need checks after deployments, route changes, or vendor documentation updates.

Where should internal follow-up pages live?

Use the site’s internal index and editorial trail so operators can find the current runbook: /sites/ai-cost-controls/posts/ and /sites/ai-cost-controls/editorial/.

Sources checked