Last reviewed: August 4, 2026

Direct answer

Embedding dimension storage cost should be decided before a full index build, not after the vector database reaches its memory limit. Choose the smallest model-supported dimension that passes a retrieval test on your own documents and queries. Then assert that the API actually returned that dimension before loading any production index.

Keep two cost mechanisms separate. OpenAI’s checked documentation says its embedding requests are billed by input tokens. A shorter output vector therefore should not be assumed to reduce the embedding API charge. The same documentation says larger embeddings generally consume more downstream compute, memory, and storage, and it documents a dimension control for supported models. That downstream footprint is where dimension selection can materially change capacity requirements.

A useful first estimate is:

raw_vector_bytes = vector_count * dimensions * bytes_per_value
planning_memory = 2 * raw_vector_bytes

The second line is specifically Weaviate’s rule of thumb for holding an HNSW vector index in memory, not a universal hosted-service quote. For one million 768-dimension vectors stored as four-byte values, the raw vector payload is about 3.072 billion bytes; applying that rule gives about 6.144 billion bytes for planning. Moving to 1,536 dimensions doubles those figures before metadata, graph settings, replicas, backups, or vendor-specific overhead are considered.

The operator decision is therefore not simply “small vectors are cheaper.” It is: choose candidate dimensions, measure retrieval quality, verify the route contract, estimate the resulting resource envelope, and approve the smallest candidate that meets the quality and reliability gates.

Who this is for

This workflow is for teams building semantic search, recommendations, classification, or retrieval-augmented generation with embeddings. It is especially useful when a gateway such as CometAPI sits between the application and the model provider, because the team must verify both the model’s documented behavior and the behavior observed through its configured route.

It also gives FinOps owners a reviewable boundary between two ledgers: token-priced embedding requests and the memory, storage, and compute used by the resulting vector index. Platform engineers, search owners, and budget approvers should agree on that boundary before a corpus-wide import.

Key takeaways

  • A lower dimension can reduce vector memory and storage without automatically lowering a token-priced embedding request.
  • There is no safe universal dimension. Use the smallest candidate that passes a representative retrieval-quality gate.
  • Never infer success from the requested parameter alone. Record and validate the returned vector length.
  • Treat memory formulas as capacity estimates. Index type, vector representation, graph settings, compression, replicas, and implementation overhead can change the result.
  • Pilot on a bounded sample before a full build. A rejected pilot is inexpensive evidence; a rejected production index can require another corpus-wide embedding run.

Sources checked

  • CometAPI’s public text-embedding-3-small model page confirms that the model has a current public catalog page in CometAPI’s model offering. Use the live commercial contract, not an assumed historical rate, when building the budget.
  • OpenAI’s vector embeddings guide says embedding requests are billed by input tokens, shows usage metadata, documents default lengths of 1,536 for text-embedding-3-small and 3,072 for text-embedding-3-large, and describes supported dimension reduction.
  • Google’s Gemini embeddings documentation documents text and multimodal embedding workflows, distinguishes aggregation from using the Batch API for multiple embeddings, and emphasizes consistent task formatting for retrieval and other uses.
  • Weaviate’s resource-planning guide provides a memory-estimation rule, a worked float32 example, HNSW connection overhead considerations, and dimension reduction as a primary memory-control tactic.

Contract details to verify

Define the decision record

Before calling the API, freeze a small evaluation pack with representative documents, queries, and relevance judgments. Record a minimum quality score, a latency ceiling, and a maximum estimated index footprint. Candidate dimensions should come from the current model contract rather than arbitrary truncation.

For each candidate, record:

  1. The model identifier and configured gateway route.
  2. The requested and returned dimensions.
  3. Input-token usage reported by the response, when available.
  4. The vector representation assumed by the index, including bytes per value.
  5. The evaluation-set version and retrieval metric.
  6. Index settings that affect memory, including compression and HNSW configuration.
  7. The pricing source checked and the person who approved the budget.

Do not treat an upstream provider parameter as automatically supported by an intermediary route. Confirm that the configured CometAPI route accepts the requested field, returns the expected vector length, and preserves any required normalization behavior. If the public contract does not say, a bounded response test is the evidence.

Happy path

  1. Select a representative sample rather than the easiest documents in the corpus.
  2. Estimate sample input tokens and the full-corpus token envelope.
  3. Generate embeddings at two or more documented candidate dimensions.
  4. Reject any response whose vector length differs from the requested contract.
  5. Load each candidate into an isolated test index with production-like settings.
  6. Measure retrieval quality, import behavior, query latency, and estimated memory.
  7. Approve the smallest dimension that clears every required gate.
  8. Store the decision record with the model, dimension, evaluation version, and rollback condition.
  9. Start the full build only after the budget owner and search owner approve the same record.

In a successful pilot, the requested and returned dimensions match, token usage is captured, retrieval quality stays above the agreed floor, and the resource estimate fits the budget. Production monitoring then compares actual vector count and memory against the approved envelope.

Error path

If a request asks for 768 dimensions but returns 1,536, stop the import and mark a dimension-contract mismatch. Do not silently accept the default and revise the spreadsheet later. Check whether the model supports the requested setting, whether the gateway passes it through, and whether the response belongs to the expected model route.

If the API rejects the field, choose a documented dimension or another approved model; do not remove the field and continue an unbounded build. If the vector length is correct but retrieval quality fails, retain the higher-dimension candidate or revise the retrieval design and rerun the same evaluation pack. If reported usage or current billing units are unavailable, pause financial approval until request records can be reconciled with the cost ledger.

Log evidence without logging content

A sanitized operator event can look like this:

{
  "event": "embedding_dimension_trial",
  "request_id": "req-0427",
  "route": "cometapi",
  "model": "text-embedding-3-small",
  "sample_id": "eval-a",
  "requested_dimensions": 768,
  "returned_dimensions": 768,
  "input_tokens": 4821,
  "vector_count": 250,
  "vector_value_type": "float32",
  "estimated_raw_vector_bytes": 768000,
  "retrieval_recall_at_10": 0.92,
  "p95_query_ms": 41,
  "status": "accepted",
  "error_code": null
}

Keep document text, query text, personal data, and credentials out of cost logs. Store short internal identifiers, numeric usage, the requested contract, the observed response shape, evaluation results, and a normalized error code. That is enough to reproduce the budget decision without copying sensitive source content into the finance pipeline.

Failure modes

  • Assuming fewer dimensions means fewer billed input tokens. The checked OpenAI contract prices embedding usage by input tokens. Forecast API spend from the applicable billing unit and forecast index resources from vector size.
  • A route ignores or rejects the dimension control. Validate returned length on every pilot response and fail closed before bulk ingestion.
  • Mixed dimensions reach one index. Pin the accepted model-and-dimension contract in the index schema and quarantine mismatches instead of coercing them.
  • Manual truncation changes vector behavior. Prefer the model’s documented dimension control. OpenAI’s guide warns that manually changed dimensions require appropriate normalization.
  • The benchmark is unrepresentative. A public benchmark is useful context, but approval should depend on the team’s own documents, queries, relevance judgments, and error tolerance.
  • Task formatting drifts. Google’s documentation stresses consistent task formatting. Pin formatting rules with the model contract and include them in regression tests.
  • The memory forecast counts only vector values. Add index, connection, metadata, import, replication, and service-specific overhead. Weaviate’s guide notes that HNSW connections and garbage collection affect memory beyond raw vectors.
  • Compression is treated as free capacity. Compression can trade memory for recall. Test compressed and uncompressed candidates against the same quality gate.
  • A model or price changes after approval. Pin the model identifier, date the source check, and require a new pilot when the model contract, dimension behavior, vector representation, or billing unit changes.
  • Retries create duplicate work. Use idempotent chunk identifiers and reconcile completed vectors before resubmitting a failed bulk job.

FAQ

Does a smaller embedding dimension lower the CometAPI API bill?

Do not assume it does. The checked OpenAI documentation says its embedding requests are billed by input tokens, while vector dimensions affect the returned representation and downstream resource use. The public CometAPI page confirms a model listing, but dimension-based billing is not established by the evidence reviewed here. Verify the current CometAPI billing unit and route behavior before approval.

What embedding dimension should a team choose?

Choose the smallest model-supported dimension that meets the team’s measured retrieval-quality, latency, and compatibility requirements. Start with a representative evaluation pack and at least two candidates. A dimension that works for one corpus or task is not automatically safe for another.

Can dimensions be changed after an index is built?

Treat a dimension change as an index migration unless the vector-store contract explicitly supports another method. Build an isolated candidate index, re-embed a bounded sample, compare retrieval results, and plan rollback before replacing production data. Never mix vectors of unverified lengths in the same fixed-dimension index.

Is the two-times-memory formula universal?

No. It is a Weaviate HNSW planning rule of thumb from the checked resource guide. It is useful for an early envelope, not a substitute for the configured vector store’s own calculator, index settings, replication policy, storage pricing, and observed import behavior.

Should dimension reduction or compression come first?

Test the model-supported dimension candidates first because dimension determines every vector’s basic shape. Then test compression as a separate index decision. Keep the evaluation set fixed so the team can see which change affected recall, latency, and memory.

What should trigger a new dimension review?

Review again when the model changes, the corpus changes materially, task formatting changes, retrieval quality falls, index settings change, or actual memory departs from the approved envelope. A price change alone may alter the budget without changing the quality decision, but it still requires a refreshed cost record.

Reader next step

Start with a small, representative corpus slice and compare two documented dimensions. Before running it, count input tokens before approving the embedding budget . Put the resulting token estimate, vector-memory estimate, retrieval score, and query latency into a single table, then build a unit-cost scorecard for the workload .

Record where the current model and billing facts came from, and place them on a defined pricing-source refresh cadence . The approval gate should require matching requested and returned dimensions, a passing retrieval test, and an index estimate within budget.

When that contract and pilot plan are ready, Start with CometAPI .