Embedding models, generation, storage, and limitations

Overview

• An embedding model creates vectors; an embedding pipeline manages generation, versions, and storage.
• An embedding is a derived index representation, not a replacement for the original text.
• Different models, dimensions, or preprocessing choices can produce incompatible vector spaces.

Why this matters

Search must remain consistent as documents change and models are upgraded.

An everyday example

A library catalog must be rebuilt or migrated when its classification system changes.

A technical example

Store {chunkId, text, vector, embeddingModel, dimensions, contentHash, tenantId, documentVersion}.

How it works

1. Normalize and chunk
2. Hash content
3. Embed changed chunks in bounded batches
4. Validate dimension and finite values
5. Upsert
6. Publish a completed document version
7. Remove obsolete vectors.

Architecture and flow

Ingestion job → embedding adapter → versioned chunk store → index → query adapter using matching version.

The insurance assistant example

A revised policy creates a new indexed version. New answers should no longer cite withdrawn wording.

When to use it

Whenever similarity-based retrieval needs persistent, refreshable data.

When to avoid it

Do not repeatedly embed unchanged documents, compare vectors from unrelated models, or embed secrets without approved handling.

Interview answer

“I version the embedding model and corpus, retain source text and metadata, and use incremental re-embedding plus retrieval evaluation for upgrades.”

Benefits, trade-offs, and production considerations

• Embeddings can struggle with negation, precise numbers, rare identifiers, and domain language.

• “Covered” and “not covered” may appear close.

• Use keyword search, filters, reranking, or exact rules to compensate.

• Embeddings may leak information and require the same tenant isolation and deletion controls as source data.

• Raw float32 storage estimate: number of vectors × dimensions × 4 bytes.

• One million 768-dimensional vectors use about 3.1 GB for raw vector values alone; text, metadata, index structures, replicas, and backups add overhead.

• Estimate total database storage separately.