A vector stored as JSON text is five times its size, and read one number at a time

Status

To do

Component

server

Priority

High — decides whether a RAG corpus fits at all

Blocked by

SAPE-37 — its correctness tests are this ticket's guard

Blocks

—

Why it exists

The owner's vector requirement is specifically a vector database that can serve RAG. RAG corpora are large: tens of thousands to millions of chunks, each carrying an embedding of 384, 768 or 1536 dimensions. Today a vector can only be stored the way every value is — as a JSON array in the document body, with every element a number, which is a float64.

What that costs, measured

Realistic embedding values (small floats around zero, produced as float32 by the model and widened to float64 on the way in), encoded the way a body stores them:

dimension	as JSON today	float32 binary	ratio
384	8,040 B	1,536 B	5.23×
768	16,123 B	3,072 B	5.25×
1536	32,129 B	6,144 B	5.23×

At corpus scale, for the vectors alone and before a single byte of chunk text: a million chunks at 1536 dimensions is 30 GiB as JSON, against 5.7 GiB as float32. The earlier benchmark on this board measured this store on a 512 MiB machine. The first number does not fit on the disk of the machine this product targets; the second does not fit in its memory either, which is a separate problem, but it is a problem of a different size.

And the size is not the worst of it. A distance computation during a scan must turn that text back into numbers — 1536 of them, per row, on every search. Exact search (SAPE-37) walks every row, so it pays that parse for every row it walks.

One detail worth knowing: a float32 value widened to float64 prints with the full precision of the wider type, so 0.05 as produced by a model can arrive as 0.04999999701976776. The text is not only large; it is large carrying digits that were never in the embedding.

What closing it means

• A vector is stored as a vector: a declared dimension and float32 elements, in a compact binary form, not as text.
• The answer does not change. SAPE-37's tests compute the expected top-k from outside the package; they are the guard here. If a search returns different rows after the storage changes, the change is wrong, not the test.
• Precision is stated, not implied. Storing float32 is a narrowing from today's float64, and ISS-35 exists precisely because a narrowing that happens silently is data loss. For an embedding the narrowing restores the value the model actually produced — but that has to be a declared property of a vector field, not a surprise.

Where it meets the type system

This is one entry of SAPE-29's list, decided by the owner's requirement rather than by argument: a RAG-capable vector store needs a vector type. It does not settle the rest of that list. It is also close kin to SAPE-31 — a vector is, underneath, a fixed-length run of bytes — and whichever lands first should shape the other.

Why it waits for SAPE-37

Correctness first, then compactness. Exact search is being built over the JSON form, which is slow and large but correct, and its tests check the answer against a brute-force top-k computed outside the package. Changing the storage underneath an answer that is already pinned is a safe change. Changing both at once would leave nothing to say which of the two broke.