A container limited to 512 MiB with swap disabled
(mem_limit and memswap_limit equal, read back off the running container
with docker inspect: 536870912 for both). The workload is the mistake an application
actually makes — an operation declared once with a limit far above what the collection held at the
time, invoked again after the collection grew. Rows of 2 KB.
| rows asked for | ≈ row bytes | what the server did |
|---|---|---|
| 5,000 | 9.8 MiB | answered with 5,000 rows |
| 10,000 | 19.5 MiB | hung up; still serving afterwards |
| 15,000 – 40,000 | 29 – 78 MiB | same: hung up, still serving |
| 45,000 | 87.9 MiB | did not come back; the server was gone |
After the last rung, docker inspect reported status exited, exit code
137, OOMKilled true, and RestartCount 0 — the restart
count is recorded because the verdict “it only hung up” would be a lie if anything had
brought the process back. Peak memory sampled from the host during that phase was
352.2 MiB over 31 samples, and that is a lower bound: sampling is about one
second apart, and the moments before an OOM kill are exactly the ones most likely to fall between
two samples.
protocol.MaxPayload is 16 MiB and it is enforced — that is the
hung up row, and it is why SAPE-31 was corrected. But look at
where the check sits:
func Encode(frame Frame) ([]byte, error) {
if len(frame.Payload) > MaxPayload {
return nil, fmt.Errorf("%w: %d bytes", ErrPayloadTooLarge, len(frame.Payload))
}
The cap is a check on len of a byte slice that already exists. For it to refuse, the
server must first have read every row into memory and marshalled all of them. The cap does not
bound the work; it reports on work already done. At 45,000 rows the process reached the memory
limit while building the payload it was about to be told it could not send.
The consequence is the one that matters on the target machine: steady-state memory for this workload never left 16–20 MiB on any of the three profiles. The store is not what fills a small VPS. One unbounded read is.
Measured by reading the three paths that answer a caller, not by assuming they are alike:
Explore — the operator shell — is capped.
internal/store/explore.go defines MostRows = 1000 and
asOperation applies it whether or not the caller asked for a limit:
if operation.Limit <= 0 || operation.Limit > MostRows { operation.Limit = MostRows }.
At 2 KB rows that is about 2 MB. The ad-hoc path, the one with no declaration behind it,
is the one that cannot hurt the server.Subscribe streams. internal/server/subscribe.go
reads at most batchSize = 512 changes while holding the database, releases it, and
then sends one frame per change — not one frame holding all of them. A
follower can replay a log of any length and no payload is ever larger than a single change.Invoke has neither. db.read(...) returns a result and
the handler calls json.Marshal(result) on the whole of it. Nothing between the
declaration's limit and the built payload puts a number on how large that may get.So the uncomfortable half of this is where the gap sits. The exploratory path, which nobody wrote down in advance, is protected by a ceiling the server owns. The declared path, which this product's entire argument rests on — cost written down before anything runs — is the one with no ceiling of its own. A declaration asking for a million rows is legal, and the server will try to honour it.
That is worth stating precisely rather than as an indictment: the declared limit is a promise about the operation, not a budget the server holds itself to. Those two things read the same in a document and behave differently the day a collection outgrows what its author imagined — which is exactly the workload that produced the measurement above. It is the same shape as W7 one layer in: there, a signature and a sandbox proved everything except what a call cost.
Neither half of the fix has to be invented. A server-owned ceiling exists, in
explore.go. Streaming a large answer exists, in subscribe.go. Both were
written for other reasons by people who did not have this ticket in front of them.
Measured 2026-09-21 on 737e149 (the image reported 737e149-dirty, because
the first run of a benchmark cannot be made from a tree that does not yet contain the benchmark;
the server image copies only go.mod, go.sum, internal/ and
cmd/, none of which the benchmark touched). macOS 26.3.1 on arm64, Docker 29.6.1,
Docker Desktop. Plaintext TCP, encryption at rest off, one connection with one request outstanding
at a time. Docker Desktop on macOS is a Linux VM whose disk behaviour is not a VPS's
— that caveat bites the write figures hardest, but the wall itself is a memory limit, and a memory
limit in a VM is still a memory limit.
The wall was only run on the 512 MiB profile. The row count at which the 1 GiB and 2 GiB profiles die is not measured. It can be inferred from ~88 MiB of rows costing ~352 MiB of process, but an inference is not a measurement and this page does not record one as if it were.