← 1.0.0 scope SAPE-36

Measure this store under the limits it has to live in

Status
Done
Shipped in
1.0.0 — 04f877e, corrected by f145063
Component
release
Blocked by
Blocks

Why this ticket exists at all

The owner asked for containers that measure sapedb under a small VPS's limits. Until then, this store had never been benchmarked — the site's own “nothing is sold on speed” convention was true in the strongest sense: there were no numbers. It is recorded here rather than left as a commit nobody can see, because a board that only carries what fits a pre-written scope is a board that stops describing the work.

What it is

./bench/run.sh builds the server image from the tree it sits in, runs three profiles — 1 cpu/512m, 1 cpu/1g, 2 cpus/2g — drives them over the wire with this repository's own client, samples the server container's memory from outside, and writes a report plus the raw records. Limits are read back off the running container with docker inspect rather than copied from the compose file: a limit that did not apply is the easiest way for a run like this to be worthless.

memswap_limit equals mem_limit, and that line is the difference between a benchmark and a decoration. Without it the container swaps instead of hitting the limit, every number comes back healthy, and the machine actually worried about — a small VPS whose real failure is running out of memory — is the one thing never measured. It is written as the same variable, not a copy of its value, so the two cannot drift apart in an edit.

What the first run found

Steady-state memory never left 16–20 MiB on any profile, 512m included. The store is not what fills a small machine. One unbounded read is: a scan whose declared limit was far above what the collection held took the process to 352 MiB and it was OOM-killed — exit 137, OOMKilled true, RestartCount 0. That became ISS-37 and ISS-38.

And the number SAPE-32 was built against: one declared delete per row decays from 1097 to 304 rows/s over 25,000 deletes — a factor of 3.5, and the same curve on all three profiles, which is what makes it a property of the store rather than of the afternoon.

The correction, because it matters more than the numbers

The generated report told the reader to “read the ratios between profiles”. For insert that is the one ratio this rig cannot support: the profiles are measured in turn on a host nothing holds still, so position in the run and profile identity are the same variable. The giveaway is in the numbers themselves — insert gets slower as the limits get larger, which is not something more cpu and more memory can do.

bench/README.md said so; the report did not, and the report is the document somebody opens. The warning is now emitted by the generator directly above the insert table, so it cannot be separated from the numbers it qualifies. Scan and delete keep the cross-profile comparison, and for a stated reason: there the three profiles agree to within a few percent, and agreement across separated runs is evidence rather than coincidence.

What it does not claim

Acceptance criteria

  1. Three constrained profiles, limits read back off the running container rather than assumed — met, and swap is confirmed disabled on each.
  2. Every number carries the commit, the limits, the host and the range it was measured over — met; the raw records are committed beside the report.
  3. The benchmark does not run in the normal test suite — met: the load generator is package main needing a live server, and there is no func Benchmark in the tree.
  4. The report states which comparisons it supports and which it does not — met, after f145063 corrected it.