An encryption test is flaky: it searches ciphertext for a 3-character string

Type

Bug — in the test, not in the product

Found while

SAPE-17

Triage

Defer

Status

Open

Severity

Medium — a red build on a security-shaped assertion, for no reason

Description

TestAnEncryptedDatabaseKeepsItsDocumentsOffTheDisk (internal/store/dump_test.go:417) writes 50 documents to an encrypted database, takes the durable image, and asserts that six plaintext strings do not appear anywhere in it:

// dump_test.go:448
for _, text := range []string{"Secret number 7", "author", "ann", "articles", "by_author", "s07"} {
	if bytes.Contains(image, []byte(text)) {
		t.Errorf("the file holds %q in the open", text)
	}
}

Two of the six are three characters: "ann" and "s07". In a buffer of random bytes, a given three-byte sequence turns up by chance often enough to matter, and the page nonces come from crypto/rand, not from the simulated disk's seed — so the ciphertext differs on every run even though vfs.NewSim(66, …) looks deterministic.

Evidence

• Measured, not estimated. On this machine, 2026-09-20: go test -run 'TestAnEncryptedDatabaseKeepsItsDocumentsOffTheDisk$' -count=2000 ./internal/store/ → 13 failures in 2000 runs (≈ 0.65%).
• Every failure was one of the two short needles. Both fire; the counts were split between "ann" and "s07". Neither of the four longer strings failed once.
• A first run of -count=200 passed clean, which is the point: at this rate a developer will see it perhaps once a month and read it as noise, and CI will see it about once in every 150 runs of that package.
• The four longer needles are what actually carry the assertion, and they cost nothing — "Secret number 7" is 15 bytes and will never collide.

Triage

Does it lose data, break a promise on a public surface, or stop somebody installing and running? No, to all three. The product is correct: the encryption works, and the test is the thing that is wrong. Nobody installing sapedb runs this.

Defer. It is worth saying why this feels wrong and still is not a 1.0.0 item: the assertion it flakes on is a security claim, and a security test that goes red at random is the kind people learn to re-run. But the triage question is about the shipped artifact, and by that question this does not qualify. The fix is also a two-line edit that can land any time, which is an argument for doing it soon — not for making it a release gate.

If deferred, what it costs to wait

• The real cost is behavioural, and it accrues immediately: the first few times this goes red on a change that had nothing to do with encryption, somebody re-runs and moves on. Once that habit exists, a genuine failure of this assertion looks exactly like the noise.
• It does not get more expensive to fix — it is unrelated to any frozen surface. Drop the two three-character needles, or make the short ones long enough to be collision-proof ("ann" → the full author value, "s07" → the full slug with its field name).
• A better version of the fix, if anybody is in there anyway: assert on the indexed values with enough length to be meaningful, and state in the test why length is the parameter that matters, so nobody reintroduces a short needle for readability.