The covering index is built, decoded per entry, then discarded

Type

Debt

Found while

SAPE-20

Triage

Defer

Status

Open

Severity

Low — wasted work on every scan, never a wrong answer

Description

An index may be declared to carry fields (Index.Include). The write path builds that payload for every entry: Collection.include (internal/store/collection.go:489) marshals the carried fields into the entry's value. The read path decodes it — Collection.Scan (internal/store/scan.go:137-141) unmarshals it into Found.Include for every entry it visits — and then the only caller that matters throws it away.

internal/store/invoke.go:246-253, the scan branch of a declared operation:

return collection.Scan(operation.Index, within, func(entry Found) bool {
	document, found, err := collection.Get(entry.Key)
	...
	return visit(entry.Key, document)
})

So the cost is paid three times — written on every put, decoded on every entry of every scan, and then a full document fetch anyway — and the saving it exists for is never taken.

Evidence

• grep -rn '\.Include' --include='*.go' finds entry.Include read in exactly one place: internal/store/store_test.go:238, a test. The other hits are index.Include, which is the declaration, not the entry: spec.go:232,248,256-257, collection.go:490,495, internal/cli/complete.go:137.
• No server path reads it. invoke.go is the only route from a declared operation to Scan, and it calls collection.Get(entry.Key) unconditionally.
• The promise is in the type's own doc, internal/store/scan.go:110-112: "Include is the fields the index carries, when it was declared to carry any — enough for a read that never touches the document." True of the data structure, false of every read that actually happens.

Triage

Does it lose data, break a promise on a public surface, or stop somebody installing and running? No, no, and no. Every answer is correct; the document fetch returns the same fields the index carried plus the rest. Nobody can observe this except by measuring how long a scan takes, and no declared behaviour changes when it is fixed.

The one thing that is a promise is the sentence in scan.go's doc — but it is a Go doc comment on an internal/ package, not a public surface. Fixing the sentence costs nothing and can happen any time; fixing the mechanism is a change to the hottest loop in the store and wants a benchmark first. Defer.

If deferred, what it costs to wait

• Nothing on compatibility. Using entry.Include when the operation's projection is a subset of the carried fields is purely internal: same rows, same order, same wire shape.
• The cost is a number nobody has: a covering index is advertised as the reason to declare include, and the first person who benchmarks sapedb against something else will measure a scan that is doing a random document read per entry. That is a bad first number, and first numbers are sticky.
• The comment is the real risk while this waits. Somebody declares include on the strength of it and gets no faster. Rewording scan.go:110-112 to say what is true today is the cheap half and should not wait for the mechanism.