Client.Subscribe and NextChange exist; the loop was measured against a
real sapedbd — seed before the daemon starts, subscribe, then
Establish, Declare, insert, update and delete while watching, each
Change applied into a blank store, and the two compared collection by collection, spec by
spec, document by document field by field in both directions, and across every version of
every operation.
The last out-of-scope line below says this ticket "does not rewrite Apply; it is
already correct and already tested." That was wrong, and expensively so. Two real
defects were in store.Apply the whole time, and
TestAReplicaFedTheLogEndsUpTheSame — the green test quoted above as proof — stayed green
with both of them present, because both are invisible to a document comparison:
Apply installed a spec carrying
its number without advancing the id generator. A database that followed and was then promoted would
hand out id 1 again for its next own collection, and write documents into the
middle of another collection's keyspace. That is corruption, not inconvenience.record() writes both the
entry and its done marker; Apply wrote only the entry. A retry after a
failover, aimed at a database that had followed, would write a second time — the
exact case write ids exist to prevent.Both surfaced only because somebody kept going past a green test. It is the strongest argument this project has produced for its own rule that one passing case states very little.
A measurement trap was walked into and out of on the way, and is recorded because the escape is the
useful part: the first version of the "trimmed log" case used Subscribe(0), which is
refused whether or not any trimming happened — so it measured nothing. It was replaced by probing
Following.Oldest to prove the trim had already occurred, and only then
subscribing at the entry a positive control had just reached successfully.
Subscribe has no operator gate, while
Explore does. Needs a decision, and it freezes at the tag.store.Retain, so the log
has no ceiling and too_far_behind cannot occur on a real sapedbd at all.Replication and follower mode were brought into 1.0.0 by direct product decision — see SAPE-27's "Why now" for the honest reason, which is not this project's usual data-loss/public-promise/install-and-run triage. This ticket is the first half: consume, over the wire, a change that a different process wrote to a different database, and end up with the same document, declaration, or deletion locally.
The brief for this ticket described Change as "only ever generated, never applied." That is
not what measuring the code shows, and it is worth saying plainly because it changes the size and shape
of the work: Store.Apply already exists, is exported, and is exercised by tests
that do almost exactly what this ticket's acceptance criteria ask for.
(Read the section above before trusting the rest of this one: "already tested" turned out to mean
"tested in the ways that could not see either bug in it.")
func (s *Store) Apply(change Change) error switches on every Change.Kind —
put, delete, declare, operation, drop,
read (a no-op that still records) — refuses an out-of-order entry as
ErrOutOfOrder, and treats an entry at or below the latest applied LSN as already-there
(if change.LSN <= latest { return nil }).TestAReplicaFedTheLogEndsUpTheSame builds
a primary with two collections, 120 mixed puts/deletes, a mid-stream index change (a redeclare), and a
drop; replays the whole log into a blank store via Apply; and compares the two through
sameStores — every collection, every document, every index entry, not a count. It then
replays the same log a second time and asserts nothing changes, and applies one
out-of-order entry and asserts ErrOutOfOrder.TestOperationsTravelDownTheLogToo confirms declared
operations — including two versions of the same name — replay correctly and the newest still wins,
exactly as it does on the primary.
What is actually missing is narrower than "build Apply": Store.Apply has
zero callers outside internal/store's own tests. And the wire already carries exactly the right
payload to feed it: internal/server/subscribe.go's drain sends each
store.Change as json.Marshal(change) on an Event frame
(frame type 9). Nothing decodes an Event frame's payload back into a store.Change
on the receiving end — internal/wire/wire.go and the public sapedb.go package
have no Subscribe method at all, client-side. The two halves — a tested Apply
and a tested Event stream carrying exactly its input type — have never been connected across
a process boundary.
Make Apply reachable from outside internal/store, and prove it end to end
across the wire rather than in one process copying Go values directly — decode a
store.Change from the JSON bytes an Event frame carries, the way a real
follower will have to, and apply it into a separate database that never saw the primary's memory.
Subscribe/Event frames that already exist — decoded from wire JSON into
store.Change values on the receiving side, and applied in order into a second, blank
database in a separate process. Met.Apply reachable lives in a place that does not require
internal/store to grow a wire dependency. Met — the decoding lives on the client
side; the store package still knows nothing about frames.Store.Apply's own logic. It is already correct and already
tested; this ticket wires it up, it does not rewrite it.