Two fixture copies kept identical by hand; nothing compares them

Type

Debt

Found while

SAPE-19

Triage

Defer

Status

Open

Severity

Medium — the artifact that drifts is the conformance spec

Description

fixtures/frames.json is the wire-format conformance artifact: the Go server reads it, the TypeScript client reads it, and it is published as the specification other languages build against. It exists as two files in two repositories, kept byte-identical by somebody remembering to copy it.

There is a comparison, and it only runs for whoever happens to have both checkouts side by side: packages/ecosy-sapedb/tests/shipped.test.mjs:50-64 reads ../../sapedb/fixtures/frames.json and, if the path does not exist, calls t.skip and passes.

Evidence

• Measured: the two files are identical today — md5 = d081de2dd5354f63459390b600fc055b on both packages/sapedb/fixtures/frames.json and packages/ecosy-sapedb/fixtures/frames.json. So this is debt, not a live defect.
• The comparison exists but is conditional. Its own comment: "The two repositories keep byte-identical copies by hand and nothing compares them. A drift window of ten minutes has already gone unnoticed by both suites. This is the comparison, for whoever has both checkouts."
• The client's CI does run it — and only in one of two jobs. ecosy-sapedb/.github/workflows/test.yml has a job "without a server" that only checks out itself (the compare skips), and a second job that git clone --depth 1 …/sapedb beside the workspace before running the suite (the compare runs). So the guard is real, and it is one workflow edit away from being silently gone: a skip is a pass.
• Nothing on the Go side compares anything. frame_test.go:29, declare_test.go:221 and fixture_decode_test.go:64 each read the server's own copy and never look at the client's.

Triage

Does it lose data, break a promise on a public surface, or stop somebody installing and running? No. The two copies agree today, and the artifact has a comparison in the one CI configuration where both sides are present. Nothing a user can run is wrong.

The honest reading is that this is an assurance gap rather than a defect: the guard passes by skipping, so its green tells you nothing about whether it ran. That is worth fixing and it is not worth holding a release for — and the fix that matters (one source of truth, published from one place) is a packaging decision, not a patch. Defer.

If deferred, what it costs to wait

• The failure mode is quiet and asymmetric: whoever edits one copy gets a green suite locally and a green suite in the job that does not clone. The drift shows up as a third-party client built against the wrong bytes, which is the most expensive place to find it.
• Two cheap mitigations that do not need the packaging decision: make the skip fail in CI (skip only when an explicit local-dev flag is set), and add the mirror comparison on the Go side so neither repository is the only one watching.
• Waiting does not get harder over time — there is no compatibility surface here — but the number of third-party clients built on the fixture only goes up, and each one raises the cost of a drift that does slip through.