Declare for collections: adding one should not mean stopping the server

Status

Done

Version

1.0.0

Component

server

Priority

Highest — a prerequisite for the external-operations group, not a peer of it

Blocked by

—

Blocks

— (was SAPE-11, SAPE-12; both unblocked by this landing)

What shipped, and where it differs from what was asked

Shipped as frame 14, establish — not as an extension of frame 13, which is what this ticket proposed. The reason is the freeze rule this same backlog writes down: frame 13 is already Declare and already accepts exactly one payload shape, {"operation": …}. Widening it would have changed what an existing frame accepts, which is precisely the thing SAPE-2 says the tag forbids. New behaviour takes a new frame; 14 was the next free number, and it is now spent.

The consequence is worth stating because the original text above argues the other way: "adding collections later means either a second frame for a thing the first frame is already named after, or changing what frame 13 accepts." The first horn is what happened. Frame 13 is Declare and carries operations; frame 14 is establish and carries collections. That naming is slightly awkward forever, and it was still the cheaper of the two.

Acceptance: seven of eight measured, the eighth not yet reachable

• 1–7 met, each measured against a real sapedbd: declare-then-write on one connection with no restart; the daemon's pid compared before and after, not inferred from the socket staying up; other connections serving across the declaration; the collection and its documents surviving a restart; an invalid spec refused with apply's own validation and leaving nothing partial behind; the operator gate; the audit entry.
• 8 is not met and cannot be tested yet. It requires SAPE-12's worked packaged operation, which does not exist. Nothing about criterion 8 has been demonstrated — it is carried forward to SAPE-12 rather than counted here. Marking this ticket Done on seven of eight is a deliberate call, and this line is the record of it.

Reach, measured: the frame is exercised from the CLI, from all three clients, and from the desktop workbench — which gained a "New collection" form, an in-place upgrade path that says UPGRADES IT IN PLACE before the button is pressed, and a red banner naming an index the form would delete, also before the button is pressed. The upgrade case is measured the way that matters: three documents written before adding an index, then read back both by primary key and through the new index, proving the index was built over existing data rather than merely declared.

What it turned up

ISS-21: the refusal this frame raises most — store.ErrIncompatible, "this does not match what was declared before" — is not in codeFor, so it reaches a client as "failed". It was harmless while that error could only happen offline inside apply, where a human reads the sentence. This frame gave it a program as its audience.

Description

Extend frame 13 so a collection can be declared against a running daemon, the way an operation already can. Same connection, same proof of the server's secret, same audit entry. Adding a collection stops being an outage.

Why this is now the highest-priority ticket in the group, not a side item

pipelines/tasks/0071 named three shapes of external operation the user actually wants: (1) one operation sapedb is missing, (2) several operations folded into one — function and procedure, in the Pascal/Ada sense — and (3) one operation that solves a whole textbook problem end to end, through the shapes it publishes. Measuring shape (3) is what changes this ticket's rank: it is not one operation, it is a packaged business module, and a module carries its own storage — collections, indexes, rollups — alongside the vocabulary (operations) and the typed surface (shapes) built on top of it.

internal/server/declare.go is the frame-13 handler, and it is measured, not assumed: it declares exactly one type, store.Operation (declare.go:46, json:"operation"). A repository-wide search of that file for Spec or Collection — the type and the field a collection declaration needs — returns zero matches. So a KindDeclaration external operation (the shape 0071 recommends for v1: a signed bundle of declarations, no third-party code) can carry its Operations over the wire today, but not the Collections they read and write. It cannot install into a database that does not already have the collection by some other means — and the only other means is stopping the daemon and running sapedb apply by hand, which is exactly the outage this ticket exists to remove.

This is why SAPE-14 now sits ahead of, not beside, the tickets it blocks: SAPE-11 (listing installed external operations) and SAPE-12 (the worked external operation, published as a challenge) both assume an external operation can be installed on a database that did not already anticipate it. For the shape of external operation the user cares about most — a self-contained module, not a single missing verb — that assumption is false until this ships.

Why now (original)

It is a public-surface asymmetry, and the surface is the one 1.0.0 freezes hardest. Frame 13 is Declare — not DeclareOperation — and a client library's Declare(operation) is the shape every client author is copying right now. Adding collections later means either a second frame for a thing the first frame is already named after, or changing what frame 13 accepts, and SAPE-2 freezes frame roles at the tag. The cheap moment to make the frame whole is before anyone has written against half of it.

Measured 2026-09-20. Both halves exist in the store: internal/store/store.go:144 has func (s *Store) Declare(spec Spec) (*Collection, error) and internal/store/ops.go:367 has DeclareOperation. Only one of them is reachable over the wire. Searching every non-test file for callers of .Declare( outside the store finds exactly one: internal/cli/cli.go:683 — the apply path, which works on the database file directly. Nothing in internal/server/ calls it, so the only way to add a collection is to stop the daemon, run apply against the file, and start it again. The engine method has been there the whole time; what is missing is the frame handler in front of it.

Re-confirmed while measuring task 0071: internal/server/declare.go declares a declaring struct with exactly one field, Operation store.Operation (declare.go:44-53). Grepping the file for Spec and for Collection each return nothing.

Acceptance criteria

1. A connected client declares a collection against a running daemon and then writes a document into it over the same connection, without a restart in between.
2. The daemon's process is the same one before and after — asserted on the process, not inferred from the connection staying up.
3. Other connections keep working across the declaration. A second connection invoking an operation on a different collection succeeds while the declaration happens.
4. The collection survives a restart, with its documents. Declare over the wire, write, stop, start, read back — the same values.
5. An invalid specification is refused with the same validation as apply, and refusing leaves no partial collection behind: after the refusal, the catalogue does not list it.
6. The declaration is refused on a connection that has not proved the server's secret, exactly as Explore and operation-declare are.
7. The audit trail records it with the account that declared it, readable afterwards.
8. A packaged external operation (SAPE-12's worked example, once it exists) installs into a database with none of its collections pre-declared, by sending its collection declarations over frame 13 before its operations — one connection, one signed bundle, no sapedb apply step. This is the criterion 0071 added; the seven above are what SAPE-19 already proved for operations. Not met — carried to SAPE-12, see above.

Out of scope

• Altering or dropping a collection over the wire. Adding one is additive; changing a collection with documents in it is a migration and belongs with the format rules in SAPE-6. Partly overtaken: adding an index to an existing collection does now work, and builds over existing documents. Dropping a collection still needs the server stopped — no wire frame exists for it.
• Declaring an index on an existing collection separately. The specification carries its indexes.
• Removing the apply path. It stays; it is what works on a file nobody is serving.
• Transactional declaration of several collections at once. One per frame.