Storage & Postgres

All of Subspace’s durable state lives in one embedded PostgreSQL database. There is no second store to keep consistent, no cross-file cursor to stitch atomicity, and no message broker rebuilt inside the process. A user edit commits atomically with its reindex, its command-log row, and its outbox notice. An agent step’s checkpoint, output nodes, and approval card commit in one transaction or not at all.

One database, schemas as namespaces

Postgres schemas namespace the domains, but they are namespaces only: every table shares one transaction boundary, one write-ahead log, and one backup stream.

Schema	Holds
kb	the document graph: pages, nodes, links, labels, deps, the command log, aliases, files, view state
comms	mirrors and identity: mail, instant messaging, calendar, and the CRM resolution index
ops	the spine: events, tasks, reminders, agent runs, schedules, devices, secrets, the outbox, cursors
search	derived indexes: embeddings plus tsvector generated columns on the corpus tables

Because it is one database, any cross-domain invariant is a single BEGIN … COMMIT. Per-page write serialization is one pg_advisory_xact_lock on the page id, held for exactly the transaction, with no queue objects and no worker threads. Postgres MVCC gives concurrent writers for free: a Gmail backfill and a keystroke commit side by side.

The kb tables

The document graph is the heart of the store. The key tables:

Table	Purpose
pages	one row per page: title, slug, dir_id, kind (outline / directory / element), element config, metadata JSON, and last_seq (the per-page command sequence, stamped in-transaction)
nodes	one row per bullet: parent_id, rank (the fractional order key), text, an optional element payload, and props
links	reference edges (link / tag / embed); backlinks are just a query by destination
labels	labeled values ([[Page]]:cash), unique per page and label
deps	dataflow edges for formula and function recompute
kb_commands	the audit and undo log: seq, a UNIQUE client-minted id for retry dedup, device, page, and payload
page_aliases	old titles kept after a rename, so links keep resolving
files	content-addressed file metadata: sha256, name, mime, size, provenance
view_state	per-device, per-window UI state (folds, layout), keyed by (device_id, window_id, page_id)

Note

Folds are view state, never document state. Whether a bullet is collapsed lives in view_state for one device and window, so it is never broadcast to other windows and never written into the page. This keeps the document canonical across every surface.

The command model chapter covers how the reducer writes these tables; the outliner and page metadata pages cover them from the user’s side.

Embedded Postgres under Node

Python has pgserver (a pip package bundling Postgres and pgvector); Node has no equally trodden equivalent. Rather than compromise, Subspace owns its Postgres binaries.

Why own the binaries

In-process WASM Postgres (PGlite) is single-connection, which rules out a pool, a dedicated LISTEN connection, and cross-connection advisory locks, the exact mechanisms this design runs on. Off-the-shelf embedded-Postgres npm packages ship no pgvector and pin one major, so there are no N-1 binaries for an upgrade. Owning the build removes both limits: CI builds Postgres (server plus pg_upgrade, pg_dumpall, pg_receivewal) and compiles pgvector against those exact headers, published as per-platform npm optional dependencies (about 35 MB compressed each). Adding another extension later is one line in a checked-in build recipe.

The pg-embed supervisor

A small supervisor (about 250 lines in the server) manages the database as a child process. First run performs initdb --data-checksums. Config is templated: a unix socket inside ~/Subspace/pg/, no TCP listener at all, shared_buffers=128MB, and archive_command wired when backup is configured. The launchd unit that supervises the server transitively supervises Postgres, so a crash restarts both. The database is a child process the user never sees.

Caution

Postgres binds a unix socket only. There is no network listener, so the database cannot be reached over TCP from any machine, which removes an entire class of exposure. See security and trust.

The major-version upgrade path

Minor Postgres bumps are a binary swap on the same catalog. Major bumps are handled explicitly. A release that bumps the Postgres major ships both majors’ binaries for that release train. On boot, pg-embed compares PGDATA/PG_VERSION with the bundled major:

1. Force a fresh backup

   A full base backup and a final WAL archive run before anything touches the data directory.

2. pg_upgrade –link into a new data dir

   pg_upgrade --link hard-links the cluster into a new data directory, so the upgrade takes minutes even at tens of gigabytes.

3. Start, health-check, archive

   The new cluster starts, passes a health check, and completes one WAL archive cycle.

4. Delete the old dir

   Only then is the old data directory removed. A pg/upgrade-state.json journal makes the two-directory swap restartable if the process is interrupted.

If pg_upgrade ever fails, the fallback is pg_dumpall | psql. Because both majors’ binaries are present, that logical path always exists. The whole sequence is exposed as subspace doctor --pg-upgrade and is exercised in CI against a seeded previous-major cluster before any major-bump release ships.

The content-addressed blob store

File bodies do not live in Postgres. They live in a content-addressed directory (blobs/ab/cd/<sha256>), while the files table holds only metadata. The write order is deliberate:

1. Write and fsync a temp file

   The body is written to a temporary file and fsynced.

2. Rename into the content path

   The temp file is atomically renamed to its sha256-derived path.

3. Then commit the files row

   Only after the body is safely on disk does the files row commit.

So a crash leaves at worst an orphan blob (swept by a periodic directory-versus-table garbage collection), never a files row pointing at a missing body. Because blobs are immutable and content-addressed, the blob replicator uploads each new blob off-machine within seconds, with no window where a replicated row references an unreplicated body. See files and blobs for the user side and backup and restore for replication.

Related

The command modelHow the reducer writes the kb tables in one transaction.

The outbox dispatcherThe ops.outbox and ops.cursors change feed.

Backup & restoreContinuous WAL archiving, blob replication, and restore drills.

Self-hostingMoving one data directory from laptop to always-on box.