Files & blobs

Every file Subspace stores, an upload, an email attachment, a code cell’s image output, a mirrored Google Doc, lands in the same content-addressed blob store. A file element in a page is a thin pointer into that store plus a viewer, and a file can even own a page, turning a PDF into an annotatable outline.

The blob store

Blobs live on disk as blobs/ab/cd/<sha256>, sharded two levels deep by the first four hex characters of the content hash so no directory ever holds an unreasonable number of files. Writes are crash-safe by construction: a new blob is written to a temp file, fsynced, renamed into place, and only then does the referencing database row (a kb.files row, a mail attachment row, a code-cell output) commit. If the process crashes between the rename and the row commit, the result is an orphaned blob, never a row pointing at missing bytes.

Orphans are swept by a weekly blob GC: a blob-gc row in ops.schedules, seeded idempotently on boot for Sunday 03:15, fires a { systemJob: 'blob-gc' } target through the ordinary scheduler, the same single-flight, watermark-tracked, scheduled-task machinery any other scheduled job uses. The sweep diffs the blob directory against every table that can reference a blob, kb.files, mail body blobs, headers.attachments references, and code-cell image outputs, and deletes orphans older than a 24-hour grace window, so a blob mid-upload is never mistaken for garbage.

Uploading a file

attach file in the page header uploads into the blob store and drops a file node in place:

images preview inline;
PDFs render through pdf.js with page navigation;
everything else gets a plain download row.

A file can own a page

The frame action open as page ↗ mints a page owned by the file (metadata.file.fileId points back at it): the viewer sits on top, an ordinary outline sits below. This is the annotation surface, you can write bullets alongside a PDF the same way you’d write bullets anywhere else, and those bullets are ordinary nodes that link, tag, and search like any other page content.

Ask this file

Every inline file frame has an ask this file toolbar action. The same action appears in the header of a file-owned page. Enter a question and Subspace opens a writable annotation page, appends /ask <question>, and starts the first-party file-qa agent there. If you can read an existing annotation page but cannot write it, Subspace creates a private page for your run instead. The answer lands as cited child bullets beneath the /ask node.

The run is scoped to the file you selected, not merely prompted to prefer it. Its checkpoint pins that file id, annotation page, and vector-reference prefix; lexical file_text and vector retrieval are filtered to that file before results are fused. The agent only gets kb.search, kb.read, and kb.grep, and those tools cannot escape the selected-file scope even if a tool argument names another file. It has no approval-requiring or write tools. An unreadable file behaves as missing and creates neither a page nor a run.

Text extraction feeds search

On upload, Subspace extracts text where it can: PDFs through pdf-parse, .docx through mammoth, ordered slide text from .pptx, worksheet names and CSV-shaped cell content from .xlsx, and plain text, Markdown, CSV, JSON, and XML files. Extracted text feeds search.file_text, so file contents, not just filenames, are findable through search’s ⌘K as file hits and through ask this file.

SUBSPACE_FILE_TEXT_MAX_BYTES sets the extracted-text output cap in bytes. It must be a positive integer; the default is 8 MiB and the maximum is 64 MiB. Corrupt PDFs and corrupt, encrypted, or oversized Office archives fail closed and contribute no extracted text. Images/OCR and audio transcription remain explicit provider-and-capability-broker lanes: the generic extractor returns no text for them and never calls a provider implicitly.

Connected provider mirrors

Google Drive, Dropbox, and SharePoint (via Microsoft Graph) connect as read-only mirrors behind one shared connector contract: list / download / upload / changes.

Change tracking is provider-native: opaque Drive page tokens, Dropbox cursors (with long-poll), and Graph delta links persist in ops.provider_sync_state, and a cursor only advances once every file it named has actually landed, so a crash mid-sync re-processes rather than skips.
Materialization: mirrored blobs and their kb.files rows appear as file-owned pages under connected-files/. Each carries provider_meta (the provider’s file id, revision, and mirror timestamp). Google Docs export as Markdown and are flagged lossy if you ever write them back. An upstream deletion marks the mirror but keeps the local blob and page history intact.
Read-only by default, with an explicit write-back. Provider pages reject ordinary edits in both the outliner and the command path. The replace upstream contents action sends the mirrored revision as a CAS precondition (If-Match for Drive/Graph, Dropbox’s update rev). If the upstream revision has moved since the mirror was taken, Subspace refreshes the mirror, preserves your edited candidate as a second file rather than dropping it, and files a provider-merge card in Needs confirmation for a human to reconcile.
Credentials: Drive reuses the Google grant already established for mail and calendar (new connections request the Drive scope; connections made before Drive support was added need to reconnect once). Dropbox reads DROPBOX_ACCESS_TOKEN; SharePoint reads MICROSOFT_GRAPH_ACCESS_TOKEN and MICROSOFT_GRAPH_DRIVE_ID. On boot these import into encrypted ops.secrets, and every connector call reacquires its access token through the capability broker, so a rotated token doesn’t require a restart.
Status and manual sync: GET /provider-sync reports authenticated status for every connected provider; POST /provider-sync/:provider[?account=…] lets an admin trigger one connector’s sync on demand. Mirror refreshes are system-origin writes and therefore do not fire page triggers by default. An intentional automation boundary can pass the exact pageTriggers=fire query parameter to a manual sync or provider write-back; that request uses a distinct, audited system-device flavor while polling stays loop-safe. Tests run against a deterministic Drive fixture by setting PROVIDERS=fake together with SUBSPACE_PROVIDER_SYNC_FAKE=1.