Skip to content
Subspace

Trackers

A tracker provider maps an experiment-tracking platform’s native API onto the mlops.tracker contract described in contracts and conformance: list runs, fetch detail, and optionally stream metric history, artifacts, sweeps, a model registry, or alerts. Subspace ships three platform tracker providers alongside the credential-free filesystem tracker: MLflow, Weights & Biases, and Dagster.

MLflow

Section titled “MLflow”

plugins/mlflow is the tracker reference implementation, the plugin every other tracker provider’s REST integration is modeled on. It maps core plus the metrics.history, artifacts, and registry capability groups, and deliberately declares no launcher, MLflow is read-side only in Subspace. A binding page carries metadata.mlops.tracker {provider: 'mlflow', trackingUri, experimentIds?}.

  • Plugin fn tools: fn.mlflow.runs.list, fn.mlflow.runs.get, fn.mlflow.metrics.final, fn.mlflow.metrics.series, fn.mlflow.artifacts.list, fn.mlflow.artifacts.fetch, fn.mlflow.models.list, and fn.mlflow.models.get, all sandboxed fns shipped by the plugin (sources in plugins/mlflow/fns/). Every response normalizes to the contract schemas before it reaches a page.
  • file: tracking URIs are credential-free. Point a binding at a local MLflow file store and it works with no account, confined below the binding page’s metadata.fs.path and served by the core engine tools (mlops.ingest, mlops.metrics.history, mlops.registry.sync), sandboxed fns have no filesystem, by design.
  • Remote bindings are HTTPS-only. The fns call the fixed /api/2.0/mlflow/* REST surface through sandboxed http.fetch; the manifest’s capabilities.fetchBindings: ["mlops.tracker.trackingUri"] harvests the hosts users wrote onto binding pages into the fetch allowlist (the binding page is the consent object). Requests carry time and byte caps, and artifact fetches only inline small files as base64 (larger ones stay references).
  • Ingest and registry pages: the boot/on-demand sweep keeps a monotonic per-binding watermark, relists strictly since it, and idempotently materializes mlops/run and mlops/experiment pages. The core mlops.registry.sync tool creates deterministic mlops/model pages and links versions back to their producing run page when known; the mlops-core dashboard’s model registry table reads these directly.

Weights & Biases

Section titled “Weights & Biases”

plugins/wandb is the fullest tracker: it maps core plus every one of the v1 read capability groups, metrics.history, artifacts, sweeps, registry, and alerts, and it is also a full mlops.launcher (see launchers). Native tools cover runs, history, files, sweeps, and model collections on the tracker side, plus queue submit/status/cancel/capacity and sweep submission on the launcher side.

  • Native sweeps run through W&B’s own sweep API rather than the generic Cartesian fallback. A polyfill-parity test proves the native sweep expands the same finite search space expandSweepSpace/submitSweepPolyfill would produce for a provider without native sweep support, so a search space behaves identically whether or not the provider has first-class sweeps.
  • Alerts webhooks: fn.wandb.alerts.subscribe wires W&B run alerts into the same event pipeline other providers feed via ingest.
  • wandb-sweep-board is W&B’s provider-keyed view: a sweep/group board over the tracker’s mlops/experiment pages, distinct from the generic mlops-core dashboard.
  • External mutations (queue submit, sweep submission) are approval-gated, and every submit or sweep effect is claimed on its call id so a crash-replay never double-launches.

Dagster

Section titled “Dagster”

plugins/dagster is the launcher reference (see launchers for that side), but it is also a tracker: tracker core normalizes Dagster run listings, mapping numeric mlops/metric/* tags to metric values and parameter mlops/param/* tags to params. fn.dagster.assets.list keeps Dagster’s asset model native rather than forcing it through the run-centric contract; partition and run-config knobs stay in providerOptions.dagster until a second consumer justifies promoting them into the shared schema.

Brokered fetch for credentials

Section titled “Brokered fetch for credentials”

MLflow (remote), W&B, and Dagster all reach an external API, and none of their tools ever hold a raw secret. The provider fns run in the plugin sandbox and their manifests declare brokered credentials the host injects after the domain-allowlist check: wandb.api_key renders as Authorization: Basic {base64(api:{secret})} narrowed to wandb.ai, and dagster.api_token renders as both Authorization: Bearer … and Dagster-Cloud-Api-Token on binding-page endpoints. Requests are HTTPS-only, timeout- and response-size-bounded, redirect-closed when credentialed, and secrets never enter the sandbox, a page body, or a launch config, only the host-side broker sees the raw value. See tools and approvals for how the broker and its grants work generally.

The metric-history tool and chart viewer

Section titled “The metric-history tool and chart viewer”

Every tracker provider that declares metrics.history exposes it behind one shared shape: mlops.metrics.history {page} (ungated, read-only) returns every numeric series for a run’s page as {run, series: [{key, step, value}]}. For the filesystem tracker this reads metrics.jsonl directly, one line per step, an explicit numeric step field wins, and a run with only a final metrics.json degrades to a single step-0 point per key. Platform providers map the same call onto metrics.series (MLflow, W&B) or the tag-derived series (Dagster).

The mlops-metric-chart viewer, shipped by mlops-core, binds to this tool as a step viewer: an ESM component rendering a multi-series SVG line chart with fixed-order categorical hues and a legend, shown inline in a run page’s step detail regardless of which tracker produced the run. See MLOps overview for where the viewer sits in the mlops-core plugin, and contracts and conformance for the underlying metricPoint shape and the metrics.history capability module every tracker’s conformance suite runs.