Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Deploy Your Own SynDB Node

This scenario covers the three deployment models for SynDB, from a local evaluation stack to a production kubernetes deployment to a federated node.

1. Evaluation — Docker Compose

The fastest way to run SynDB locally:

git clone --recurse-submodules https://github.com/memorycircuits/SynDB.git
cd SynDB
cp .env.example .env
syndb dev sync-versions
syndb stack prepare
syndb stack up

This starts PostgreSQL, ClickHouse, SeaweedFS (S3), Meilisearch, the API, and the web UI.

System requirements: 8 GB RAM, 4 CPU cores, 20 GB free disk.

Services:

ServiceURLDefault port
Web UIhttp://localhost:80908090
APIhttp://localhost:80808080
API docshttp://localhost:8080/docs(browser)
ClickHouse HTTPhttp://localhost:81238123
ClickHouse nativelocalhost:90029002
PostgreSQLlocalhost:54335433
S3 (SeaweedFS)http://localhost:90009000

What gets created

  • A Docker network syndb
  • PostgreSQL database syndb with migrations applied
  • ClickHouse database syndb with table schemas created
  • S3 buckets for mesh, SWB, search, job, and ETL data
  • API server with token authentication
  • Web UI served by the Dioxus Fullstack server

2. Production — Kubernetes

The primary production path uses Helm charts on a Kubernetes cluster (developed against the NRP Nautilus platform).

Prerequisites

  • Kubernetes 1.28+
  • Helm 3.14+
  • A StorageClass that supports ReadWriteOnce PVCs
  • Optional: Ingress controller for HTTP access

Install

helm upgrade --install syndb infrastructure/helm/syndb-hub \
  --namespace syndb --create-namespace

For ETL workers:

helm upgrade --install syndb-etl infrastructure/helm/syndb-etl \
  --namespace syndb

Configuration

Refer to the environment reference for the full list of configuration variables. Key settings:

VariablePurposeExample
CLICKHOUSE_HOSTClickHouse addressclickhouse.syndb.svc
POSTGRES_HOSTPostgreSQL addresspostgres.syndb.svc
S3_ENDPOINTS3-compatible object storehttps://s3-west.nrp-nautilus.io
PASETO_SECRETToken signing key(generate with openssl rand -hex 32)

3. Federation node

A federation node is a standard SynDB deployment with federation enabled. It registers with a hub and makes its datasets available for cross-cluster queries while retaining data ownership.

Configuration

Add these to the hub’s Helm values:

federation:
  enabled: true
  hub:
    listenAddr: "/ip4/0.0.0.0/tcp/4001"

Add these to each cluster’s Helm values:

federation:
  enabled: true
  cluster:
    hubEndpoint: "https://hub.syndb.example.com"
  nodeName: "my-cluster"

Registration

Once the cluster is running, register it with the hub:

syndb federation register \
  --name my-cluster \
  --api-url https://my-cluster.example.com

The hub and cluster establish a libp2p connection. The cluster’s datasets appear in the hub’s catalog within seconds.

Resource estimates

DeploymentCPURAMStorageNotes
Evaluation4 cores8 GB20 GBDocker Compose, single machine
Production8-16 cores32-64 GB500 GB+ SSDKubernetes, 3+ ClickHouse nodes
Federation4-8 cores16-32 GB100 GB+Additional libp2p overhead

Next steps