Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Node Setup

This guide walks through joining the SynDB federation as a node operator.

Prerequisites

  • ClickHouse instance with a syndb database
  • Network reachability to the hub (or mDNS on the same LAN)
  • The federation password (provided by the hub administrator)
  • syndb-cli binary (with federation feature)

Step 1: Initialize

syndb federation init \
  --cluster_name "my-lab-node" \
  --clickhouse_endpoint "clickhouse.mylab.edu" \
  --clickhouse_http_port 8123 \
  --clickhouse_port 9440 \
  --federation_password "$SYNDB_FEDERATION_PASSWORD" \
  --institution "My University" \
  --contact_email "[email protected]"

This command:

  1. Bootstraps a libp2p swarm and discovers the hub via mDNS or configured multiaddrs
  2. Registers the node with the hub (presenting the federation password)
  3. Applies any pending ClickHouse schema migrations
  4. Saves configuration to ~/.config/syndb/federation.json

Optional flags

FlagDefaultDescription
--listen_addrOS-assignedlibp2p listen address (e.g., /ip4/0.0.0.0/udp/4001/quic-v1)
--descriptionHuman-readable cluster description

Step 2: Verify

# Show federation config
syndb federation status

# Test connectivity (3s mDNS discovery + hub + ClickHouse check)
syndb federation test

federation test performs:

  1. Bootstraps a temporary libp2p swarm with mDNS discovery
  2. Looks up the hub in the DHT
  3. Tests ClickHouse connectivity

Step 3: Sync Schema

If the hub has newer schema migrations:

# Preview changes
syndb federation sync-schema --dry_run true

# Apply
syndb federation sync-schema

This uses an HTTP fallback via SYNDB_HUB_URL if libp2p is unavailable.

Step 4: Confirm Registration

List all federated clusters to verify your node appears:

export SYNDB_HUB_URL="https://api.syndb.xyz"
syndb federation clusters

Environment Variables

VariableRequiredDefaultDescription
SYNDB_FEDERATION_PASSWORDYesShared secret for hub registration
SYNDB_HUB_URLFor HTTP fallbackHub API URL (e.g., https://api.syndb.xyz)
FEDERATION_CLUSTER_NAMEYes (node mode)Unique cluster identifier
FEDERATION_NODE_FLIGHT_PORTNo50052Internal Flight gRPC port
FEDERATION_NODE_FLIGHT_ADVERTISENolocalhost:50052Advertised Flight endpoint
FEDERATION_ENABLE_MDNSNotrueEnable mDNS for LAN discovery
FEDERATION_LISTEN_ADDRNoOS-assignedlibp2p listen address
FEDERATION_HUB_MULTIADDRSNoComma-separated hub multiaddrs for WAN
FEDERATION_CLUSTER_NATIVE_PORTNo9440ClickHouse native port for remote() queries

Docker Compose (Development)

For local development, the federation profile starts a hub and one node:

docker compose --profile federation up -d

This starts:

  • clickhouse-node — ClickHouse on HTTP 8124, native 9003
  • clickhouse-node-setup — Creates federation user on the node
  • clickhouse-hub-fed-setup — Creates federation user on the hub
  • syndb-node — Federation daemon with Flight on 50052, libp2p on 4001

All services use network_mode: host and discover each other via localhost.

Removing a Node

syndb federation logout

This deletes ~/.config/syndb/federation.json. The hub administrator can also deactivate the cluster via DELETE /v1/federation/clusters/{id}.