Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Node Setup

This guide walks through joining the SynDB federation as a node operator.

Prerequisites

  • ClickHouse instance with a syndb database
  • Network reachability to the hub (or mDNS on the same LAN)
  • The federation password (provided by the hub administrator)
  • syndb binary with federation support

Step 1: Initialize

syndb ops federation init \
  --cluster-name "my-lab-node" \
  --clickhouse-endpoint "clickhouse.mylab.edu" \
  --clickhouse-http-port 8123 \
  --clickhouse-port 9440 \
  --federation-password "$SYNDB_FEDERATION_PASSWORD" \
  --institution "My University" \
  --contact-email "[email protected]"

This command:

  1. Bootstraps a libp2p swarm and discovers the hub via mDNS or configured multiaddrs
  2. Registers the node with the hub (presenting the federation password)
  3. Applies any pending ClickHouse schema migrations
  4. Saves configuration to ~/.config/syndb/federation.json

Optional flags

FlagDefaultDescription
--listen-addrOS-assignedlibp2p listen address (e.g., /ip4/0.0.0.0/udp/4001/quic-v1)
--descriptionHuman-readable cluster description

Step 2: Verify

# Show federation config
syndb ops federation status

# Test connectivity (3s mDNS discovery + hub + ClickHouse check)
syndb ops federation test

federation test performs:

  1. Bootstraps a temporary libp2p swarm with mDNS discovery
  2. Looks up the hub in the DHT
  3. Tests ClickHouse connectivity

Step 3: Sync Schema

If the hub has newer schema migrations:

# Preview changes
export SYNDB_HUB_URL="https://api.syndb.xyz/v1"
syndb ops federation sync-schema --dry-run

# Apply
syndb ops federation sync-schema

sync-schema currently uses an HTTP fallback endpoint and expects SYNDB_HUB_URL to point at the hub API base.

Step 4: Confirm Registration

List all federated clusters to verify your node appears:

export SYNDB_HUB_URL="https://api.syndb.xyz/v1"
syndb ops federation clusters

Environment Variables

VariableRequiredDefaultDescription
SYNDB_FEDERATION_PASSWORDYesShared secret for hub registration
SYNDB_SERVER_URLNohttps://api.syndb.xyzDefault server URL for the CLI root command tree
SYNDB_HUB_URLFor sync-schema and clusters fallback flowsHub API base including /v1 (for example https://api.syndb.xyz/v1)
FEDERATION_CLUSTER_NAMEYes (node mode)Unique cluster identifier
FEDERATION_NODE_FLIGHT_PORTNo50052Internal Flight gRPC port
FEDERATION_NODE_FLIGHT_ADVERTISENoderivedAdvertised Flight endpoint for remote delegation
FEDERATION_ENABLE_MDNSNotrueEnable mDNS for LAN discovery
FEDERATION_LISTEN_ADDRNoOS-assignedlibp2p listen address
FEDERATION_HUB_MULTIADDRSNoComma-separated hub multiaddrs for WAN
FEDERATION_CLUSTER_NATIVE_PORTNo9440ClickHouse native port for remote() queries

Docker Compose (Development)

For local development, the federation profile starts a hub and one node:

docker compose --profile federation up -d

This starts:

  • clickhouse-node — ClickHouse on HTTP 8124, native 9003
  • clickhouse-node-setup — Creates federation user on the node
  • clickhouse-hub-fed-setup — Creates federation user on the hub
  • syndb-node — Federation daemon with Flight on 50052, libp2p on 4001

All services use network_mode: host and discover each other via localhost.

Removing a Node

syndb ops federation logout

This deletes ~/.config/syndb/federation.json. The hub administrator can also deactivate the cluster via DELETE /v1/federation/clusters/{id}.