Kubernetes & Helm

Production deployment on Kubernetes using Helm charts.

Charts Overview

Chart	Description
`syndb-hub`	Hub deployment (API, UI, depends on `syndb-clickhouse`)
`syndb-federation-node`	Federation node (syndb-node, depends on `syndb-clickhouse`)
`syndb-clickhouse`	Shared ClickHouse subchart (used by both hub and node)
`syndb-etl`	ETL batch jobs (download, prepare, import, graph-precompute)
`nautilus`	Umbrella chart for the NRP Nautilus cluster deployment

Charts are located under infrastructure/helm/.

Hub Deployment

The hub chart deploys the full SynDB stack. Key values:

syndb-clickhouse:
  clusterName: syndb-hub
  shardRegions:
    - name: dc1
      region: dc1
      replicas: 3

api:
  image: syndb-api-rust:latest
  flightPort: 50051
  resources:
    requests:
      cpu: "1"
      memory: 2Gi

The chart also creates a remote_servers.xml ConfigMap for ClickHouse cluster topology.

Node Deployment

Deploy a federation node at your institution:

syndb-clickhouse:
  clusterName: syndb-node
  shardRegions:
    - name: dc1
      region: dc1
      replicas: 2

nodeApi:
  enabled: true
  image: syndb-api-rust:latest
  flightPort: 50052
  libp2pPort: 4001
  hubMultiaddrs: "/ip4/<hub-ip>/udp/4001/quic-v1"
  federationPassword: "<shared-secret>"
  resources:
    requests:
      cpu: 500m
      memory: 512Mi

When nodeApi.enabled=true, the chart deploys:

A Deployment running syndb-node with Flight (TCP) and libp2p (UDP) ports
A Service exposing both ports
Environment variables auto-populated from values (cluster name, endpoints, passwords)

In Kubernetes, mDNS is disabled — use hubMultiaddrs for explicit hub discovery.

ETL Jobs

ETL runs as Kubernetes Jobs:

jobs:
  hemibrain:
    download:
      enabled: true
      resources:
        requests: { cpu: "500m", memory: "2Gi" }
        limits: { cpu: "600m", memory: "4Gi" }
    import:
      enabled: true
      resources:
        requests: { cpu: "1", memory: "4Gi" }
        limits: { cpu: "1200m", memory: "6Gi" }

Important: Kubernetes Jobs are immutable. Before running helm upgrade when resource values changed, delete failed or running ETL jobs:
kubectl delete job -n syndb -l app=syndb-etl --field-selector status.successful!=1

emptyDir warning: emptyDir volumes default to tmpfs and count against the pod’s memory cgroup limit. Add expected emptyDir data size to the memory limit.

Applying Changes

just nautilus-apply

Or manually:

helm upgrade --install syndb infrastructure/helm/nautilus/ \
  -n syndb --create-namespace \
  -f infrastructure/helm/nautilus/values.yaml