Docker Compose
Local development and single-machine deployment using Docker Compose.
Base Stack
cargo run -p cli --features dev -- stack up
Starts the core services:
| Service | Port | Description |
|---|---|---|
syndb-api | 8080 (HTTP), 50051 (Flight) | REST API + Arrow Flight |
syndb-ui | 8090 | Web frontend |
postgres | 5433 | Metadata, users, access control |
clickhouse | 8123 (HTTP), 9002 (native) | ClickHouse data warehouse |
s3 | 9000 (API), 9001 (console) | MinIO object storage |
meilisearch | 7700 | Meilisearch full-text search |
All services use network_mode: host — they bind directly to the host network.
Local Search Smoke
The compose stack includes Meilisearch and wires it into the API. To rebuild the local dataset search index from PostgreSQL and query the public search endpoint:
nix develop . -c syndb test meilisearch-local
This smoke path:
- ensures the stack is up
- runs
syndb data search reconcileagainst the local PostgreSQL and Meilisearch services - queries
http://localhost:8080/v1/search/fulltext?q=test&limit=5
The response may legitimately contain zero hits on a fresh stack, but the command should complete successfully and return valid JSON from the API.
Federation Profile
docker compose --profile federation up -d
Adds federation services on top of the base stack:
| Service | Port | Description |
|---|---|---|
clickhouse-node | 8124 (HTTP), 9003 (native) | Node ClickHouse |
clickhouse-node-setup | — | Creates federation user on node |
clickhouse-hub-fed-setup | — | Creates federation user on hub |
syndb-node | 50052 (Flight), 4001/UDP (libp2p) | Federation node daemon |
Note: The
federationandfederation-worldprofiles share port 8124 and are mutually exclusive.federation-worldruns 5 regional ClickHouse nodes for benchmarking only.
ETL Profile
Run dataset imports:
docker compose --profile etl run syndb-etl etl <dataset> <command>
Example:
docker compose --profile etl run syndb-etl etl hemibrain download
docker compose --profile etl run syndb-etl etl hemibrain import --data-dir /data/Hemibrain --table neurons --dataset-id <uuid>
Version Management
All service versions are defined in versions.nix. After changing versions (built with Nix):
syndb dev sync-versions
This regenerates .env with the correct image tags.
Image Building
Build container images from Nix:
cargo run -p cli --features dev -- stack prepare
This builds the local development images used by Compose: syndb-api-rust:dev, syndb-etl:dev, and syndb-ui:dev.
Volumes
| Volume | Service | Content |
|---|---|---|
clickhouse-data | clickhouse | ClickHouse data |
clickhouse-node-data | clickhouse-node | Node ClickHouse data |
postgres-data | postgres | PostgreSQL data |
minio-data | s3 | S3 object storage |
meilisearch-data | meilisearch | Search index |
Cleanup
ClickHouse creates files with UID 100100 and restrictive permissions. To clean volumes:
podman unshare rm -rf <volume-path> # requires Podman (https://podman.io)
Prefer keeping data in Docker volumes rather than bind mounts to avoid permission issues.