Jobs System
Long-running queries execute asynchronously through the job system. Submit a job, check its status, and download results when ready.
Requires Academic verification.
Workflow
Submit job → Job queued → Job running → Job completed → Download result
→ Job failed (check error, rerun)
Submit a Query Job
curl -X POST -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
https://api.syndb.xyz/v1/jobs \
-d '{"query": "SELECT * FROM neurons WHERE dataset_id = '\''...'\''", "format": "arrow"}'
Returns a job_id for tracking.
Submit a Graph Job
curl -X POST -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
https://api.syndb.xyz/v1/jobs/graph \
-d '{"dataset_id": "...", "analysis": "full"}'
Check Status
curl -H "Authorization: Bearer $TOKEN" \
https://api.syndb.xyz/v1/jobs/{job_id}
| Status | Meaning |
|---|---|
pending | Queued, waiting for a worker |
running | Currently executing |
completed | Results available for download |
failed | Execution error (check error_message) |
cancelled | Cancelled by user |
List Your Jobs
curl -H "Authorization: Bearer $TOKEN" \
https://api.syndb.xyz/v1/jobs
Download Results
curl -H "Authorization: Bearer $TOKEN" \
https://api.syndb.xyz/v1/jobs/{job_id}/result \
-o result.arrow
- Query jobs: Arrow IPC format (readable by pandas, polars, DuckDB)
- Graph jobs: JSON
Cancel a Job
curl -X DELETE -H "Authorization: Bearer $TOKEN" \
https://api.syndb.xyz/v1/jobs/{job_id}
Rerun a Job
curl -X POST -H "Authorization: Bearer $TOKEN" \
https://api.syndb.xyz/v1/jobs/{job_id}/rerun
Creates a new job with the same parameters.
Configuration
| Parameter | Default | Environment Variable |
|---|---|---|
| Max concurrent workers | 4 | JOB_QUEUE_MAX_WORKERS |
| Result TTL | 24 hours | JOB_RESULT_TTL_HOURS |
| Max result size | 1 GB | JOB_MAX_RESULT_BYTES |
Results are stored in S3 and automatically cleaned up after the TTL expires.