DGX Spark/Guides/Running multiple containers and services on a DGX Spark
Operations

Running multiple containers and services on a DGX Spark

By Samuel Seidel · Published September 9, 2026

For three or four services on one DGX Spark, a docker-compose file is usually all you need: one machine with one GPU doesn't need a cluster scheduler. A Spark comes with Docker and the CUDA stack preinstalled and gives you root on a dedicated machine, so a single container is quick to start. A rented GPUwerk Spark has Docker inside your instance on some machines only; the others run a plain container without it, and you can't choose which (see the tenant environment page). This page covers running several together (a model server, a gateway in front of it, maybe a small database or vector store) and the signs that a workload has outgrown compose.

Why docker-compose is the right default

For a single Spark, docker-compose covers most of what a multi-container setup actually needs: defined services with their own images and environment variables, a shared network so containers reach each other by name instead of hardcoded IPs, restart policies so a crashed container comes back without manual intervention, and named volumes so data on the 1TB NVMe storage survives a container restart. A typical compose file for a self-hosted LLM stack might define three services: the model server (vLLM or similar) with GPU access, a gateway like LiteLLM in front of it handling routing and keys (see the LiteLLM doc), and maybe a lightweight database backing the gateway's key and usage tracking.

The GPU is the one resource that needs explicit handling in the compose file. In most serving setups only one process can efficiently hold the model in memory, so typically one service (the model server) declares GPU access in its compose block while everything else runs CPU-only and talks to it over the network. That's a config detail, not an orchestration problem, and it's the same whether there's one container on the box or five.

When compose starts to strain

A handful of signals suggest a plain compose setup has been outgrown. Watch for them rather than solving them in advance:

None of these apply to most single-Spark deployments. A model server, a gateway, and a couple of support services on one machine is squarely inside what compose was built for, and reaching for a heavier orchestrator before hitting one of these signals mostly adds operational overhead: more moving parts to patch, monitor, and understand, for a problem the box doesn't actually have yet.

Practical habits worth having from the start

These cost little to set up, even in a simple compose deployment, and save real time later:

For budgeting GPU memory across the containers running on one Spark, see the GPU memory management guide. A first engagement can help design a container setup for a specific multi-service deployment.

First top-up: pay $20, get $30 in credit

Docker and CUDA ready, root in your own instance from minute one.

Rent a dedicated Spark from $0.79/h, billed by the minute, and run your compose stack on GPUwerk's own Sparks, with no driver setup.

Create account Read the memory guide