Setup guide

DGX Spark setup: unboxing to first token

By the GPUwerk fleet team · Updated August 26, 2026 · 4 min read

We rack DGX Sparks for a living. This is the setup path we actually use, first boot to a served model, including the two or three places where the official quick-start leaves you guessing.

Before you plug it in

Power: the Spark draws up to 240 W from an ordinary wall socket, no special circuits. Give it ventilation room; it's quiet, but it does move air under sustained load.
Network: for a desk setup, the RJ45 port is fine. The two QSFP ports (ConnectX-7) matter only when you link two Sparks or push serious data, don't buy optics you don't need yet.
Peripherals: HDMI works out of the box, and DGX OS ships a full Ubuntu desktop: the Spark is a perfectly usable workstation with a monitor and keyboard, and plenty of owners run it exactly that way (local development, CUDA work, even daily driving). If yours will serve a team instead, you need the peripherals once for first boot and can run it headless after that.

1, First boot and DGX OS

The Spark ships with DGX OS (an NVIDIA-tuned Ubuntu 24.04) preinstalled, with the full CUDA stack already in place. First boot walks you through a standard Ubuntu setup: user account, locale, network. Two things to do immediately:

# bring the OS and NVIDIA stack current, first thing, before anything else
sudo apt update && sudo apt full-upgrade -y

# confirm the GPU stack is alive
nvidia-smi

If nvidia-smi shows the GB10 and a driver version, the hard part of a normal GPU-box setup, drivers, is already done for you. That's the point of DGX OS: don't replace it with vanilla Ubuntu, you'd be signing up to maintain the Arm + CUDA stack yourself.

2, Set up SSH (workstation or server, you want this)

# enable SSH if the installer didn't
sudo systemctl enable --now ssh

# from your laptop: copy your key, then disable password login
ssh-copy-id you@spark.local
sudo sed -i 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
sudo systemctl restart ssh

Key-only SSH is not paranoia, a machine like this ends up holding your models, your data, and eventually your team's prompts. It's also how our entire fleet runs: passwords are disabled on every GPUwerk node. Give the box a static IP or DHCP reservation while you're in the router anyway; a model server that changes address is a support ticket generator.

3, Choose your serving stack

Two sane starting points, different jobs:

Ollama: the fast path. One binary, pulls quantized models by name, perfect for evaluating what the machine can do. Start here on day one.
vLLM: the serving path. Higher throughput, proper batching, an OpenAI-compatible API. Move here when the Spark becomes a team resource rather than an experiment.

# day one: ollama and a first model
curl -fsSL https://ollama.com/install.sh | sh
ollama run qwen3:32b "Say hello from the GB10."

That's first token. On a 32B model at Q4 you'll be using roughly 20 GB of the 128 GB, which is the moment most people understand why they bought this machine: try ollama run llama3.3:70b next, a model no consumer GPU can hold, and watch it just load.

4, Serve it to your tools

# vLLM with an OpenAI-compatible endpoint on the LAN
pip install vllm
vllm serve Qwen/Qwen3-32B-AWQ --host 0.0.0.0 --port 8000

# any OpenAI-style client now works against it
curl http://spark.local:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"Qwen/Qwen3-32B-AWQ","messages":[{"role":"user","content":"hello"}]}'

Point your editor plugins, internal scripts, or a chat UI like Open WebUI at that base URL and the Spark quietly becomes team infrastructure. When that happens, revisit step 2 and think about who on the network can reach port 8000, an unauthenticated LLM endpoint on a flat office LAN is a data-governance hole of its own.

What we'd tell a friend before they start

Expect capacity, not speed. The Spark's 273 GB/s memory bandwidth means single-stream generation is measured, not blistering. Its superpower is holding 70B–120B-class models at all, and serving multiple users via batching.
Budget an evening, not a weekend. The genuinely fiddly parts of GPU servers (drivers, CUDA versions) are pre-solved. Most of your time goes into the decisions above, not debugging.
Try before you buy. Everything in this guide runs identically on a rented GPUwerk node, same OS, same hardware, root over SSH. An hour of testing answers "will my workload fit" better than any spec sheet.