DGX Spark cluster with 4 or 8 nodes: what NVIDIA supports, which switch, and the tok/s you get

NVIDIA has officially supported DGX Spark clusters of up to eight nodes since 22 October 2026, either through a switch or as a switchless ring, for 1 TB of memory in total. Its setup tool, NVIDIA Sync, still configures at most three Sparks cabled directly or four through a switch, with eight-node support due by the end of 2026, so bigger clusters are set up by hand. Four or more nodes on a switch need a 200 GbE model with QSFP56-DD ports and RoCE v2. In NVIDIA's own benchmark Qwen3.8-27B NVFP4 decodes at 11.86 tok/s on one Spark, 36.34 on four and 52.58 on eight.

Memory is 128 GB per node. About 121.6 GiB of each is addressable, and the nodes are separate pools joined by 200 Gb/s links, not one shared memory (see the memory calculator). Two-node setup is covered in the two-node cluster guide and not repeated here.

Does NVIDIA support more than four DGX Sparks?

Yes, but only its technical blog says so yet. The post of 22 October 2026, published for GTC Berlin, announces "official support for eight-node DGX Spark clusters" and benchmarks 1, 2, 4 and 8 nodes. NVIDIA's other pages had not caught up when I read them on 7 November 2026:

• The product page FAQ still says "Up to 4 DGX Spark systems can be connected" and that "clustering 4 systems requires an external network switch". It rates four Sparks at models "up to 700B parameters" and recommends a power-of-two node count.
• The clustering docs say the Cluster Assistant "supports up to three DGX Spark systems connected directly through cables, and up to four systems when using a switch".
• The Connect Multiple DGX Sparks and vLLM cluster playbooks cover "two to four" Sparks and require the April 2026 DGX OS release or later.
• Only the switch playbook on GitHub says the setup "can be scaled up to more DGX Spark devices connected via the same switch".

So the eight-node claim that circulated on blogs is right, with a caveat: it is supported, not automated. Expect to configure addressing, NCCL and the inference engine yourself until NVIDIA Sync catches up.

What switch do I need for a DGX Spark cluster?

A managed 200 GbE switch with QSFP56-DD ports and RoCE v2; NVIDIA has not named a model. The product page lists a "200 GbE switch (QSFP56-DD, RoCE v2)" for four Sparks. The switch playbook asks for a "QSFP switch with at least 4 QSFP56-DD ports (at least 200Gbps each)", accepts 400 Gb/s ports split by breakout cables, and wants all Spark ports in one bridge because some switches allow hardware offload on one bridge only. It also warns that auto-negotiation can land below 200 Gb/s; check with ethtool and force 200G-baseCR4 if needed. In NVIDIA's 2025 forum thread on switches, the reply quoted from NVIDIA said only that clustering is supported "with compatible QSFP cables and Ethernet switches".

In practice owners use MikroTik. The spark-vllm-docker networking guide by eugr, a co-author of NVIDIA's eight-node post, names the CRS812-DDQ and CRS804-DDQ as examples.

Petronella Technology Group sells Spark cable kits, so read its figures as a vendor's. It measured 196 Gb/s per node through a CRS812 using both PCIe halves of the port, and 109 Gb/s for a single flow across the two switches. It used RouterOS 7.24.1 and had to set breakout lanes to 200G by hand because they did not auto-negotiate.

How do I cable four or eight Sparks?

One cable per Spark into the switch, or two per Spark in a ring. Each Spark has two QSFP ports, and each port shows up as two Linux interfaces because the ConnectX-7 connects through two PCIe Gen 5 x4 links; the networking guide puts each half at up to 100G and says you need both for full bandwidth. NVIDIA's approved direct-attach cables are the Amphenol NJAAKK-N911 (400 mm; NJAAKK0006 is the 0.5 m version) and the Luxshare LMTQF022-SD-R, per the clustering docs.

• Switch (star): the switch playbook uses the second QSFP port, the one farther from the RJ45 jack, on every Spark (enp1s0f1np1 and enP2p1s0f1np1). Management traffic stays on the 10 GbE port or Wi-Fi.
• Ring: port 0 of each Spark goes to port 1 of the next, and the last closes the loop. A ring connects only neighbors: in an eight-node ring 8 of the 28 node pairs have a direct link, per Petronella, and NVIDIA's blog says a ring "cannot express" a split into two groups of four.

Cable cost, as one reseller prices it: $568 for a four-node ring kit, $874 for a four-node switch fabric plus the switch, and $1,748 to $1,907 for an eight-node fabric plus two switches (Petronella, updated 26 October 2026).

How do I set up NCCL and RoCE on four or more Sparks?

Same OS on every node, one subnet per interface half, jumbo frames, then NCCL tests before any model. The outline below follows NVIDIA's playbooks and eugr's guide; take exact flags from the current pages, since they change between DGX OS releases.

1. Update every node to the same DGX OS release (April 2026 or later), create the same username on all of them and set up passwordless SSH. The playbook's discover-sparks script does the SSH part.
2. On a switch: enable the ports, put them in one bridge, and confirm 200000Mb/s per link with ethtool.
3. Address each interface half on its own subnet. The switch playbook uses 192.168.100.x/24 and 192.168.101.x/24; eugr's guide says "DO NOT use the same subnet on both twins" and defaults to MTU 9000.
4. Build NCCL and nccl-tests. NVIDIA's NCCL playbook (updated 14 October 2026) builds NCCL v2.31.2-1 for sm_121 and pins nccl-tests v2.19.7.
5. Run an all-gather test across all nodes, then start the engine with NVIDIA's vLLM cluster playbook or spark-vllm-docker.

The four-node test from the switch playbook:

export UCX_NET_DEVICES=enp1s0f1np1
export NCCL_SOCKET_IFNAME=enp1s0f1np1
export OMPI_MCA_btl_tcp_if_include=enp1s0f1np1

mpirun -np 4 -H <IP1>:1,<IP2>:1,<IP3>:1,<IP4>:1 \
  --mca plm_rsh_agent "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no" \
  -x LD_LIBRARY_PATH=$LD_LIBRARY_PATH \
  $HOME/nccl-tests/build/all_gather_perf

For a ring, eugr's setup-cluster.sh takes the nodes in physical cable order and writes the NCCL settings a ring needs (NCCL_ALGO=Ring, NCCL_NET_PLUGIN=none, NCCL_IB_SUBNET_AWARE_ROUTING=1, NCCL_IB_MERGE_NICS=0):

./setup-cluster.sh 192.0.2.11 192.0.2.12 192.0.2.13 192.0.2.14 --topology ring --dry-run
./setup-cluster.sh 192.0.2.11 192.0.2.12 192.0.2.13 192.0.2.14 --topology ring

The same guide says physical NCCL inference has been validated on four nodes and that larger rings still need validation. On a switch, NVIDIA's blog adds ROCEnante, a Spark-tuned all-to-all backend merged into FlashInfer that falls back to NCCL if its startup test fails.

How fast is a four- or eight-Spark cluster?

Decode grows with node count but well short of linear; prefill barely moves. These are the published runs I could find:

NVIDIA's Qwen rows use llama-benchy at concurrency 1 with no speculative decoding. Its GLM-5.3 row uses multi-token prediction, which explains most of the gap to Petronella's 15.1 tok/s. NVIDIA also reports near-linear scaling for training: nanochat pre-training ran 7.79x faster on eight nodes, 97.4% efficiency. None of these are GPUwerk measurements.

The pattern is what the hardware predicts. Tensor parallelism splits each layer's weights, so each node reads less per token and decode rises; but every layer then waits on an exchange over a 200 Gb/s link, and that cost grows with node count. On a dense 27B model the eighth node adds less than the second did. A ring loses more because traffic has to hop through neighbors: NVIDIA measured the eight-node ring about 24% behind the switch on decode. For an MoE model like Qwen3.8-Flash-Next, Petronella's single-user figure barely moved from four to eight nodes while total throughput at 32 streams kept rising. More nodes mostly buy capacity and concurrency, not per-user speed.

What can 512 GB or 1 TB run?

Four nodes hold roughly a 700B-parameter model at 4-bit and eight nodes about twice that, before KV cache. My arithmetic, using about 117.6 GiB per node for weights and KV after OS overhead:

4 nodes: 4 x 117.6 GiB = 470 GiB = 505 GB
         at 0.56 bytes per parameter (4-bit with scales): about 900B parameters
8 nodes: 8 x 117.6 GiB = 941 GiB = 1,010 GB
         at 0.56 bytes per parameter: about 1.8T parameters
Subtract KV cache for your context and users from these totals.

NVIDIA's own rating for four Sparks is "up to 700B parameters", which leaves room for context. On eight nodes the published runs are GLM-5.3 at about 470 GB in NVFP4, and a 492 GiB DeepSeek-V4.1-Flash build that loaded but could not yet serve in September 2026. Later runs with newer checkpoints, and how many Sparks each of the newest large MoE releases needs, are in the guide to large MoE models on multiple Sparks.

What do four or eight Sparks cost compared with an 8-GPU server?

An eight-node cluster costs about $60,000 to buy, which is roughly 40 days of renting an 8x B200 node. The calculation:

Hardware prices exclude VAT, shipping and power. Cable prices are Petronella's; switch price is MikroTik's suggested price. Lambda's page, read 7 November 2026, says these prices take effect on 16 November 2026. $60,000 divided by $61.52 is about 975 hours.

The two are not the same machine. An 8x B200 node keeps each GPU's weights in its own HBM and links the GPUs over NVLink, so it serves many users at speeds a 200 Gb/s Ethernet cluster cannot approach; the H100, H200 and B200 rental price comparison covers that market and the 8-GPU server page explains how such servers are quoted. Eight Sparks draw far less power: the supplies are rated 240 W each, and the power guide puts inference at 60 to 90 W per Spark.

My verdict: build four Sparks if you need a 400B to 700B model on your own desk for a few users, and accept 30 to 40 tok/s class speeds on mid-size dense models. Go to eight only for 1T-class models or for training experiments, where NVIDIA's numbers show near-linear scaling. For a team-facing endpoint, rent GPUs by the hour. Either way, test a pair first: GPUwerk rents two cabled Sparks at $1.79 per hour (pricing), enough to see how your model scales with tensor parallelism before you buy a switch. If you are weighing a single big box instead, see DGX Spark vs DGX Station.

FAQ

Sources

Every figure above is attributed to its source in the sentence or table where it appears. Pages were read on 7 November 2026 unless a date says otherwise.

• Running Frontier-Class AI Models Across 8 NVIDIA DGX Spark Nodes, NVIDIA Technical Blog, 2026-10-22
• NVIDIA DGX Spark product page and FAQ, NVIDIA
• DGX Spark clustering, NVIDIA
• DGX Spark Release Notes, NVIDIA
• Connect Multiple DGX Sparks for Distributed Workloads, NVIDIA build.nvidia.com
• Connect Multiple DGX Spark through a Switch (playbook README), NVIDIA on GitHub, 2026-07-21
• Set Up NCCL for Multi-Node GPU Communication, NVIDIA build.nvidia.com, 2026-10-14
• Serve LLMs with vLLM Across DGX Sparks, NVIDIA build.nvidia.com, 2026-10-22
• Connecting multiple DGX Spark Units Ethernet Switch Recommendations, NVIDIA Developer Forums, 2025-09-23
• spark-vllm-docker: cluster networking, eugr on GitHub
• DGX Spark Cluster: 4 to 8 Nodes, Switch vs Ring, Petronella Technology Group, 2026-10-26
• CRS812 DDQ, MikroTik
• Lambda pricing, Lambda