What to know before buying a DGX Spark

We sell these machines, we rent these machines, and we run a fleet of them. That gives us an obvious commercial interest, so this guide leads with the numbers that argue against buying one. If the Spark is wrong for your workload, we would rather you find out here than after the invoice clears.

The one number that decides everything: 273 GB/s

The DGX Spark pairs a GB10 Grace Blackwell superchip with 128 GB of coherent unified LPDDR5x memory. NVIDIA markets it at "1 petaFLOP of AI performance", that figure is sparse FP4, a best-case marketing number, not something you will see in a serving log. The number that actually governs your day-to-day experience is memory bandwidth: 273 GB/s, shared between the CPU and the GPU.

Autoregressive token generation is memory-bound. Every token requires streaming the active weights out of memory, so single-stream decode speed is roughly bandwidth divided by active model size. For context: an RTX 5090 has about 1.79 TB/s, the M5 Ultra Mac Studio announced August 25, 2026 has 1.2 TB/s (Apple; the outgoing M3 Ultra was about 800 GB/s), and an H100 about 3.35 TB/s of HBM3. The Spark is an order of magnitude behind the datacenter part and less than a quarter of a top Mac Studio.

Do the arithmetic before you buy. A 70B model at FP8 is ~70 GB of weights; 273 GB/s divided by 70 GB puts the theoretical ceiling under 4 tokens/sec, and LMSYS measured 2.7 tokens/sec decode on Llama 3.1 70B FP8 under SGLang at batch 1. That is not a benchmarking artifact or a driver bug. It is physics, and no firmware update will change it.

Published benchmark numbers

These are figures from third-party reviews, not our own marketing. Where sources disagree we say so below the table.

Where the sources disagree. Single-stream reviews and concurrency reviews reach opposite verdicts on the same box, and both are correct. LMSYS's batch-1 numbers make the Spark look slow; Dendro Logic's 120× throughput scaling from batch 1 to 256 concurrent streams makes it look like a bargain server. The disagreement is entirely about what you are measuring. If you are one person typing into a chat window, believe the batch-1 numbers. If you are running a batch extraction pipeline or serving a team, believe the concurrency numbers.

Third-party 70B-class Q4 figures also vary widely, some roundups quote 35–45 tok/s for the Spark on 70B Q4, which is hard to reconcile with LMSYS's 2.7 tok/s on 70B FP8. The gap is quantization (Q4 halves the bytes moved versus FP8) plus, in some cases, aggregate rather than per-stream reporting. Treat any 70B number above ~10 tok/s per stream with suspicion unless the article states the quantization and the batch size. Ours: expect single-digit to low-teens tok/s per stream on dense 70B, and 30s on MoE models like gpt-oss 120B where only a fraction of parameters are active per token.

The honest weaknesses

• Single-stream generation is slow, permanently. Covered above. If your mental model is "a 5090 but with 128 GB," recalibrate now.
• The ARM64 + CUDA 13 software situation is still rough. The GB10 is sm_121, supported only in CUDA 13.0+, while most pip-installable ML wheels, vLLM, flash-attn, and friends, are still built against CUDA 12.x on x86. PyTorch has needed custom index URLs rather than plain PyPI. Triton's sm_121 support was in effect papered over by treating it as sm_80, which disables Blackwell-specific paths; CUTLASS FP8 kernels have failed to dispatch on sm_121; optimized FP4/MXFP4 MMA kernels and MoE configs have lagged. Simon Willison's October 2025 assessment, great hardware, early days for the ecosystem, matched ours. It has improved through 2026, but you will still hit a package that assumes x86 and CUDA 12.
• Idle power is not what an Arm box should draw. Owner reports on the NVIDIA developer forums put idle draw at roughly 40 W, though we have not seen an instrumented measurement published. Peak system power is 240 W (140 W GB10 SoC plus ~100 W for ConnectX-7, SSD and USB-C provisioning). Forum posts in the same threads described the box topping out near 100 W under load rather than 240 W and questioned whether NVIDIA de-rated it before launch; we would treat those as anecdotes rather than measurements. NVIDIA's own clarification is that the 240 W is a whole-system peak, not a sustained SoC figure. Either way: it runs warm, and it is not free to leave on.
• Noise and heat are real but modest. It stays audible-but-tolerable at a desk under sustained load. It is a small fan moving air through a dense box, so it is not silent the way a Mac Studio is. There have also been scattered reports of spontaneous reboots under sustained stress on early units.
• The price moved the wrong way. Announced at $3,999, the Founders Edition list price rose to $4,699 in the US and roughly €4,800 on the EU marketplace. That price increase changed the buy-versus-rent math materially, and anyone quoting you the old $3,999 figure is working from stale information.

What it is genuinely good at

• Holding models nothing else at this price can hold. 128 GB of unified memory means a 70B–120B-class model loads and stays resident. A 32 GB 5090 cannot do that at any speed.
• Batched and concurrent work. The Dendro Logic pipeline went from 130 to 3,900 records/hour by switching from single-stream 9B to batched gpt-oss 120B on the same box, 30× faster and better output.
• Prefill. The Blackwell cores are compute-bound-friendly: ~8,000 tok/s prefill on 8B. Long-context RAG and document ingestion look far better than decode numbers suggest.
• Being a real DGX-stack machine. Same CUDA, same containers, same NIM/NeMo path as the datacenter. It is a legitimate development target for something that will later run on a GB200, which a Mac Studio never will be.
• Fitting under a desk on a normal wall socket with data that never leaves your building. For a lot of EU buyers that, not tokens/sec, is the whole purchase rationale.

Where to actually buy one in Europe

"DGX Spark" is the NVIDIA Founders Edition. The same GB10 board ships from OEM partners at different prices, and in the EU the OEM units have generally been cheaper and easier to get than the Founders Edition.

Two things buyers routinely miss. First, SSD capacity differs by SKU (1 TB to 4 TB) and model weights are large, a 1 TB unit fills faster than you expect. Second, EU list prices carry VAT that US comparisons do not; compare ex-VAT to ex-VAT or you will misjudge the gap by ~20%.

Total cost of ownership: buy vs rent vs build a 5090 rig

Three-year view, one machine's worth of capacity, EU electricity at roughly €0.25/kWh. Power assumes a realistic duty cycle rather than 24/7 full load.

How to read that table. Owning is dramatically cheaper than 24/7 rental, a Spark pays for itself against $2.90/hr in roughly nine weeks of continuous use. That is the honest case for buying, and it is a strong one. But almost nobody runs a dev box continuously. If your real usage is a few hours a day of bursty experimentation, on-demand rental at per-minute billing costs less than the capital and never leaves you holding depreciating silicon when the GB10 successor lands.

The 5090 rig is the right answer more often than Spark vendors like to admit. If everything you run fits in 32 GB, 7B to 32B models, image generation, fine-tuning small models, a 5090 is roughly 4× faster on both prefill and decode (LMSYS: 8,519 tok/s prefill and 205 tok/s decode on GPT-OSS 20B, versus 2,053 / 49.7 on the Spark) at a comparable or higher total system price: Tom's Hardware's GPU price tracker had 5090s at a $4,381–$4,829 street range in August 2026, well above the $1,999 MSRP, which puts a complete rig level with or above a Spark. Buy the Spark when the 128 GB is the point. Buy the 5090 when speed is the point. Anyone telling you the Spark beats a 5090 at things that fit in 32 GB is selling.

Two costs the table cannot show. Cloud GPU pricing is volatile, H100 on-demand rates move by double-digit percentages within a single quarter, and RTX 5090 rentals span $0.25 to $1.49/hr across providers, so a rental budget built on today's spot rate is a guess. And owned hardware carries an operational tax: firmware, driver upgrades on a rare ARM+CUDA-13 combination, backups, and the afternoon you lose when a vLLM release breaks sm_121. That tax is real whether or not it appears on an invoice; it is the reason our on-premise offering is managed rather than a box on a pallet.

So: is a DGX Spark worth it?

• Buy one if you need 70B–120B models running on hardware you physically control, if your work is batched or multi-user rather than one impatient human waiting on tokens, or if you are developing for the DGX/NIM stack and want a local target.
• Don't buy one if your models fit in 32 GB (get a 5090), if you need fast single-stream chat on large dense models (nothing at this price does that, the bandwidth is not there), or if your usage is a handful of hours a week (rent).
• Consider a Mac Studio if single-stream large-model inference at a desk is the whole job and CUDA is not required: 1.2 TB/s on the new M5 Ultra, or 614 GB/s on the $2,499 M5 Max, beats 273 GB/s on exactly that axis. You give up the CUDA ecosystem, ConnectX-7 clustering, and any path to datacenter parity.
• Rent first, either way. An hour on a rented Spark answers "does my model fit and is the speed tolerable" definitively, for less than the cost of lunch. Everyone we have talked out of a purchase this way has thanked us; so have the people who went ahead and bought after testing.

If you do buy, our setup guide covers first boot through serving a model, including the parts the quick-start skips.