DGX Spark memory bandwidth measured: how close real tests get to 273 GB/s

Measured GPU-side bandwidth on a DGX Spark is 200 to 247 GB/s, or 73 to 90% of NVIDIA's 273 GB/s, depending on the test: clpeak's global-memory read measured 234 to 247 GB/s and a STREAM triad 200 to 213 GB/s on four makers' GB10 units in June 2026. Single-user decode on dense models turns 73 to 87% of 273 GB/s into tokens in llama.cpp and vLLM, by my calculation, which is the same share an Apple M5 Max reaches and more than an RTX 5090 gets on the same 7B test. One CPU core sees only 38 GB/s.

Shares of 273 GB/s are my arithmetic. The uma_bw author's first public run reported impossible figures (millions of GB/s) before a fix, so I treat its 165.7 GB/s as low-confidence. NVIDIA's documentation lists 4,266 MHz and LPDDR5X-8533; 8,533 million transfers per second times 32 bytes is 273.1 GB/s.

What does NVIDIA's 273 GB/s mean?

It is the theoretical peak of the memory bus that the CPU and GPU share, and no published test reaches it. NVIDIA's hardware documentation lists "16 channels (256 bit) LPDDR5X 8533". Chips and Cheese notes that NVIDIA's Hot Chips slides allow up to 9,400 MT/s while dmidecode on a shipping unit reports 8,533, so the shipping figure is the lower one. The DGX Spark specs page explains what the shared pool means for VRAM; this page is about how much of the 273 GB/s you actually get.

The 64 GB OEM configuration carries the same 273 GB/s in NVIDIA's table, per the 64 GB vs 128 GB comparison. Nobody has published a bandwidth test of a 64 GB unit that I could find.

Why do STREAM, clpeak and PyTorch disagree?

They move different mixes of reads and writes, and the GB10 is fastest when it only reads. clpeak's global-memory test is read-dominated and lands at 242 to 247 GB/s. A STREAM triad computes a = b + s x c, two reads and one write per element, and lands at 200 to 213 GB/s. A device copy is one read and one write, 216 to 218 GB/s. memtest_vulkan shows the same split inside one run: writes at about 177 GB/s, the verifying reads at 193 to 215 GB/s.

My reading of this: decode reads weights and the KV cache and writes almost nothing, so the read-heavy clpeak figure is the better ceiling for tokens per second, and the triad figure is the better one for training steps and data processing that write as much as they read. The five triad results also bound unit-to-unit spread: 200 to 213 GB/s across a Founders Edition, an HP ZGX Nano, a Lenovo PGX and an ASUS GX10 on two drivers. That is 6%, small next to the gap between test types. Sustained behaviour by maker is the subject of the planned GB10 sustained performance by OEM guide.

What share of 273 GB/s does llama.cpp and vLLM decode reach?

Between 73% and 87% on dense models, by my calculation, which works out to about 200 to 240 GB/s of useful weight reads. Each generated token reads every weight once, so implied bandwidth is decode tok/s times the bytes of weights read per token. Dense models make this clean; mixture-of-experts models do not, because routing overhead and partly-read expert tensors blur the byte count.

GB per token is the file size reported by llama-bench converted from GiB (Llama 2 7B: the 3,825,807,040-byte file on Hugging Face), or parameters times bytes per weight for the vLLM and SGLang rows; the 70B AWQ figure is the one the benchmarks page uses. The embedding table is read one row per token, so these slightly overstate bytes read. Jeff Geerling's file is named Qwen3-30B-A3B, but llama-bench reports a dense qwen3 32B with 32.76B parameters, so I list it as Qwen3-32B.

The spread has a pattern. Simple 8-bit and 16-bit kernels (Q8_0, BF16 GEMV) reach 85 to 87%, the K-quants 73 to 79%, and an early SGLang FP8 run from launch month 60%. My inference is that dequantization work per byte, not memory, accounts for most of the gap on the K-quants. For gpt-oss-120b the benchmarks page's decode arithmetic uses about 3 GB per token; 58.72 tok/s on llama.cpp build 7941 makes that 176 GB/s, a lower share that fits the routing overhead of a mixture-of-experts model.

How does it compare with Strix Halo, Apple M5 and the RTX 5090?

Measured the same way, the Spark's memory system is as efficient as any of them; it is just much narrower than a discrete GPU. llama.cpp keeps public scoreboards for CUDA, Vulkan and Apple Silicon that all run one file, Llama 2 7B at Q4_0, with llama-bench's tg128 test. That gives the implied bandwidth below.

Implied GB/s is tg128 times the 3.83 GB file, my arithmetic. Spec bandwidth: Apple's from the scoreboard's own table, Strix Halo's from llm-tracker.info, the RTX 5090's from NVIDIA's comparison page and the RTX PRO 6000's from NVIDIA's product page. Scoreboards read 7 November 2026; each row names its llama.cpp commit on the scoreboard.

The large chips convert a smaller share because a 7B model is too small for them, in my reading. At 290 tok/s each token takes 3.4 ms on an RTX 5090, so fixed per-token costs such as kernel launches and sampling weigh more than they do at 17 ms per token on the Spark. The M5 Ultra also splits work across two dies. A microbenchmark points the same way for Strix Halo: rocm_bandwidth_test measured 212.4 GB/s inside the GPU, 83% of 256 (llm-tracker.info), against 89 to 90% for the Spark's clpeak read. Full head-to-heads are on DGX Spark vs Strix Halo and DGX Spark vs RTX 5090.

Does the CPU get the same bandwidth?

No. The 256-bit bus is there for the GPU, and the Arm cores reach a fraction of it. Chips and Cheese measured 38 GB/s from DRAM for one Cortex-X925 core and 26 GB/s for one Cortex-A725, with one cluster of five plus five cores pulling over 100 GB/s. Under a 231 GB/s GPU load, CPU memory latency went beyond 351 ns, against 113 ns idle. On a single-threaded copy with one tool across machines, the GB10 is the slowest of five:

All runs by Jeff Geerling with tinymembench, single thread. Multi-threaded CPU bandwidth on GB10 has not been published as one number; Chips and Cheese gives per-core and per-cluster figures only.

Two practical consequences. CPU-side inference on a Spark gets nowhere near GPU decode speed, and offloading layers to the CPU buys no capacity because there is no separate VRAM to overflow; the llama.cpp CPU offload guide is for discrete GPUs. And CPU-heavy preprocessing on the same box competes with the GPU for one bus. Core speeds are covered in the planned GB10 CPU performance guide. Copies between the CPU and GPU views of the same memory are slower again: 58.9 GB/s pinned on CUDA and 122.7 GB/s on Vulkan in clpeak.

Do drivers, OS updates or firmware change it?

Not measurably in the tests published so far, with one exception. clpeak's three global-memory results moved by under 0.5% between Ubuntu 24.04 with driver 580.159.03 and Ubuntu 26.04 with driver 610.43.02 on the same GX10. The GX10's triad went from 205 to 200 GB/s across the same driver change. memtest_vulkan's verify read rose from 193 to 215 GB/s, an 11% change, but only on Vulkan.

The exception is a unit stuck in a low-power state, which forum posters report clears only with a full power cycle. One owner saw clocks fall to about 513 MHz under load and everything run about three times slower (NVIDIA forum, February 2026); healthy units run near 2.6 GHz in the nvfp4bench thread. If your tg128 on the Llama 2 7B file is far under 57 tok/s, check that before blaming software.

What number should you plan with?

Use 220 GB/s for single-stream estimates on dense models: tok/s is about 220 divided by gigabytes read per token. Llama 3.3 70B at about 37 GB gives 5.9 tok/s by that rule, and GPUwerk measured 6.0. A mixture-of-experts model lands lower than the rule: Qwen3-Coder-30B-A3B's roughly 1.7 GB of active weights predicts 129 tok/s, and it measured 80.9. The memory calculator checks whether the model fits before you do this sum.

# read-heavy and triad bandwidth on an owned Spark (needs nvcc, CUDA 13)
git clone https://github.com/secYOUre/nvfp4bench && cd nvfp4bench
cmake -B build && cmake --build build -j
./build/nvfp4bench --bandwidth

# the scoreboard test, comparable with every row above
llama-bench -m llama-2-7b.Q4_0.gguf -ngl 99 -fa 0,1

My take: the Spark's bandwidth is not underdelivering. It converts its 273 GB/s as well as an M5 Max converts 614, and better than an RTX 5090 converts 1,792 on small models. It is a 273 GB/s machine, so pick models by bytes read per token, which is the whole case for mixture-of-experts models on this box. For many users at once the same bus is shared, as the planned concurrent users guide shows.

FAQ

Sources

Every figure above is attributed to its source in the sentence or table where it appears. Pages were read on 7 November 2026 unless a date says otherwise.

• DGX Spark hardware overview, NVIDIA documentation
• GB10 really does hit ~1 PFLOP NVFP4 (2:4 sparse), measured, with an open-source tool, NVIDIA Developer Forums, 2026-06-17
• Ubuntu 26.04 + drivers 610 + cuda-toolkit 13.3 + ZFS on GX10 (clpeak and memtest_vulkan results), NVIDIA Developer Forums, 2026-06-25
• DGX Spark GB10: Magistral 24B BF16 ~4.85 tok/s, GEMV decode, NVIDIA Developer Forums, 2026-10-20
• GB10 Hardware Baseline: First Direct Measurements and Findings, NVIDIA Developer Forums, 2026-04-25
• Inside Nvidia GB10's Memory Subsystem, from the CPU Side, Chips and Cheese, 2025-12-31
• Dell Pro Max with GB10 (NVIDIA Spark OEM model), sbc-reviews issue 92, Jeff Geerling on GitHub, 2025-11-21
• Framework Desktop Mainboard, sbc-reviews issue 80, Jeff Geerling on GitHub, 2025-08-07
• Intel Core Ultra 7 265K Custom PC, sbc-reviews issue 93, Jeff Geerling on GitHub, 2025-11-25
• M4 Mac mini, sbc-reviews issue 57, Jeff Geerling on GitHub, 2024-11-11
• M3 Ultra Mac Studio (512 GB), sbc-reviews issue 95, Jeff Geerling on GitHub, 2025-12-08
• Investigating 513MHz cap for GPU, NVIDIA Developer Forums, 2026-02-22
• Performance of llama.cpp on Nvidia CUDA, ggml-org/llama.cpp discussions, 2026-10-16
• Performance of llama.cpp with Vulkan, ggml-org/llama.cpp discussions, 2026-11-06
• Performance of llama.cpp on Apple Silicon M-series, ggml-org/llama.cpp discussions, 2026-10-27
• DGX Spark benchmark results (llama-bench), ggml-org/llama.cpp on GitHub, 2026-02-05
• Benchmark Dell Pro Max with GB10, ai-benchmarks issue 34, Jeff Geerling on GitHub, 2025-11-22
• Strix Halo, llm-tracker.info, 2025-08-08
• Compare GeForce Graphics Cards, NVIDIA
• NVIDIA RTX PRO 6000 Blackwell Workstation Edition, NVIDIA
• TheBloke/Llama-2-7B-GGUF files, Hugging Face