Does it fit on a DGX Spark? LLM memory calculator
A 70B model at 4-bit needs about 37 GiB of weights, which leaves about 84 GiB of a DGX Spark's 121.6 GiB addressable unified memory for KV cache and overhead; at FP16 (about 140 GB) the same model does not fit. gpt-oss-120b fits too: its MXFP4 weights total 65.2 GB (60.8 GiB), leaving about 61 GiB. Enter your own model below; the formulas and their limits follow the calculator. I run GPUwerk, which rents DGX Sparks, so weigh the framing with that in mind.
Two worked answers
| Model | Weights | Left of 121.6 GiB | Verdict |
|---|---|---|---|
| Llama 3.3 70B, AWQ 4-bit | about 37 GiB (GPUwerk log) | about 84 GiB, about 82 GiB after 8,192 tokens of KV for one user | Fits. Memory leaves room for a team; speed does not (GPUwerk measured 6.0 tok/s single-user) |
| gpt-oss-120b, MXFP4 | 65.2 GB (60.8 GiB), Hugging Face repo | about 61 GiB before KV and overhead | Fits. Between the 40 and 75 GB lines, so a few users fit if the KV cache stays small |
The 37 GiB Llama weights, the 6.0 tok/s and the 121.6 GiB figure are from GPUwerk's vLLM doc, Llama 3.3 70B page and benchmarks page. The gpt-oss-120b size is the sum of the MXFP4 safetensors files in openai/gpt-oss-120b on Hugging Face, checked 2026-10-11; llama.cpp reports the MXFP4 GGUF as 59.02 GiB in discussion #16578. GPUwerk has not measured gpt-oss-120b's memory footprint. Subtraction and KV size are my arithmetic.
Calculator
Everything runs in your browser and nothing is stored or sent. Pick a preset or type your own numbers. Results appear below the form and update as you type.
The 4 GiB overhead default is my assumption, based on the models page's "a few gigabytes" for activations and the CUDA context. The 64 GB option scales GPUwerk's observed 121.6 GiB by half; GPUwerk has not run that variant. Two-node mode assumes weights and KV split evenly.
How the estimate works
Three formulas, all plain arithmetic:
- Weights (GB) = total parameters in billions x bytes per parameter. A 70B model is about 140 GB at FP16, 70 GB at FP8 and 37 GB at 4-bit, the same figures as the models page.
- KV cache (bytes) = 2 x layers x KV heads x head dimension x bytes per value x tokens x sequences. The 2 is one key and one value per head.
- Decode ceiling (tok/s) = 273 GB/s divided by the GB of active-parameter weights read per token. This is an arithmetic ceiling, not expected speed. GPUwerk's benchmarks page puts real engines at roughly 50 to 80% of it and shows where GPUwerk's own runs fell.
Bytes per parameter are averages, not exact. FP16 is 2 and FP8 is 1. The 4-bit formats (INT4, AWQ, NVFP4, MXFP4) land between 0.5 and 0.56 once scales are counted, and this page defaults to 0.56. A 2-bit GGUF is about 0.3. Dynamic GGUF quants mix bit widths, so published file sizes beat any formula.
The verdict follows the models page rule of thumb. Under about 40 GB of weights per node leaves room to serve a team, over about 75 GB is a single-user model, and a negative headroom means it does not fit. The 40 to 75 GB band between them is my reading of that rule: it fits, but KV cache decides how many users fit.
Checking the formula against GPUwerk's logs
GPUwerk's vLLM runs logged how many KV tokens fit at 0.85 GPU memory utilisation. The logs are the ground truth; the formula only has to land near them.
| Model | KV per token | Logged KV tokens at 0.85 | KV size by formula | Cross-check |
|---|---|---|---|---|
| Llama 3.3 70B AWQ | 327,680 bytes (80 layers, 8 KV heads, 128 dim, FP16) | 207,120 | 63.2 GiB | 37 GiB weights + 63.2 GiB KV = 100.2 GiB, against a 0.85 x 121.6 = 103.4 GiB budget. The remaining 3.2 GiB is engine overhead. |
| Qwen3-Coder 30B-A3B AWQ | 98,304 bytes (48 layers, 4 KV heads, 128 dim, FP16) | 924,912 | 84.7 GiB | Leaves 18.7 GiB of the 103.4 GiB budget for weights and overhead. Parameters x 0.56 gives about 16 GiB of weights, which is consistent. |
KV tokens and the 37 GiB figure: GPUwerk's Llama 3.3 70B and Qwen3-Coder 30B-A3B pages and logs. Layer, head and dimension counts: the models' config.json files on Hugging Face (Llama 3.3 70B mirror, Qwen3-Coder, gpt-oss-120b), checked 2026-10-11. The cross-check sums are my arithmetic.
One unit wrinkle: GPUwerk's benchmarks page and Llama 3.3 70B page divide 273 GB/s by 37 to get a 7.4 tok/s ceiling, treating the logged 37 GiB as 37 GB. Converted (37 GiB is 39.7 GB), the ceiling is 6.9 tok/s, and the calculator uses that figure. Against 6.9, the measured 6.0 tok/s is about 87% of the ceiling, a little above the 50 to 80% band.
Why unified memory is different
A discrete GPU has its own VRAM, and the model either fits there or spills over a slow PCIe link. The DGX Spark has one pool of 128 GB LPDDR5x shared by the CPU and GPU. GPUwerk's logs show about 121.6 GiB of it addressable, and that has to cover the operating system, Docker, the page cache and any other process as well as your weights and KV cache.
NVIDIA's DGX Spark release notes (checked 2026-10-11) list display-reserved memory as adjustable in the July 2026 release: 2 GB by default, 4 GB via BIOS. If you run a display, check what your own system reports with free -h before sizing a model to the last gigabyte. GPUwerk also runs its Sparks without swap since a memory-hardening pass on 9 September 2026, so running out is an out-of-memory error, not a slowdown.
The pool is also why bandwidth rather than capacity limits speed: 273 GB/s feeds every token, so a dense 70B model reads about 37 GiB per token while a 3B-active mixture-of-experts model reads under 2 GB. See the specs page for the full hardware list.
Models that do not fit one Spark
The calculator includes two examples that fail. Both rest on aggregator pages, not model cards or first-party GPUwerk figures, so treat the sizes as indicative.
| Model | Size used | Source (aggregator) | Why it fails |
|---|---|---|---|
| DeepSeek V4.1-Flash, Q2_K | 324 GB (763.2B parameters) | llmrun.dev | About 2.5 times one Spark's 121.6 GiB (130.6 GB), and more than two Sparks hold together (about 261 GB) |
| Kimi K3, 4-bit | about 1,570 GB (2.8T parameters x 0.56) | slash-digital.io | 12 times the capacity of one Spark |
Checked 2026-10-11. The 4-bit Kimi size is my multiplication, not a published quant.
Two Sparks, and the 64 GB variant
Two linked Sparks give you two 121.6 GiB pools, not 256 GB for a single model. The calculator splits weights and KV evenly and checks each node, which is optimistic because real splits are rarely perfectly even. For setup, see the two-node cluster guide. To try a model on real hardware before buying, the how-to-rent page explains renting by the minute.
NVIDIA's product page lists a 64 GB variant sold through OEM partners (see the specs page), and The Register's 2 October 2026 report covers the announcement, including the unchanged 273 GB/s bandwidth (checked 2026-10-11). GPUwerk has not run one, so the calculator's 60.8 GiB figure is half of 121.6 and nothing more. At that size, the 4-bit 70B example (37 GiB) would leave about 24 GiB before overhead and KV.
FAQ
How much memory does a 70B model need?
Weights are parameters times bytes per parameter: about 140 GB at FP16, 70 GB at FP8 and 37 GB at 4-bit, as the models page lays out. On a DGX Spark with about 121.6 GiB addressable, FP16 does not fit, FP8 (70 GB) fits but sits just under the 75 GB line where it becomes a single-user model, and 4-bit leaves about 84 GiB before KV cache. Llama 3.3 70B adds 327,680 bytes of KV cache per token at FP16, so 8,192 tokens for one sequence is about 2.7 GB.
Does gpt-oss-120b fit on a DGX Spark?
Yes. The MXFP4 weights in OpenAI's Hugging Face repository total 65.2 GB (60.8 GiB), which leaves about 61 GiB of the 121.6 GiB addressable memory for KV cache and overhead. GPUwerk has not measured this model's footprint or speed. The roughly 60 tok/s often quoted for it under llama.cpp is a third-party report (tg32 on an empty context, falling to about 40 tok/s at 32,768 tokens of context), not a GPUwerk measurement.
What is unified memory for LLMs?
The DGX Spark has one 128 GB pool of LPDDR5x that the CPU and GPU both use, so there is no separate VRAM. Weights, KV cache, the operating system, Docker and the page cache all draw from the same pool, and a model that is 'too big for VRAM' is simply too big for what is left of that pool. It also means 273 GB/s of bandwidth, not capacity, sets decode speed.
Does two Sparks mean 256 GB for one model?
No. Two linked Sparks are two separate 121.6 GiB pools joined by a network link, not one 256 GB pool. A model has to be split across them, for example with tensor or pipeline parallelism, and each node still needs its share of the weights, KV cache and overhead. GPUwerk has not published a two-node inference benchmark, so this page checks fit per node and makes no speed claim. The two-node cluster guide covers the setup.
How accurate is this calculator?
It is an estimate, not an exact figure. It leaves out engine buffers, CUDA graphs, quantisation metadata beyond the bytes-per-parameter average, and engine-specific allocation such as vLLM's pre-sized KV pool. For Llama 3.3 70B AWQ the estimate and GPUwerk's benchmark logs agree within about 3 GiB, and the logs are the ground truth. Test a real load before you commit to a configuration.