DGX Spark vs RTX 4090 for LLMs: what fits in 24 GB vs 128 GB, tok/s, prefill, power and price

An RTX 4090 decodes models that fit in its 24 GB about 2.7 to 3.3 times faster than a DGX Spark (221.95 against 83.43 tok/s on gpt-oss-20b, 188.96 against 56.74 on llama.cpp's Llama 2 7B test) and processes prompts 1.8 to 4 times faster. It cannot hold Llama 3.3 70B or gpt-oss-120b; the Spark's 128 GB runs gpt-oss-120b at 58.7 tok/s in llama.cpp, against about 22 tok/s on a 4090 with experts offloaded to system RAM. On 7 November 2026 a used 4090 sold for a $2,731 eBay median and the 128 GB Spark listed at $6,950.

Rival speeds are third-party reports; GPUwerk has run no RTX 4090. The two llama.cpp gpt-oss-20b figures are from different builds (6210 and 7941) and slightly different tests (tg128 and tg32). The PC price is my sum, explained below.

RTX 4090 vs DGX Spark: specs side by side

The 4090 has 3.7 times the bandwidth and less than a fifth of the memory. That one ratio predicts most of what follows, because token generation reads every active weight once per token.

Read 7 November 2026: NVIDIA RTX 4090 page (memory, power, NVLink), The FPS Review (bus, clock and 1,008 GB/s), NVIDIA DGX Spark hardware page, vLLM FP8 documentation (FP8 compute from compute capability 8.9). Full Spark specs are on the specs page.

The FP4 row matters if you plan to use NVFP4 checkpoints: on a 4090 they load as compressed weights with 16-bit math, as the RTX 4090 vs RTX 5090 article explains with vLLM's hardware table. The architecture row cuts the other way. A 4090 PC is x86, where every Python wheel exists; the Spark is Arm64 with a newer GPU target, and some packages still need nightly builds, as the PyTorch on DGX Spark guide covers.

Which models fit in 24 GB and which need 128 GB?

Everything up to about 35B parameters at 4-bit fits a 4090 with a 32k context; 70B dense models, gpt-oss-120b and the 100B-class MoE models do not. The table adds each GGUF file to its KV cache and a 1.5 GB allowance for the CUDA context and buffers (my assumption), against 24 GiB (25.8 GB) on the 4090 and about 110 GB usable on a Spark after the operating system.

KV bytes per token = full-attention layers x KV heads x head dim x 2 (K and V) x 2 bytes (FP16)

Llama 3.3 70B (80 layers, 8 KV heads, head dim 128):
  80 x 8 x 128 x 2 x 2 = 327,680 bytes per token
  x 32,768 tokens      = 10.74 GB
  total = 42.52 GB (Q4_K_M) + 10.74 + 1.5 = 54.8 GB   -> no on a 4090, yes on a Spark

gpt-oss-120b (18 of 36 layers full attention, 8 KV heads, head dim 64):
  18 x 8 x 64 x 2 x 2 = 36,864 bytes per token -> 1.21 GB at 32k
  total = 63.39 + 1.21 + 1.5 = 66.1 GB                  -> no on a 4090, yes on a Spark

File sizes from the Hugging Face listings for Llama 3.3 70B, gpt-oss-120b, Qwen3.6-35B-A3B and Qwen3.5-122B-A10B, read 7 November 2026. Layer and head counts from the Llama 3.3, gpt-oss-120b and Qwen3.8-27B config files; the Qwen models' linear-attention layers keep a small fixed state that I left out. All sums are mine. Per-model cache sizes for other models are in KV cache size by model.

Context is the trap on the 4090. The same Qwen3.8-27B that fits at 32k overflows at 128k unless you quantize the cache, while the Spark holds it at any length the model supports. If your workload is long-document RAG or a coding agent that resends 100k tokens, the 24 GB card runs out before the model does.

How much faster is an RTX 4090 on models that fit?

About three times faster in decode and two to four times in prefill, close to the 3.7 times bandwidth ratio. The pairs below use the same model and quantization on both machines; only the first two rows also share one benchmark setup.

Tokens per second, single stream. The scoreboard rows are different contributors' machines and llama.cpp commits (4090 at 2241453, Spark at 5acd455). Hardware Corner's 4090 figure is Qwen3 30B-A3B at a 4k context; the Spark figure is GPUwerk's own Qwen3-Coder-30B-A3B AWQ run on vLLM 0.27.1, a different engine and quant.

The bandwidth arithmetic agrees. Divide each machine's bandwidth by the bytes read per token and the 4090's ceiling is 3.7 times the Spark's; the measured decode leads of 2.7 to 3.3 times sit just under that, because engines reach a different share of peak bandwidth on each machine. Prefill depends on compute instead, and the gap there ranges more widely: 4.0 times on a dense 7B, 1.8 times on gpt-oss-20b. For a single user chatting with a 20B model, both are fast enough; for batch jobs over many short prompts the 4090 finishes sooner.

What happens when the model does not fit the 4090?

The Spark wins outright, by about 2.7 times on gpt-oss-120b and by default on dense 70B. A 4090 can still run a large MoE model by keeping expert weights in system RAM with llama.cpp's --n-cpu-moe, and reported runs collected in the MoE CPU offload article put gpt-oss-120b at about 22 tok/s on a 4090 with --n-cpu-moe 28. The Spark runs the same 63.39 GB file entirely in its own memory at 58.72 tok/s decode and 2,443.91 tok/s prefill on llama.cpp build 7941.

Prefill suffers most under offload. On a 24 GB RTX 3090 with 64 GB of DDR5-5200, one owner measured gpt-oss-120b at 26.90 tok/s decode but only 364.80 tok/s prefill on a 2,048-token prompt (#15396, build 7760, January 2026), against 2,443.91 on the Spark. Larger batches helped: the same owner reached 672.56 tok/s on a 32,768-token prompt. I found no matching 4090 prefill figure, but the RAM, not the card, sets that limit. By my arithmetic a 30,000-token prompt takes about 45 seconds on that offload setup and under 20 seconds on the Spark, whose llama.cpp prefill rate falls from 2,443.91 to 1,567.08 tok/s as the context fills to 32k.

Dense 70B has no workaround on one 4090. Llama 3.3 70B needs about 55 GB at a 32k context by the calculation above; on a Spark GPUwerk measured it at 6.0 tok/s single-stream (AWQ, vLLM 0.27.1, 2 September 2026), slow but usable for batch work. Two 24 GB cards change that picture, which the sibling comparison of a multi-RTX 3090 rig against the Spark covers with reported 70B speeds.

What does each cost in November 2026, and what does it cost to run?

A 4090 PC costs about $1,700 less than a 128 GB Spark and draws several times more power. The used-card price is the eBay sold median of $2,731 (99 sales over 90 days to 7 November 2026) recorded in the RTX 4090 vs RTX 5090 article; new 4090s have been out of US retail since July 2025 according to the same page. The Spark's $6,950 US list price and its history are on the price page.

# RTX 4090 PC, parts from ComputingForGeeks' budget LLM build (Amazon US, 20 October 2026)
Ryzen 7 9700X                         $309.99
ASRock X870E Taichi Lite              $299.99
64 GB DDR5-6000                       $999.99
Corsair HX1200i                       $279.99
Enthoo Pro 2 Server case              $179.99
Samsung 990 PRO 2 TB                  $409.99
Thermalright PA120 SE                  $31.41
                                    ---------
Platform                            $2,511.35
+ used RTX 4090 (eBay median)       $2,731.00
= about $5,242

DGX Spark 128 GB (US list)          $6,950
DGX Spark 64 GB (partners)          from $4,999

Part prices from ComputingForGeeks, which listed them for a dual-RTX 3090 build; I removed its two cards and added one used 4090. Its 64 GB of RAM is the amount the MoE offload article suggests for gpt-oss-120b on a 24 GB card. US prices exclude sales tax. The sum is mine.

Per dollar the answer flips with model size. On gpt-oss-20b the 4090 PC gives 42 decode tok/s per $1,000 against 12 for the Spark; on gpt-oss-120b it gives about 4 against 8.4, by my division of the speeds above. The 64 GB Spark undercuts the 4090 PC, but 64 GB does not leave room for gpt-oss-120b's context.

Power is a smaller line but not zero. NVIDIA rates the 4090 card at 450 W; the Spark's whole system runs on a 240 W supply and drew 60 to 90 W during LLM inference in ServeTheHome's measurements, collected in the power guide. At Eurostat's EU household average of EUR 0.2832 per kWh for the first half of 2026, the card alone at its rating costs EUR 0.13 an hour and the Spark at 90 W EUR 0.03, my arithmetic. Single-stream decode rarely holds a 4090 at 450 W, so treat that as a ceiling, and add the rest of the PC.

Verdict: which should you buy?

• Models under about 20 GB, one person, speed matters: a used RTX 4090. It is roughly three times faster and costs less as a complete PC. I would buy one for gpt-oss-20b, Gemma-class 26B MoE models or a 27B at 4-bit with modest context.
• 27B to 35B with long context or several sessions: neither is ideal. 24 GB is tight; a 32 GB RTX 5090 or a Spark gives room, compared in DGX Spark vs RTX 5090.
• gpt-oss-120b, Qwen3.5-122B-A10B or other 60 to 110 GB models: the Spark. A 4090 either cannot load them or runs them from system RAM at a third of the speed with slow prompts.
• Dense 70B for one user: the Spark holds it at 6 tok/s; two 24 GB cards are faster if you will build and power a bigger PC.
• Gaming, CUDA on Windows or x86-only tools: the 4090. The Spark is an Arm Linux box.

If you are on the fence because of one large model, test that model before buying. A rented GPUwerk Spark runs it on the same GB10 chip by the minute, and the memory calculator checks the fit for your context length first.

FAQ

Sources

Every figure above is attributed to its source in the sentence or table where it appears. Pages were read on 7 November 2026 unless a date says otherwise.

• GeForce RTX 4090 specifications, NVIDIA
• NVIDIA GeForce RTX 4090 Founders Edition Review (specifications), The FPS Review, 2022-10-11
• DGX Spark hardware overview, NVIDIA
• Performance of llama.cpp on Nvidia CUDA (discussion #15013), ggml-org/llama.cpp on GitHub
• guide: running gpt-oss with llama.cpp (discussion #15396), ggml-org/llama.cpp on GitHub, 2025-08-18
• DGX Spark benchmark results (benches/dgx-spark), ggml-org/llama.cpp on GitHub, 2026-02-05
• RTX 4090 LLM benchmarks, Hardware Corner, 2026-03
• Llama-3.3-70B-Instruct-GGUF files, Unsloth on Hugging Face
• Llama-3.3-70B-Instruct config.json, Unsloth on Hugging Face
• gpt-oss-120b-GGUF files, ggml-org on Hugging Face
• gpt-oss-120b config.json, OpenAI on Hugging Face
• Qwen3.6-35B-A3B-GGUF files, Unsloth on Hugging Face
• Qwen3.5-122B-A10B-GGUF files, Unsloth on Hugging Face
• Qwen3.8-27B config.json, Qwen on Hugging Face
• FP8 W8A8 (LLM Compressor), vLLM documentation
• Dual RTX 3090 for Local LLMs: 70B Benchmarks and Build, ComputingForGeeks, 2026-10-21
• Electricity price statistics, Eurostat