DGX Spark vs NVIDIA A100 80 GB for LLM inference and fine-tuning: specs, tok/s, rental and used prices

An A100 80 GB is the faster machine on anything that fits in it: 2,039 GB/s of HBM2e on the SXM version against the DGX Spark's 273 GB/s, and 200.90 against 56.74 tok/s on llama.cpp's Llama 2 7B decode test. The Spark has 48 GB more memory and FP8 and FP4 tensor math that Ampere lacks, and it rents for $0.79 an hour against $1.19 to $2.32 for an on-demand A100 80 GB on 7 November 2026 (Silicon Data's A100 index: $1.59). By my arithmetic the A100 is about a third cheaper per token in batch serving and slightly dearer for a single user.

A100 speeds are third-party reports; GPUwerk has run no A100. GPUwerk publishes this page and rents the Spark, so weigh the comparison with that in mind.

A100 80 GB vs DGX Spark: specs side by side

The A100 has 7.1 to 7.5 times the bandwidth and about 3.4 times the BF16 tensor throughput that a burn test measured on GB10 units; the Spark has 1.6 times the memory and the newer formats.

Read 7 November 2026: NVIDIA A100 page, NVIDIA DGX Spark hardware page. The Spark FP16 figure is a synthetic burn test on eight GB10 units by Petronella (before a September 2026 firmware update; 77.6 to 81.3 after), set against the A100's specification, so the 3.4 times is a rough ratio. Measured Spark compute is the subject of the sibling guide on DGX Spark TFLOPS.

The missing formats matter in practice. vLLM's FP8 documentation says FP8 computation needs compute capability 8.9 or newer; on the A100 an FP8 checkpoint loads as weight-only W8A16. vLLM's gpt-oss recipe runs the MXFP4 model on Ampere through Marlin kernels and the Triton attention backend, while TensorRT-LLM refused the same model in GPUStack's A100 test with "WFP4A16 MoE is unsupported on SM80". The format differences themselves are in NVFP4 vs MXFP4 vs FP8.

Which models fit in 80 GB and which need 128 GB?

Every 70B model at 4-bit and gpt-oss-120b fit one A100; 8-bit 70B and the 100B-class MoE quants are where 80 GB runs out. I subtract 1.5 GB for the CUDA context (my assumption) and give the Spark about 110 GB after its operating system.

File sizes from Hugging Face, read 7 November 2026. Llama 3.3 70B's FP16 cache is 320 KiB per token by its config (80 layers, 8 KV heads, head dimension 128), which converts the free memory into tokens. My arithmetic throughout; the memory calculator does the same for other models.

With MIG, one A100 can also split into seven 10 GB instances for several small models, something the Spark cannot do. The gpt-oss-120b model page covers that model's Spark settings in detail.

How much faster is an A100 on the same model?

About 3.5 times in single-stream decode on a small dense model and more under heavy batching; prefill is only about 1.4 times faster on llama.cpp's test.

GPUStack ran one A100 SXM 80 GB (its page describes a two-GPU node; the runs without tensor parallelism use one card) and gives time per output token rather than single-stream tok/s, so 91 is 1,000 divided by 11.03 ms at four concurrent requests. The Spark rows come from different engines, workloads and dates, so read them as orders of magnitude. The ceiling divides each machine's bandwidth by about 37 GB of AWQ weights.

The 3.5 times decode gap is less than the 7.5 times bandwidth gap, most likely because a 3.56 GiB model is too small to keep an A100's memory system busy (my inference); on a 70B dense model the gap should approach the bandwidth ratio, as the ceiling row shows. Prefill depends on compute, and the A100's lead there is smaller. Long prompts on a 70B are where the Spark feels slow: GPUwerk measured Llama 3.3 70B prefill at 391 tok/s on an 8,192-token prompt, about 21 seconds before the first token. GPUStack also found engine choice matters as much on Ampere as the card: SGLang v0.5.3 produced 329.69 output tok/s against vLLM's 2,276.40 on the same A100 and workload.

Which is better for fine-tuning?

The A100 for compute-bound jobs that fit in 80 GB; the Spark for jobs that only fit in 128 GB. Fine-tuning is mostly matrix math, and the A100's 312 TFLOPS of dense BF16 is about 3.4 times the 88.6 to 91.4 TFLOPS FP16 that Petronella's burn test measured on GB10 units. NVLink at 600 GB/s also lets A100 servers scale one job across eight GPUs, where two Sparks share a 200 Gb/s link.

# memory for full fine-tuning with mixed-precision AdamW (my accounting, before activations)
bytes per parameter = 2 (BF16 weights) + 2 (gradients) + 4 (FP32 master copy) + 8 (two Adam moments) = 16

Llama 3.2 3B : 3.2B x 16 = 51 GB  -> fits A100 80 GB and Spark
8B model     : 8B   x 16 = 128 GB -> fits neither without offload or 8-bit optimizers
70B QLoRA    : about 35 to 40 GB of 4-bit weights plus adapters and activations -> fits both

NVIDIA's own Spark fine-tuning figures do not settle the speed question. Its October 2025 technical blog gives Llama 3.3 70B QLoRA as 759.79 tokens per second in its table and 5,079.4 in its text, and the other two rows differ by the same factor of about six, so I cannot tell which is right. The fine-tuning guide covers the Spark workflow. For a LoRA run on a 7B to 30B model, a rented A100 will likely finish sooner; for a QLoRA run on a 100B-class MoE model whose 4-bit weights exceed 70 GB, only the Spark of the two holds it.

What does an A100 80 GB cost to rent in November 2026?

From $1.19 to $2.32 per GPU-hour on demand on the providers' own pages, 1.5 to 2.9 times the Spark's $0.79. All rows read 7 November 2026, US dollars, excluding tax.

Compute Census put the median on-demand A100 rate at $1.75 on 16 October 2026 across the providers it tracks (Compute Census). Lambda's page says its listed prices take effect on 16 November 2026. H100, H200 and B200 rates for the same providers are in H100, H200 and B200 rental prices.

Hourly price is not cost per token. Using the throughput rows above, my arithmetic for gpt-oss-120b output tokens:

cost per million tokens = hourly price / (tok/s x 3,600 / 1,000,000)

Heavy load
  A100 SXM, RunPod Community $1.39/h at 2,276.40 tok/s -> $1.39 / 8.19 M  = $0.17 per M
  DGX Spark rental $0.79/h at 862.8 tok/s            -> $0.79 / 3.11 M  = $0.25 per M
One user
  A100 at about 91 tok/s                              -> $1.39 / 0.327 M = $4.26 per M
  DGX Spark at 60.7 tok/s                             -> $0.79 / 0.219 M = $3.62 per M

The heavy-load rows use different workloads (ShareGPT on the A100, 1,500-token prompts with JSON output on the Spark), so the ratio is approximate. Most users do not keep either machine at full load.

A busy A100 is the cheaper token factory; for one person the two are close, and the Spark holds bigger models for the money. The comparison with the next generation up is on DGX Spark vs H100.

Should you buy a used A100 or a DGX Spark?

For a desk, the Spark; a used A100 is a datacenter part that needs a server around it. I could not verify a used A100 80 GB sale price on 7 November 2026: eBay and the price comparison sites I tried blocked automated access. Compute Census gives an indicative $7,000 to $9,500 for Q3 2026, from Compute Exchange via IntuitionLabs, and labels it as asking prices and indicative ranges rather than observed sales. That is at or above the price of a new 128 GB Spark at $6,950, before the A100's host system, 300 to 400 W of cooling and power, and no warranty.

Rent against buy is simple arithmetic. At Silicon Data's $1.59 an hour, $8,250 (the middle of that range) buys about 5,190 A100 hours, or seven months around the clock; the Spark's $6,950 buys about 8,800 hours at $0.79, a calculation the rent-or-buy page works through with your own usage.

Verdict: when does each win?

• Serving a model under about 75 GB to many users: rent A100s, or an H100 if the budget allows. Batch throughput per dollar favors the A100 by about a third in my calculation above.
• LoRA or full fine-tuning that fits in 80 GB: A100, for its 312 TFLOPS of BF16 and NVLink.
• Models of 75 to 110 GB, or long contexts on 70B: the Spark. Qwen3.5-122B-A10B at 4-bit does not leave the A100 room for a cache.
• FP8 or NVFP4 checkpoints: the Spark computes in those formats; the A100 only stores them.
• One developer evaluating models: the Spark at $0.79 an hour. Single-user cost per token came out slightly lower in my calculation, and the memory is larger.

FAQ

Sources

Every figure above is attributed to its source in the sentence or table where it appears. Pages were read on 7 November 2026 unless a date says otherwise.

• NVIDIA A100 Tensor Core GPU, NVIDIA
• DGX Spark hardware overview, NVIDIA
• Performance of llama.cpp on Nvidia CUDA (discussion #15013), ggml-org/llama.cpp on GitHub
• Optimizing GPT-OSS-120B Throughput on NVIDIA A100 GPUs, GPUStack
• GPT OSS recipe, vLLM recipes
• FP8 W8A8 (LLM Compressor), vLLM documentation
• Pricing, RunPod
• GPU pricing, Hyperstack
• Pricing, JarvisLabs
• Pricing, Verda
• GPU cloud pricing, Lambda
• A100 Rental Price Index (SDA100RT), Silicon Data, 2026-11-07
• NVIDIA A100 Price: New, Used and Rented in 2026, Compute Census, 2026-11-01
• How NVIDIA DGX Spark's performance enables intensive AI tasks, NVIDIA Technical Blog, 2025-10-24
• DGX Spark firmware update: what we learned on 8 GB10 units, Petronella Technology Group, 2026-09-26
• gpt-oss-120b-GGUF files, ggml-org on Hugging Face
• Qwen3.5-122B-A10B-GGUF files, Unsloth on Hugging Face