Docs/Quantization
Reference

Quantization on a DGX Spark: MXFP4, NVFP4, AWQ, GGUF explained

Samuel Seidel · Published September 9, 2026

A Spark has 128 GB of unified memory and 273 GB/s of bandwidth to move weights through it. Quantization is what decides whether a model fits in the first number and how fast it runs against the second. This page tells you what MXFP4, NVFP4, AWQ and GGUF are, which engine runs each, which format the published Spark measurements and the GPUwerk presets use, and which one to pick.

Why quantization matters on this box

Weight size is parameters times bytes per parameter. A 4-bit format stores each parameter in roughly half a byte, against 2 bytes at FP16, so quantizing a model to 4-bit is usually the difference between it fitting on one Spark and not. That arithmetic, and the reason decode speed is bandwidth-bound rather than compute-bound, is worked through in full on the benchmarks page and the models page. This page does not repeat that math; it covers the formats themselves.

The formats, and what they are

Four names come up constantly around Spark deployments, and they are not interchangeable options for the same job. Two are hardware formats, one is a portable community format, and one belongs to a different engine entirely.

MXFP4

MXFP4 is a 4-bit floating point format with native support on Blackwell, the GB10's architecture. gpt-oss-120b ships at MXFP4. Published Spark measurements put it at roughly 60.5 tok/s under llama.cpp and 33.5 tok/s under vLLM, both single-stream, sourced on the benchmarks page. (The GPUwerk console preset for gpt-oss-120b uses a different build, Unsloth's UD-Q4_K_XL GGUF under llama.cpp.)

NVFP4

NVFP4 is NVIDIA's own 4-bit floating point format, also with native Blackwell support, and it is the format NVIDIA ships its Nemotron family in. Published measurements cover two Nemotron models at NVFP4 under vLLM: Nemotron Super 49B at 5.79 tok/s single-stream, a dense model reading all 49B parameters per token, and Nemotron-3-Super-120B-A12B at 22.7 to 23.7 tok/s, a mixture-of-experts model reading only its 12B active parameters. Same format, same engine, an order of magnitude apart in speed, because the parameter count that matters for decode is active, not total.

AWQ

AWQ is a 4-bit integer format from the open-source community rather than a vendor. It has no Blackwell-specific hardware path the way MXFP4 and NVFP4 do, but it has the broadest model coverage of the 4-bit formats and is the well-trodden path in vLLM when a model has no native FP4 build. GPUwerk's own fleet numbers for Qwen3-Coder-30B-A3B, 80.9 tok/s single-stream and 3,235 tok/s aggregate at 256 concurrent, are AWQ under vLLM, with the raw benchmark log at qwen3-coder-30b-awq-spark.txt. The Qwen3 Coder 30B console preset is that AWQ build. The Llama 3.3 70B fleet measurement was AWQ too; that model is no longer a console preset.

GGUF

GGUF is not a peer of the three formats above. It is llama.cpp's own container and quantization format, covering everything from Q4_K_M to full precision, and it runs under llama.cpp or Ollama, not vLLM. A model available only as GGUF gets served through llama.cpp; a model available only as AWQ or NVFP4 gets served through vLLM. That split is the whole reason gpt-oss-120b's two rows on the benchmarks page use two different engines: the GGUF build runs on llama.cpp, and the same MXFP4 weights are also available as a vLLM-compatible checkpoint.

FormatTypeEngineMeasured on a Spark
MXFP44-bit float, native BlackwellvLLM or llama.cppgpt-oss-120b
NVFP44-bit float, native Blackwell, NVIDIA's ownvLLMNemotron Super 49B, Nemotron-3-Super-120B-A12B
AWQ4-bit integer, community, broad coveragevLLMQwen3-Coder-30B-A3B, Llama 3.3 70B (both on the GPUwerk fleet)
GGUFllama.cpp's own container and quantization schemellama.cpp, Ollamagpt-oss-120b (llama.cpp build), Qwen3-Coder-30B (Q8_0)

Which one to reach for

Choose the engine first, then the format:

  1. One user at a time (you, on your own instance): llama.cpp with a GGUF build. It is the faster single-stream engine.
  2. A shared endpoint (an app, a team, an agent fleet): vLLM, with the 4-bit format the model ships in: NVFP4 for a Nemotron-family build, AWQ otherwise, since most models have an AWQ build.
  3. Check what exists. Not every model has builds for both engines. gpt-oss-120b and Qwen3-Coder-30B have both (the benchmarks page measures each on both engines). The Nemotron numbers here are vLLM-only; search for a community GGUF conversion before assuming there is none.

To compare both engines on your own model, rent a DGX Spark by the minute and run each one. How NVFP4, MXFP4 and FP8 differ in design and in published accuracy is covered in NVFP4 vs MXFP4 vs FP8.

FAQ

What is the difference between MXFP4, NVFP4 and AWQ?

MXFP4 and NVFP4 are both 4-bit floating point formats with native support on Blackwell, the GB10's architecture; NVFP4 is NVIDIA's own scheme and is what the Nemotron family ships in. AWQ is a 4-bit integer format from the wider open-source community, not tied to any one vendor's silicon, and it is the more broadly supported path in vLLM when a model has no native FP4 build. The published Spark numbers on the benchmarks page are for gpt-oss-120b at MXFP4, Nemotron Super 49B and Nemotron-3-Super-120B-A12B at NVFP4, and Qwen3-Coder-30B-A3B at AWQ (the last one measured on the GPUwerk fleet).

Is GGUF the same thing as MXFP4 or AWQ?

No. GGUF is llama.cpp's own file format and family of quantization types. A GGUF file runs under llama.cpp or Ollama, not vLLM; AWQ and NVFP4 checkpoints run under vLLM, not llama.cpp. A GGUF file can carry MXFP4 weights, which is how gpt-oss-120b runs on llama.cpp. Pick the format by the engine you serve from, not by preference.

Which quantization format should I use on a DGX Spark?

For a single user, GGUF under llama.cpp: it is the faster single-stream engine and gpt-oss-120b measures roughly 60.5 tok/s decode that way, against 33.5 tok/s under vLLM. For a shared endpoint, use vLLM with whichever 4-bit format the model actually ships in: NVFP4 when a Nemotron build exists, AWQ otherwise, since AWQ has the broadest model coverage of the 4-bit options.

See alsoFull benchmark numbers across engines See alsoWhich models fit in 128 GB
First top-up: pay $20, get $30 in credit

Test a format on real hardware, not a spec sheet.

A dedicated Spark, deployed in minutes.

Create account