NVFP4 vs MXFP4 vs FP8: how the formats differ, how accurate they are, and what runs them (November 2026)

NVFP4 stores 4-bit values in blocks of 16 with an FP8 scale per block and an FP32 scale per tensor (4.5 bits per value); MXFP4 uses blocks of 32 with a power-of-two scale (4.25 bits); FP8 spends 8 bits per value. In published post-training results with plain rounding, NVFP4 keeps 94.8 to 99.8% of 16-bit accuracy on 8B to 70B models while MXFP4 keeps 88.1 to 96.8% (ICLR 2026 MR-GPTQ study), and FP8 is effectively lossless (Red Hat). Native FP4 math needs Blackwell, including RTX 50 cards and the DGX Spark's GB10; Hopper and Ada run FP8 natively and FP4 only as weight-only. Checked 7 November 2026.

Format definitions: NVIDIA's NVFP4 introduction (24 June 2025), Table 1 of the Microscaling (MX) paper and the FP8 formats paper. The 128x128 FP8 block is DeepSeek-V4-Flash's weight_block_size; UE8M0 FP8 scales on SM100 are from the TensorRT-LLM docs. Weight sizes are my arithmetic for 120 billion parameters with every weight in that format; real checkpoints keep some layers at higher precision and come out larger.

How do NVFP4, MXFP4 and FP8 differ?

All three are floating-point formats with a shared scale; they differ in how many values share a scale and how precisely that scale is stored.

A 4-bit E2M1 value can only be 0, 0.5, 1, 1.5, 2, 3, 4 or 6 (plus sign), so the scale does most of the work. The bits-per-value figures follow from the block layout:

MXFP4: (32 values x 4 bits + one 8-bit E8M0 scale) / 32 = 4.25 bits
NVFP4: (16 values x 4 bits + one 8-bit E4M3 scale) / 16 = 4.50 bits
       (+ one FP32 scale per tensor, negligible)
FP8:   8 bits per value + per-tensor or per-block FP32 scales

MXFP4's E8M0 scale is a pure power of two. A block's largest value therefore lands anywhere between half of the grid's top and the top, and part of the 8-level grid goes unused on many blocks; that is my reading of why it loses accuracy. An E4M3 scale has 3 mantissa bits and can put the largest value close to 6 in nearly every block. NVIDIA measured the effect directly: mean squared error of 0.72 with E8M0 scale encoding against 0.08 with E4M3. Halving the block from 32 to 16 values also means one outlier distorts fewer neighbours.

FP8 is a different trade. E4M3 gives up range for an extra mantissa bit and E5M2 does the reverse; vLLM's FP8 KV cache accepts either. At 8 bits there is enough precision that a per-tensor or per-channel scale is usually sufficient, which is why FP8 works on Hopper and Ada without microscaling hardware.

How accurate is NVFP4 compared with BF16 and FP8?

Within about 1 point of FP8 or BF16 on large models in NVIDIA's published tables; the gap widens on small models and is wider for MXFP4.

W4A4 = 4-bit weights and activations; W4A16 = 4-bit weights, 16-bit activations. RTN = round-to-nearest, the simplest post-training method. The NVIDIA model cards are vendor results on vendor checkpoints; the MR-GPTQ and Red Hat numbers come from academic and Red Hat teams using vLLM. The Nemotron card does not label its first column; I read it as the BF16 baseline.

The MR-GPTQ authors attribute MXFP4's loss to its power-of-two scales and note that NVFP4's small groups blunt classic outlier-mitigation tricks, which is why their method helps MXFP4 most (88.1 to 93.3 on the 8B). Size matters more than format above 30B: on Llama-3.3-70B, RTN MXFP4 already keeps 96.8%. The oldest data point shows the floor. Casting LLaMA-7B straight to MXFP4 for weights and activations, with no calibration, took wikitext perplexity from 9.49 to 27.20 in the 2023 MX paper.

Why are gpt-oss and DeepSeek V4 fine in MXFP4 and FP4?

Because they were trained into the format, so no post-training error is added at download time.

The gpt-oss-120b card says the model was "post-trained with MXFP4 quantization of the MoE weights" and that all evaluations used the same MXFP4 weights; there is no BF16 version to lose accuracy against. DeepSeek-V4-Flash ships as "FP4 + FP8 Mixed", with FP4 experts. NVIDIA's NVFP4 pretraining paper trained a 12B model on 10 trillion tokens in NVFP4 and reports MMLU-Pro of 62.58 against 62.62 for the FP8 run, using random Hadamard transforms, stochastic rounding and a few layers kept in higher precision. Nemotron 3 Super is the first Nemotron 3 model pretrained that way; its reported Spark speeds are on the Nemotron-3-Super-120B page.

The practical rule: a model's native format beats any conversion. Converting gpt-oss from MXFP4 to NVFP4 is rarely worth it, and NVIDIA's DeepSeek-V4.1-Flash card shows that converting MXFP4 experts to NVFP4 moved six scores by between -0.4 and +1.2 points while the checkpoint grew from about 476 to 492 GiB, which fits 4.5 against 4.25 bits per expert weight.

Which GPUs run NVFP4, MXFP4 and FP8 natively?

FP4 tensor-core math needs Blackwell; FP8 needs Ada or newer; older cards fall back to weight-only kernels.

Native columns from the TensorRT-LLM support matrix; weight-only fallbacks from the vLLM hardware table (Marlin covers GPTQ, AWQ, FP8 and FP4 on Ampere, Ada and Hopper, always with 16-bit activations; Turing lacks Marlin MXFP4) and the SGLang docs (SM80 to SM90 use a Marlin W4A16 fallback for NVFP4). GB10's sm_121 and the 1 PFLOP sparse FP4 figure are on GPUwerk's DGX Spark specs page and llama.cpp doc; the SM121 row is my inference from it being Blackwell, since the TensorRT-LLM matrix does not list SM121.

Weight-only still saves memory: a 4-bit checkpoint is a quarter of BF16 on an RTX 4090 or RTX 3090 too. What you lose is the compute speedup in prefill and batched serving. The MR-GPTQ authors measured about 6x layer-wise and nearly 4x end-to-end over FP16 on an RTX 5090 with native FP4 kernels, against about 3.6x and 2.2x on a B200. For single-user decode, which is bandwidth-bound, the gain comes mostly from reading fewer bytes, so a 4-bit model decodes at similar speed on either path.

Which engines support each format in November 2026?

vLLM, SGLang and TensorRT-LLM run all three; llama.cpp runs MXFP4 widely and NVFP4 since April 2026.

• vLLM detects NVFP4 (ModelOpt or compressed-tensors) and MXFP4 checkpoints from the config, so you leave --quantization unset; the vLLM DGX Spark writeup (1 June 2026) calls NVFP4 "the biggest practical advantage" on a Spark. KV cache options are FP8 only.
• SGLang uses --quantization modelopt_fp4 for NVFP4, nvfp4_online to quantize MoE experts at load on SM100/103, and quark_mxfp4 for MXFP4 on AMD.
• TensorRT-LLM has the widest native matrix and the only NVFP4 KV cache, which needs an offline ModelOpt checkpoint with FP8 weights (what KV precision buys in memory is in KV cache size by model).
• llama.cpp runs MXFP4 GGUFs (gpt-oss, and unsloth's MXFP4_MOE quants). Blackwell-native NVFP4 merged on 28 April 2026 in PR #22196. On a DGX Spark with Nemotron 3 Super, that PR reports perplexity 4.6237 for Q4_K, 4.6283 for the NVFP4 integer fallback and 4.6577 for the W4A4 tensor-core path, and prompt processing (pp2048) up from 506 to 611 tok/s.

Setup on a Spark, including which build to use and which flag forces a working MoE backend, is on the quantization page and the vLLM doc; for consumer Blackwell, see vLLM on the RTX 5090.

Where do you find NVFP4 checkpoints, and which should you trust?

On Hugging Face, from the model vendor or NVIDIA first; read the card for which layers are FP4 and for an accuracy table.

A Hugging Face API search on 7 November 2026 returned 4,761 repositories with NVFP4 in the name and 1,908 with MXFP4 (search page). Of the NVFP4 ones, 87 were under nvidia, 63 under RedHatAI and 19 under unsloth. Vendors now ship their own: Mistral publishes Mistral-Large-4-1T-A52B-NVFP4 and Mistral-Small-4-119B-2603-NVFP4 next to the full checkpoints.

Three things to check on a card. Which layers are NVFP4: NVIDIA's Qwen3.8-27B build keeps attention and linear attention in FP8 and quantizes only the MLP and output head. The calibration data and method: that card used 2,048 samples and nvidia-modelopt 0.48.0. And whether there is a benchmark table against the original; if a community repo has none, you are the test set. The quantized vs full-precision guide covers how to run that check on your own prompts.

My recommendation: on Blackwell (an RTX 5090, an RTX PRO 6000 or a DGX Spark), use the model's native format if it was trained in one, otherwise an NVFP4 checkpoint from the vendor or NVIDIA that shows less than a point of loss on the card; use FP8 when memory allows and you want no doubt. On an RTX 4090 or H100, use FP8, and on an RTX 3090 use AWQ or a GGUF quant, since FP4 there only saves memory. The RTX 5090 is the cheapest card with native FP4. I would not convert a working FP8 or BF16 model to MXFP4 myself with plain RTN; the published numbers say NVFP4 or a calibrated method keeps several more points.

FAQ

Sources

Every figure above is attributed to its source in the sentence or table where it appears. Pages were read on 7 November 2026 unless a date says otherwise.

• Introducing NVFP4 for Efficient and Accurate Low-Precision Inference, NVIDIA Technical Blog, 2025-06-24
• Microscaling Data Formats for Deep Learning, arXiv (ar5iv), 2023-10-19
• FP8 Formats for Deep Learning, arXiv, 2022-09-29
• Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization, arXiv (ICLR 2026), 2025-10-16
• "Give Me BF16 or Give Me Death"? Accuracy-Performance Trade-Offs in LLM Quantization, arXiv, 2025-05-30
• Pretraining Large Language Models with NVFP4, arXiv, 2026-03-04
• Optimizing Inference for Long Context and Large Batch Sizes with NVFP4 KV Cache, NVIDIA Technical Blog, 2025-12-08
• nvidia/Qwen3.8-27B-NVFP4 model card, NVIDIA on Hugging Face, 2026-09-08
• nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 model card, NVIDIA on Hugging Face, 2026-03-11
• nvidia/DeepSeek-V4.1-Flash-NVFP4 model card, NVIDIA on Hugging Face, 2026-10-14
• openai/gpt-oss-120b model card, OpenAI on Hugging Face
• Quantization: supported hardware, vLLM docs
• Quantization, SGLang docs
• Quantization, TensorRT-LLM docs
• ggml-cuda: Blackwell native NVFP4 support, PR #22196, llama.cpp on GitHub, 2026-04-28
• vLLM on the DGX Spark: Architecture, Configuration, and Local Evaluation, vLLM blog, 2026-06-01
• Models search: NVFP4, Hugging Face