How to run a newly released model on a DGX Spark (GB10)

To run a new open-weight model on a DGX Spark, first check that the published weight size of your chosen build fits in the Spark's roughly 121.6 GiB of addressable memory with room for the KV cache. Then serve it with the arm64 container image the model card or NVIDIA's DGX Spark playbooks name, or build llama.cpp for sm_121 if a GGUF exists. As of 18 October 2026, Qwen3.8-27B, Nemotron 3.5 Super VL and Nemotron 3.5 Bolt fit one Spark, GLM-5.3-Flash fits only as a 2-bit or smaller GGUF, MiMo-V2.6-Flash needs two Sparks and DeepSeek-V4.1-Flash needs more than two.

None of the five checks below needs a download, and the first one can save you a 100 GB download that was never going to load. The table further down applies them to the models released in the last few months.

Does the new model fit in 128 GB?

A Spark has 128 GB of unified memory per NVIDIA's product page, but about 121.6 GiB of it is addressable for a model, which is what vLLM reports on GPUwerk nodes (see the memory calculator). The weights, the KV cache, activations and the CUDA context all come out of that one pool.

Don't estimate from the parameter count in the model name. Read the file sizes. A repo's "120B" can mean a 75 GB download at NVFP4 or 249 GB in BF16, and some releases (DeepSeek-V4.1-Flash, below) carry extra lookup tables on top of the backbone. You can sum the weight files without downloading anything:

# total size of a repo's weight files, in GB
curl -s "https://huggingface.co/api/models/<org>/<repo>?blobs=true" | python3 -c '
import json, sys
d = json.load(sys.stdin)
s = sum(f.get("size") or 0 for f in d["siblings"]
        if f["rfilename"].endswith((".safetensors", ".gguf")))
print(round(s / 1e9, 1), "GB")'

For a GGUF repo with several quantizations in subfolders, this adds them all up, so look at the file listing on the repo page for the one you want. Convert to GiB (divide GB by 1.074), subtract from 121.6, and put what is left into the memory calculator with the model's layer and KV-head counts from its config.json. If less than about 10 GiB remains, plan for a short context or a smaller build.

Is there an arm64 build of the engine?

GB10 pairs an ARM64 CPU with a Blackwell GPU at compute capability 12.1 (sm_121). Two separate things have to be right: the container or wheel has to be built for linux/arm64, and the CUDA code inside it has to target sm_121. NVIDIA's vLLM troubleshooting page for DGX Spark lists "Image not built for ARM64" as a cause of container startup failures, and the llama.cpp playbook builds with CMAKE_CUDA_ARCHITECTURES=121a-real for the second reason.

Check the image before you pull it:

# the manifest should list "architecture": "arm64"
docker manifest inspect vllm/vllm-openai:v0.32.0 | grep -i arm64

# on the Spark: the GPU should report compute capability 12.1
nvidia-smi --query-gpu=name,compute_cap --format=csv,noheader

Model-specific tags are the usual release-week path. Docker Hub lists linux/arm64 builds for the vLLM tags v0.32.0, qwen38 and mimov25-cu129, for an arm64-only tag glm53-flash-arm64-cu130, and for SGLang's latest-cu130 and nemotron3-5-super tags (checked on Docker Hub on 18 October 2026). An arm64 tag is not proof that the kernels target GB10: NVIDIA's GLM-5.3-Flash NVFP4 card, which names the arm64-only tag, lists GB200 as its test hardware. Treat a clean startup log as a hint, and send a real completion request before you trust it.

NVIDIA also publishes a multi-architecture vLLM container on NGC (nvcr.io/nvidia/vllm), whose notes mention DGX Spark. Its tags are versioned by year and month (the newest on 18 October 2026 was 26.10.pre1-py3), and NVIDIA's troubleshooting page recommends it when a startup fails on a missing ARM64 image. For a model released this week, though, we would start with the upstream vLLM tag its card names, since a monthly container may predate the model.

And pip? It works better than it used to: vLLM 0.32.0 on PyPI ships a manylinux_2_28_aarch64 wheel. If you go that way, confirm PyTorch sees the GPU before you blame the model:

python3 -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_capability())"
# expect: True (12, 1)

Where are NVIDIA's DGX Spark playbooks?

At build.nvidia.com/spark. The ones that matter for a new model, as listed on 18 October 2026:

• Serve LLMs with vLLM: one Spark or two clustered Sparks. Its recipe step sends you to the vLLM recipe catalog filtered to DGX Spark for anything beyond its default (Qwen3.8-27B NVFP4), and warns that the container, vLLM version, quantization format, parsers and parallel settings all have to match the model, not only its size.
• Serve LLMs with SGLang: uses lmsysorg/sglang:latest-cu130 and lists the models validated on Spark. For NVFP4 checkpoints it adds --quantization modelopt_fp4.
• Serve Models with llama.cpp: last updated 9 October 2026. It builds from source for 121a-real and says any GGUF that fits in memory works with the same workflow.
• Quantize Models to NVFP4: for when a model ships only in BF16 and no one has published an NVFP4 build yet.

The playbooks lag new releases by weeks. In release week the model card is usually the better source, and NVIDIA's own cards are often specific: the Nemotron 3.5 Super VL NVFP4 card has a vLLM command it describes as intended "for DGX Spark (GB10) only", with --gpu-memory-utilization 0.75 and one concurrent sequence, plus a separate SGLang command for one Spark.

NVFP4 or GGUF for a new model?

NVFP4 is a 4-bit floating-point format for Blackwell GPUs. NVIDIA's quantization playbook puts the memory saving at about 3.5 times against FP16 and 1.8 times against FP8, and serves the result with TensorRT-LLM or vLLM; the SGLang playbook loads NVFP4 checkpoints too. NVIDIA publishes NVFP4 checkpoints of other labs' models under the nvidia/ org on Hugging Face. For DeepSeek-V4.1-Flash and GLM-5.3-Flash the NVIDIA repo appeared six and eight days after the original; for Qwen3.8-27B it took about a month (repo creation dates from the Hugging Face API, and an original repo can be created before its public release).

GGUF is llama.cpp's format. Its advantage in release week is range: community quantizers publish a ladder of sizes, and the small end is often what decides whether a model fits one Spark at all. Unsloth's GLM-5.3-Flash GGUF repo runs from 93.1 GB (UD-IQ1_S) to 199.7 GB (UD-Q4_K_XL) and beyond. The cost is quality: below 4 bits it drops, and Unsloth's own documentation warns against its roughly 1-bit tier for agentic work, as our GLM-5.3-Flash page notes.

Our rule of thumb: take an NVFP4 build from the model's own lab or from NVIDIA if it fits, and fall back to GGUF when only a sub-4-bit file fits or when you want llama.cpp for a single user. The quantization formats page has the longer version.

Which engine supports a new model first: llama.cpp, vLLM or SGLang?

No single answer holds from one release to the next. Read the model card's deployment section and check what it actually names. From the cards of the models in the table below, read on 18 October 2026:

• vLLM was named on every card except DeepSeek's own, which points to its own inference code (NVIDIA's NVFP4 card for it names vLLM), often with a model-specific image tag or a nightly build rather than a stable release. The Nemotron 3.5 Bolt card names a nightly image and a reasoning-parser file to download, and vLLM is the only runtime it lists.
• SGLang was named for Qwen3.8-27B, Nemotron 3.5 Super VL (with a one-Spark command), GLM-5.3-Flash and MiMo-V2.6-Flash. For DeepSeek-V4.1-Flash NVFP4, the NVIDIA card says SGLang loads only the previous revision of the checkpoint.
• llama.cpp depends on someone converting the model. ggml-org, the organisation behind llama.cpp, published GGUFs for MiMo-V2.6-Flash and Nemotron 3.5 Super VL; NVIDIA published the GGUF of Nemotron 3.5 Bolt itself.

Checkpoints also change after release. NVIDIA's DeepSeek-V4.1-Flash NVFP4 repo was revised on 14 October 2026, and its card says stock vLLM images, including the model-specific one, cannot load the new revision without vLLM pull request 61530. Pin both the image tag and the model revision once something works. For the flags each engine needs on GB10, see the engine comparison.

Recent open-weight models: fits one Spark, needs two, needs more

Checked on Hugging Face on 18 October 2026. Sizes are the sums of the weight files in each repo's listing, in GB; the verdict compares them with 121.6 GiB per Spark. These are fit checks, not speed measurements: GPUwerk has not benchmarked any of these models.

Parameter counts and licences come from each model card. MiMo-V2.6-Flash is listed by its RL checkpoint; Xiaomi also publishes a -MOPD variant with the same size. The "about 52 GiB", "about 11 GiB" and "about 4 GiB" figures are my arithmetic against 121.6 GiB, before KV cache and overhead.

When do you need two Sparks?

When no build you are willing to run fits one Spark with room for context. Two Sparks are not a faster Spark: they are two 121.6 GiB pools joined over ConnectX-7, and the two-node cluster guide explains why single-user decode speed should stay close to the single-node figure. Our pricing page puts it the same way: the linked cluster's 256 GB is total memory across two nodes, not a single pool, and it needs a runtime that supports distributed execution.

That last point narrows your options in release week. NVIDIA's vLLM playbook covers two clustered Sparks, and its recipe step says to confirm a matching two-node recipe exists in spark-vllm-docker rather than assume a one-Spark recipe will scale across two. The SGLang and llama.cpp playbooks list no multi-node path for Spark. NVIDIA's clustering documentation covers the cabling.

Before renting a second node, check whether a smaller build fits one. GLM-5.3-Flash at UD-IQ1_S is the clearest trade-off case: it fits, with a quality cost you should measure on your own prompts.

A release-week run, start to finish

1. Sum the file sizes of two or three builds (original, NVFP4, a GGUF) with the command above and put them in the memory calculator.
2. Read the card's deployment section. Note the exact image tag, any nightly hash, required parsers, and the hardware it was tested on.
3. Run docker manifest inspect on that tag and look for arm64. If there is none, use the newest arm64 tag of the same engine and expect to fix things.
4. Start with a short context (--max-model-len 32768 in vLLM) and one sequence, then raise both once it answers.
5. Send a real request to /v1/chat/completions and check the answer makes sense; then pin the image tag and the model revision.

On a GPUwerk Spark, Docker, CUDA and the NVIDIA stack are preinstalled and you get root in your own container, so these steps run as written. New to renting? How to rent a Spark covers the first steps. A new release won't be a console preset: Qwen3.8-27B, for example, isn't one, per its model page. The models index lists the ones we have written up.

FAQ

Can a DGX Spark run a model the week it is released?

Often, if the weights fit and an arm64 build of the engine supports the architecture. Check the published weight size against about 121.6 GiB of addressable memory, then use the container image the model card or NVIDIA's DGX Spark playbooks name. Some cards, such as Nemotron 3.5 Super VL, include a DGX Spark command; others, such as DeepSeek-V4.1-Flash NVFP4, need an engine build that includes a specific pull request.

Should I pip install vLLM on a DGX Spark?

vLLM 0.32.0 ships a manylinux aarch64 wheel on PyPI, so pip can work, but new model support usually lands first in a dedicated container tag. NVIDIA's vLLM playbook for DGX Spark uses containers and lists a missing ARM64 image as a cause of startup failure. Check that the image has a linux/arm64 entry before pulling it.

NVFP4 or GGUF for a new model on GB10?

NVFP4 is NVIDIA's 4-bit floating-point format for Blackwell, served by vLLM, SGLang and TensorRT-LLM. GGUF is llama.cpp's format and comes in more sizes, down to sub-2-bit Unsloth Dynamic quants. Use NVFP4 when an official or NVIDIA checkpoint exists and fits; use GGUF when you need a smaller file to make the model fit one Spark, or want llama.cpp's single-user speed.

When does a new model need two DGX Sparks?

When no build you trust fits in one Spark's roughly 121.6 GiB with room for KV cache. As of 18 October 2026, MiMo-V2.6-Flash (MXFP4 GGUF 167.4 GB) needs two Sparks, GLM-5.3-Flash needs two at 4-bit, and DeepSeek-V4.1-Flash (smallest build checked 365.7 GB) needs more than two. Two Sparks are two memory pools linked over the network, so the engine must support distributed execution.