Qwen3.8-Flash-Next on a DGX Spark

Qwen3.8-Flash-Next fits on one 128 GB DGX Spark at 4 bits: a community NVFP4 recipe loads 76.48 GiB of weights and keeps the 47.68 GiB n-gram table on NVMe. Third parties report 41.7 to 50 tok/s single-stream on one Spark and up to 72.0 tok/s on two, after tuning; GPUwerk has not measured it. The console lists it as a llama.cpp preset, and this page covers the fit, that preset and the reported speeds, each with its source and date.

What it is

Parameter counts, licence name, context and release month are from the Hugging Face model card, checked 2026-10-11. Together the 125B, 51B and 4B figures come to 180B parameters; at BF16 that would be roughly 360 GB (my arithmetic), so the full-precision checkpoint does not come close to fitting. The preset size is from the console configuration; the NVFP4 sizes are from ai-muninn part 45, 2026-09-06.

How it fits in 121.6 GiB

A Spark has 128 GB of unified memory, of which about 121.6 GiB is addressable (see which models fit in 128 GB). The single-Spark vLLM recipe in ai-muninn part 45 reports 76.48 GiB of NVFP4 weights, a 17.04 GiB FP8 KV cache and a 262,144-token maximum context. That adds up to 93.52 GiB, leaving about 28 GiB (my subtraction, not a reported figure).

The n-gram table is why it works. The same post reports it as 47.68 GiB, read from NVMe instead of loaded: in its staged mode, a thread pool reads the rows each forward pass needs before the pass runs. Held in memory it would push the total to about 141 GiB, past the limit. The cost is startup time: the post reports a cold start of 12 minutes 54 seconds, of which 585.81 seconds went to weights.

The recipe is not stock vLLM. It pins a vLLM nightly and bind-mounts nine Python files, four of them custom n-gram and MTP implementations and the rest upstream pull requests and a modified quantization file. Treat it as a community patch set that may break when the nightly moves.

For a rough ceiling, only the active parameters are read per token. At 4 bits, 6B active parameters are roughly 3 GB per token, so the bandwidth ceiling on 273 GB/s is near 90 tok/s (the formula and its caveats are on the benchmarks page). The reported 41.7 tok/s is under half of that; the post itself estimates its weight reads at about 60% of the 273 GB/s spec. The 90 tok/s is arithmetic, not a measurement, and the recipe's MTP speculative decoding means it is not a hard limit either. To size your own context and cache against the 121.6 GiB limit, use the memory calculator.

Run it on GPUwerk

The console lists Qwen3.8 Flash Next as a llama.cpp preset. From the console configuration, checked 2026-10-11:

• Engine: llama.cpp, served as model qwen3.8-flash-next.
• Weights: Unsloth Dynamic UD-Q3_K_XL GGUF in three shards, about 90 GB loaded.
• Context: 32,768 tokens split across 2 parallel slots, with a q8_0 KV cache and flash attention on.
• Loading: --no-mmap, so the file is read into memory rather than paged from disk.

The Q4 build was passed over because it left too little memory for parallel slots. A deploy succeeds only on a node that has these weights staged; if none has, it is refused and nothing is charged. A Spark is $0.79/hour, billed per minute against a prepaid balance (pricing). The configuration notes call this the newest architecture among the presets and the least community-tested, so expect rough edges.

Run it yourself

I do not publish launch commands for this model, because GPUwerk has not tested any on a fleet node. The recipes below are the sources of the reported speeds, and all of them are not tested by GPUwerk.

• One Spark, vLLM, NVFP4: ai-muninn part 45, including the nine patched files and the NVMe-resident table.
• One Spark, SGLang, NVFP4: the repository linked from the NVIDIA forum thread.
• Two Sparks, SGLang, TP=2: the pocharlies recipe, which keeps the 51 GB n-gram tables in FP8, reports about 74 GiB of weights per rank and a boot of about 8 minutes, and does not use the NVMe route.

For llama.cpp, build it and serve a GGUF as in the llama.cpp doc; the preset settings above are a sensible starting point. To pick between engines, see choosing an inference engine.

Reported speeds

Checked 2026-10-11. Every figure is reported by its author; GPUwerk has not reproduced any. The harnesses, prompts and patches differ, so do not read the vLLM and SGLang rows as an engine comparison. The 41.7, 51.9 and 72.0 figures are medians over the same 40-prompt ai-muninn harness, with thinking disabled in part 45.

Three caveats from the sources. Part 49 reached 72.0 tok/s with several changes at once, the largest being a draft vocabulary extended with Chinese tokens from the author's own documentation, so the gain depends on how close your text is to that corpus; the author reports 23 to 27% on everyday Chinese topics, and the post also lowered the KV memory share from 0.835 to 0.68. The same post reports turn-two time to first token on a 9.5k-token prefix falling from 3.30 s to 0.23 s after aligning prefill chunks to the 848-token cache block. And part 46 reports that the second Spark added 24.5% to decode speed, which the author puts down to fixed per-step overhead, while the KV cache pool grew 2.07 times; vLLM also needed a full copy of the weights on each Spark. For the cabling and setup, see the two-node cluster guide.

GPUwerk has not measured this model

I have no tok/s, memory or quality figure of my own for Qwen3.8-Flash-Next, on the preset or on any NVFP4 build, and the console configuration carries only an estimate. GPUwerk has also not run a two-node inference benchmark. The numbers GPUwerk does publish are for other models on the benchmarks page. Run your own prompts on a rented Spark before relying on any figure here, and see which models fit in 128 GB for the alternatives.

FAQ