Qwen3.8-27B on a DGX Spark

Qwen3.8-27B is a dense 27B vision-language model under Apache 2.0, and it fits one 128 GB DGX Spark at every common precision: 17.6 GB as a 4-bit GGUF, 29 GB at Q8_0 and 54.7 GB in BF16. Because it is dense, every token reads all of its weights, so it is slow on a Spark's 273 GB/s: a user on NVIDIA's developer forum measured 11.4 tok/s on vLLM with the NVFP4 build and 24.7 tok/s with a 5-token MTP draft. GPUwerk has not measured it, and it is not a console preset.

What is Qwen3.8-27B?

Checked 2026-10-18 on the Qwen3.8-27B card and the FP8 card. The card gives no release day; the forum post cited below, dated 15 August 2026 (UTC), says the model was released that day.

Does it fit at BF16, FP8 and 4-bit?

Yes, all three. A Spark has 128 GB of unified memory per NVIDIA's spec page, of which about 121.6 GiB is addressable (vLLM reports this on GPUwerk nodes, and the forum post below lists 121.63 GiB). Weight sizes, from the file listing of Unsloth's GGUF repo and the forum report for NVFP4, checked 2026-10-18:

The leftover column is my arithmetic (GB converted to GiB). The NVFP4 repo and Qwen's FP8 repo state no total on their cards, so those two rows are the sums of the safetensors files in each repo's file listing; the NVFP4 sum matches the 23.4 GB download in the forum post, which is also the source for the NVFP4 layer split. The GGUF files hold the language model only; the vision projector is a separate mmproj file of about 0.93 GB in the same repo.

Memory is not the constraint for this model. Bandwidth is, and the next two sections cover what that means.

How much memory does the context take?

Less than the parameter count suggests. Only the 16 Gated Attention layers keep a cache that grows with the prompt, and each has 4 KV heads of dimension 256. The 48 Gated DeltaNet layers hold a fixed state per sequence. My arithmetic from the card's architecture:

16 layers x 2 (K and V) x 4 heads x 256 x 2 bytes = 65,536 bytes per token in BF16. This ignores the fixed DeltaNet state.

The forum measurement on vLLM is a useful check on that. At FP8 the author saw 37,169 bytes per token, about 12% more than the attention-only figure, because vLLM puts the linear-attention state in the same pool. With --gpu-memory-utilization 0.45 the run left 27.56 GiB for KV cache, enough for 777,645 tokens, or almost three requests at the full 262,144 context. The author sized 1M context at roughly 36.3 GiB of KV and needed a utilization of at least 0.53.

So the full native context fits next to even the BF16 weights. The 1M setting also fits at 4-bit or FP8, but Qwen's card warns that static YaRN may affect quality on shorter texts and recommends turning it on only when a long context is needed.

How fast is it on one Spark?

Decode on a Spark is bound by memory bandwidth, as the benchmarks page explains: tok/s is at most 273 GB/s divided by the bytes read per token. A dense model reads close to all of its weights for each token, which gives these ceilings by my arithmetic: about 5 tok/s for BF16 (54.7 GB), about 9.4 for Q8_0 (29 GB) and about 15.5 for UD-Q4_K_XL (17.6 GB). Those files include the token embedding table, of which only one row is read per token, so the real ceilings sit slightly higher. Multi-token prediction can lift a single stream above the ceiling, because one pass over the weights can produce more than one accepted token.

The one detailed single-Spark report I found is a post by user helge on NVIDIA's developer forum, dated 15 August 2026:

unsloth/Qwen3.8-27B-NVFP4 on vLLM 0.26.1rc1 (nightly, 2026-08-01), single Spark, averaged over six prompts of different kinds (code, prose, SQL, a list, an explanation, a short story). Prefill with a unique prefix per request: 1,734 tok/s at 4,566 prompt tokens, falling to 853 tok/s at 47,857. Checked 2026-10-18. These are one user's numbers, not GPUwerk measurements.

Two of the author's own cautions are worth repeating. MTP acceptance depends on the generated text, so a single prompt gives misleading speeds; the author first saw 36.0 tok/s on one prompt and 28.1 when re-running it. And the gaps between 3 and 8 draft tokens were within about 14%, close to the noise, so 5 is a mild preference.

Which engine should you use on ARM64?

A Spark is an ARM64 machine (NVIDIA lists a 20-core Arm CPU) with a Blackwell GPU, which matters more for the install than for the choice. Qwen's card names SGLang, vLLM and TokenSpeed and recommends their latest versions without naming a minimum. On a Spark, two routes have a documented path:

• vLLM if you want MTP or several users on one endpoint. The forum run above used vLLM without modification in a community nightly container built for the Spark. Our vLLM doc explains why a plain pip install on aarch64 has historically picked up a CPU-only PyTorch.
• llama.cpp for one user and a GGUF file. Build it for sm_121 as our llama.cpp doc describes. For image input, load the repo's mmproj file alongside the model. Unsloth's repo also ships a separate 1.37 GB MTP file, but its card does not say how to pair it with the main model, so I have no tested MTP command for llama.cpp.

A llama.cpp command in the form our llama.cpp doc uses, pointed at Unsloth's repo and bound to localhost (Unsloth's card gives no command of its own, and GPUwerk has not run this one):

cd ~/llama.cpp/build
./bin/llama-server \
  -hf unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M \
  --host 127.0.0.1 \
  --port 30000

The vLLM command from the forum post, with the host changed to localhost. The author launched it with --host 0.0.0.0; on a GPUwerk instance that exposes the port publicly, so add --api-key if you open it up:

vllm serve unsloth/Qwen3.8-27B-NVFP4 \
  --host 127.0.0.1 --port 8000 \
  --tensor-parallel-size 1 \
  --gpu-memory-utilization 0.45 \
  --max-model-len 262144 \
  --max-num-seqs 4 \
  --max-num-batched-tokens 8192 \
  --enable-chunked-prefill \
  --enable-prefix-caching \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_xml \
  --enable-auto-tool-choice \
  --distributed-executor-backend mp \
  --speculative-config '{"method":"mtp","num_speculative_tokens":5}'

The MTP head is inside the checkpoint, so --speculative-config needs no separate draft model. One more thing from that thread: at release, the unsloth NVFP4 repo shipped a tokenizer.json that silently cut every prompt at 2,048 tokens. The author later noted that Unsloth uploaded a fixed version. If a long prompt seems to lose its beginning, check that the truncation field in your cached tokenizer.json is null.

Qwen3.8-27B or Qwen3.6-35B-A3B?

Checked 2026-10-18. The Qwen3.6 figures are from the Qwen3.6-35B-A3B page. The two decode figures come from different reports, builds and prompts, so read them as an order of magnitude, not a ratio.

Take Qwen3.6-35B-A3B when a person is watching tokens appear, or when you want a one-click console preset. It reads roughly a ninth of the bytes per token, and that shows up directly in speed. Take Qwen3.8-27B when it answers your own prompts better than the MoE does and you can live with 11 to 25 tok/s: batch jobs, agent steps where quality per call matters more than latency, or long documents where you want the newer model. We'd measure both on your prompts before deciding, since neither has a GPUwerk number yet. For the larger Qwen3.8 on the preset list, see Qwen3.8 Flash Next.

Try it

A Spark rents for $0.79/hour, billed per minute against a prepaid balance (see the pricing page). Timing a few of your own prompts costs only the minutes the instance runs, and gives you a number for your workload instead of someone else's.

FAQ