Qwen3.6-35B-A3B on a DGX Spark

Qwen3.6-35B-A3B is a mixture-of-experts model with 35B parameters, about 3B of them active per token, and it fits one 128 GB Spark with room to spare: the 4-bit GGUF is 22.9 GB and the Q8_0 file the GPUwerk preset serves is 36.9 GB. Reported single-stream decode on one Spark is 97.7 tok/s on vLLM 0.28.0 with a 3-token MTP draft and 81 tok/s for an NVFP4 build. GPUwerk has not measured this model, and none of its Qwen3-Coder numbers carry over.

Spec

Checked 2026-10-11 on the Qwen3.6-35B-A3B card and the FP8 card. The card gives the month of release, not a day.

Memory fit

A Spark has about 121.6 GiB addressable. Weight sizes per quantization, from the files in Unsloth's MTP GGUF repo and plain GGUF repo, and the Jetson AI Lab model page, checked 2026-10-11:

The leftover column is my arithmetic (GB converted to GiB). The Jetson AI Lab figure is listed as "Parameters 24GB" for an NVFP4/AWQ 4-bit build on Jetson devices; the page does not say whether that is the download or the memory needed, and it does not mention the Spark. The FP8 card does not state a download size.

KV cache is small for a 35B model. Only the Gated Attention layers keep a per-token cache, and the card lists 10 of them, each with 2 KV heads of dimension 256. That is 10 x 2 x 2 x 256 x 2 bytes, about 20 KiB per token in BF16, so the full 262,144-token context needs about 5 GiB. This is my arithmetic from the card's architecture and ignores the fixed-size DeltaNet state. A Q8_0 KV cache, as the preset uses, halves it. Context length is not what limits this model on a Spark; speed is.

Decode on a Spark is bound by memory bandwidth, as the benchmarks page explains: tok/s is at most 273 GB/s divided by the bytes read per token. With about 3B active parameters, Q8_0 (8.5 bits per weight) reads about 3.2 GB per token, a ceiling near 85 tok/s; at 4-bit it is about 1.7 GB and near 160 tok/s, the same arithmetic the benchmarks page applies to Qwen3-Coder. GPUwerk's two measured models landed at 50 to 80% of their ceilings. Speculative decoding (MTP) can lift a single stream above that ceiling because each pass over the weights can emit more than one accepted token. All of this is my arithmetic, not a measurement.

Run it on GPUwerk: the console preset

The console lists Qwen3.6 35B-A3B as a llama.cpp preset. Its configuration in GPUwerk's control plane, checked 2026-10-11:

• Weights: Qwen3.6-35B-A3B-Q8_0.gguf (36.9 GB) from Unsloth's plain GGUF repo, served as qwen3.6-35b-a3b.
• Flags: -ngl 99 -c 65536 -np 4 -ctk q8_0 -ctv q8_0 -fa on --jinja --cache-ram 0 --no-mmap. That is 65,536 tokens of context shared by 4 slots, so about 16,384 per slot, with a Q8_0 KV cache.
• No MTP draft and no --mmproj: the preset serves the plain Q8_0 file as a text model, with no speculative decoding and, as I read the flags, no image input.

The preset gives up speed for a safe default. Q8_0 reads about twice the bytes of a 4-bit build, and the context is the 4-slot split, not the model's 262,144 tokens. Four slots also rule out MTP in llama.cpp today, which Unsloth's notes say works only with one slot. For the full context, image input or an MTP draft, run it yourself (next section). A deploy of this preset works only on a node that has the weights staged, as the models index says, so check the console for availability. GPUwerk has not published a measured tok/s for this preset.

Run it yourself

These commands come from GPUwerk's llama.cpp doc and Unsloth's model card and assume llama.cpp is built for sm_121 as that doc describes. GPUwerk has not benchmarked either variant on this model.

cd ~/llama.cpp/build
./bin/llama-server \
  -hf unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL \
  --host 127.0.0.1 \
  --port 30000

To use the MTP head for speculative decoding, add the MTP flags. This works only with a checkpoint that ships MTP weights, which is why the command uses the MTP repo. Unsloth's MTP GGUF notes (checked 2026-10-11) add two limits: MTP does not yet work with more than one slot (-np above 1) or with --mmproj, so this is a single-user, text-only server, and their example drafts 2 tokens where the llama.cpp doc uses 3:

./bin/llama-server \
  -hf unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL \
  --host 127.0.0.1 \
  --port 30000 \
  -ngl 99 -fa on -np 1 \
  --spec-type draft-mtp \
  --spec-draft-n-max 2

These MTP flags match Unsloth's model card for this GGUF, checked 11 October 2026: one slot (-np 1), because MTP does not yet work with more than one slot or with --mmproj, and a 2-token draft. GPUwerk has not run this command on a Spark.

I bound both to 127.0.0.1; the doc's 0.0.0.0 exposes the port publicly on a GPUwerk instance, so add --api-key if you open it up.

For vLLM, the vLLM recipe has a GB10 profile: its measured run used vLLM 0.28.0, a 3-token MTP draft and GPU memory utilization capped at 0.5, and it asks for vLLM 0.28.0 or later for NVFP4 (older versions load the checkpoint but do not select the faster FlashInfer decode kernel). I have not run it, so there is no tested command here. Do not copy the card's own vllm serve example, which uses --tensor-parallel-size 8. The vLLM doc covers a single-Spark setup.

Reported speeds

Checked 2026-10-11. Both are reported figures, not GPUwerk measurements, and they do not share a prompt, a stated quant or an engine, so do not subtract one from the other or read the gap as the MTP gain. noze.it itself warns that maximum configurable context and short-prompt speed are not measured together.

The sibling model GPUwerk did measure is Qwen3-Coder 30B-A3B: 80.9 tok/s single-user on vLLM 0.27.1 with AWQ, and 3,235 tok/s aggregate at 256 concurrent requests, on 2 September 2026. That is a different model with a different architecture. The shared "A3B" tells you the active-parameter count, not the speed, so do not read those figures as Qwen3.6 numbers.

On engines: llama.cpp is the simple route for one user and for GGUF files, and it is what the preset uses. vLLM is the one to test if several people will share the endpoint, since it batches concurrent requests, while Ollama by default serves one request per model at a time; the engine comparison and the benchmarks page have the reported numbers for the Qwen3-30B-A3B sibling.

When to pick it over Qwen3-Coder 30B or gpt-oss-120b

Checked 2026-10-11. The gpt-oss-120b figure is user eugr's llama.cpp result quoted on the benchmarks page; see the gpt-oss-120b page. The Qwen3-Coder figure is from its page and raw log. The columns use different engines, quants and workloads.

Pick Qwen3.6-35B-A3B when you want a general model with thinking (on by default, per the card), a long context and most of the Spark's memory left free. For image or video input, run it yourself with the projector; the preset is text only. Pick Qwen3-Coder 30B when you want numbers you can check against a public log, or when many users share one endpoint; it is the model where GPUwerk has a concurrency curve. Pick gpt-oss-120b when you need a larger model and accept lower single-stream speed. For the larger Qwen on the same preset list, see Qwen3.8 Flash Next.

Try it

A Spark rents for $0.79/hour, billed per minute against a prepaid balance (see the pricing page). Measuring your own prompts costs a few minutes of that, which is a better basis than any figure above.

FAQ