Best LLMs for 16 GB of VRAM in November 2026: what fits, how fast, and when to stop

On a 16 GB card in November 2026, gpt-oss-20b is the default pick: its 12.1 GB MXFP4 file plus a full 131,072-token KV cache comes to about 14.3 GiB, and llama.cpp generates it at 111.5 tok/s on an RTX 5060 Ti 16GB, 189.5 on a 5070 Ti and 204.9 on a 5080. Gemma 4 12B and Qwen3.5-9B are the picks for images and very long context, Gemma 4 26B-A4B and Qwen3.6-35B-A3B fit at 3-bit, and a dense 27B fits only at 3-bit with about 32k of context. Speeds are from the llama.cpp gpt-oss guide (builds 6208 and 6210); file sizes and configs were read on Hugging Face on 7 November 2026.

tok/s, single stream, llama.cpp. tg128 column: llama.cpp gpt-oss guide, builds 6208 to 6316, empty context, August 2025 onward. 4k columns: Hardware Corner's pages for the 5060 Ti 16GB, 5070 Ti, 5080, 4060 Ti 16GB and 4080 (March 2026 where dated; the 5080 page names build 8233). Bandwidth and prices: gpuprix pages for the 5060 Ti 16GB, 5070 Ti, 5080, 4060 Ti 16GB and 4080, last updated 4 November 2026, US dollars as listed by the retailer. The two speed columns differ because the 4k column measures with 4,096 tokens already in context.

How much fits in 16 GB?

About 15 GiB of weights plus cache, once the driver, display and compute buffers take their share. llama.cpp's own automatic fitting (--fit, on by default) leaves a 1,024 MiB margin per device, per the llama-server README, and I use the same margin here. The cache grows with context, so the useful formula is:

KV bytes per token = 2 (K and V) x full-attention layers x KV heads x head_dim x 2 bytes (FP16)
total = weight file + KV bytes per token x context + fixed sliding-window cache

Hybrid models make this kinder than the parameter count suggests. Qwen3.5, Qwen3.6 and Qwen3.8 keep a per-token cache only in every fourth layer (the rest are Gated DeltaNet layers with a fixed state), Gemma 4 caps most of its layers at a 1,024-token sliding window, and gpt-oss alternates full attention with a 128-token window. The sibling article on KV cache size by model goes further into the variants.

My arithmetic from the config.json files of gpt-oss-20b, Qwen3.5-9B, Gemma 4 12B, Gemma 4 26B-A4B, Qwen3.6-35B-A3B and Qwen3.8-27B, and file sizes from the Hugging Face listings of ggml-org and Unsloth (Qwen3.5-9B, Gemma 4 12B, Gemma 4 26B-A4B, Qwen3.6-35B-A3B, Qwen3.8-27B), all read 7 November 2026. 128k means 131,072 tokens. Vision adds a projector file: 175 MB for Gemma 4 12B, 918 MB for Qwen3.5-9B. Gemma 4 26B-A4B's config sets attention_k_eq_v, so its global cache may be half the figure shown.

A q8_0 KV cache halves every KV figure above and is the cheapest way to buy context on a 16 GB card. With it, Qwen3.8-27B at 3-bit reaches about 64k tokens in the same 14.2 GiB.

How fast are these models on each 16 GB card?

For mixture-of-experts models, fast on every card except the 4060 Ti 16GB; for dense models, bandwidth decides, and the gap between cards is close to the gap in GB/s. On gpt-oss-20b the 5070 Ti generates 189.5 tok/s against 111.5 for the 5060 Ti, a 1.7x gap for a 2x bandwidth gap, and the 5080's extra 64 GB/s buys 8% more.

No source I found has run the 2026 models on all five cards, so for the dense cases I work from the ceiling: a dense model reads its whole weight file per token, and tok/s cannot exceed bandwidth divided by file size. Qwen3 14B at Q4_K on the 5060 Ti makes the point: Hardware Corner reports 41.1 tok/s, and the 4060 Ti 16GB, with 288 GB/s, gets 27.4.

By my calculation: bandwidth (gpuprix) divided by file size, tok/s, rounded. These are upper bounds, not measurements. Reported dense runs land at roughly 64 to 81% of the ceiling: Qwen3 14B at Q4 is 8.5 GB per Naveen Kulandaivelu's model table, so the 5060 Ti's reported 41.1 tok/s is 78% of its 52.7 tok/s ceiling and the 4060 Ti's 27.4 is 81% of 33.9; dense 32B runs on 24 and 32 GB cards land at 64 to 70%.

So a dense 27B at 3-bit is usable on a 5070 Ti or 5080 (roughly 45 to 49 tok/s if it lands at two thirds of the ceiling) and slow on a 4060 Ti 16GB (about 15). Mixture-of-experts models read only their active experts per token, which is why gpt-oss-20b (12.1 GB on disk) is measured at 111.5 tok/s on a 5060 Ti while Gemma 4 12B Q8_0 (12.7 GB) cannot exceed 35 on the same card. The Gemma 4 26B-A4B notes show the same dense-against-sparse gap on a DGX Spark: 52 tok/s for the MoE against 6.9 for the 31B dense, both reported.

Can you run bigger MoE models with CPU offload?

Yes. llama.cpp's --n-cpu-moe N keeps the expert weights of the first N layers in system RAM (added in PR #15077, merged 4 August 2025), so a 4-bit model larger than the card still runs, at the speed of your RAM for that share of the work. The full treatment, with measured numbers, is in the sibling article on MoE CPU offload; here is the 16 GB case worked through.

Qwen3.6-35B-A3B at UD-Q4_K_XL is 22.4 GB (20.8 GiB) across 40 layers, so roughly 0.52 GiB per layer, most of it routed experts. To keep about 12.5 GiB of weights on the card, leaving room for a 32k cache and compute buffers, about 8.3 GiB must move: 16 layers' experts at the very least, more because attention weights stay on the card. Start at 16 and raise N until it loads (or let the default --fit on choose):

llama-server -m Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf \
  -ngl 99 --n-cpu-moe 16 -c 32768 -fa on \
  --host 127.0.0.1 --port 8080

Speed estimate, my arithmetic, upper bound: the file averages about 5.1 bits per weight (22.36 GB for 35B parameters), so the roughly 3B active parameters cost about 1.9 GB per token. If 16 of 40 layers' experts sit in RAM, up to 0.77 GB of that comes from system memory. Dual-channel DDR5-6000 peaks at 96 GB/s (6,000 MT/s x 8 bytes x 2 channels), so the RAM share costs at least 8 ms per token; add 2.6 ms for the card's share on a 5060 Ti or 1.2 ms on a 5080, and generation is capped near 95 to 110 tok/s before any other overhead. Faster RAM matters more here than the gap between a 5070 Ti and a 5080.

gpt-oss-120b is the stretch case. The llama.cpp gpt-oss guide gives --n-cpu-moe 32 as the starting point on a 16 GB card without a speed figure; most of its 63.4 GB then lives in RAM, so expect a fraction of the roughly 30 tok/s the same guide reports for a 32 GB RTX 5090 with --n-cpu-moe 21. A 128 GB machine runs it whole; GPUwerk's gpt-oss-120b page collects reported speeds on a DGX Spark.

Which model should you pick for each job?

Pick by what the job needs to fit, then by speed. This list ranks by fit and measured speed, not by benchmark scores.

• General chat and reasoning: gpt-oss-20b. Full context fits, it is the fastest model here on every card, and MXFP4 runs natively in llama.cpp (the quantization notes explain the format).
• Images, screenshots, scanned pages: Gemma 4 12B at Q8_0 or Q6_K (9.79 GB) with its 175 MB projector, or Qwen3.5-9B, whose config includes a vision encoder (918 MB projector).
• Very long documents: Qwen3.5-9B at Q4_K_M. Its 32 KiB per token means the full 262,144-token context fits in 13.3 GiB.
• Coding and agent loops: Qwen3.6-35B-A3B, at UD-IQ3_XXS on the card or at 4-bit with --n-cpu-moe. Agents resend long contexts, so the 5070 Ti and 5080, with 2.2x and 3.4x the 5060 Ti's prompt processing (3,904 and 5,907 against 1,738 tok/s at 32k on gpt-oss-20b, Hardware Corner), feel very different here.
• A dense model for a second opinion: Qwen3.8-27B at UD-Q3_K_XL with 32k context and a q8_0 cache, on a 5070 Ti, 5080 or 4080 only.

If I had one 16 GB card, I would run gpt-oss-20b by default and keep Gemma 4 12B loaded for images. If I were buying one this week, it would be the 5070 Ti: twice the 5060 Ti's bandwidth, 92% of the 5080's gpt-oss-20b speed, and $1,378 against $2,000 for the cheapest new 5080 on gpuprix on 4 November 2026. In Germany the same tracker had 679, 1,329 and 1,660 euros for the cheapest new 5060 Ti 16GB, 5070 Ti and 5080. Avoid the 4060 Ti 16GB for LLMs: at 288 GB/s it is the slowest card here, and gpuprix's cheapest listing ($843) was out of stock and above a new 5060 Ti.

When is 16 GB not enough?

When the model you need is dense and above 14B at 4-bit, when you need more than one long conversation in memory, or when the model is 70B-class or larger. Concretely:

• A dense 27-32B model at 4-bit: Qwen3.8-27B's UD-Q4_K_XL file alone is 17.6 GB. That needs 24 GB, the case covered in the sibling piece on a used RTX 3090, or 32 GB.
• Several users: each concurrent sequence carries its own cache. Four 32k conversations on Qwen3.8-27B need 8 GiB of cache at FP16, more than the room left beside its weights.
• gpt-oss-120b, 100B-class MoE models, and 70B dense models: these need 64 to 128 GB to run without most of the work landing in system RAM.

The step up is a 32 GB card or a 128 GB unified-memory machine; the RTX 5090 against DGX Spark comparison lays out that trade, and the general method in choosing a GPU for open-source LLMs applies. GPUwerk rents a dedicated RTX 5090 at $0.89 an hour, which is a cheap way to see whether the 4-bit build you could not load is worth the upgrade.

FAQ

Sources

Every figure above is attributed to its source in the sentence or table where it appears. Pages were read on 7 November 2026 unless a date says otherwise.

• guide: running gpt-oss with llama.cpp (discussion #15396), ggml-org/llama.cpp on GitHub, 2025-08-18
• llama-server README, ggml-org/llama.cpp on GitHub
• llama : add --n-cpu-moe option (PR #15077), ggml-org/llama.cpp on GitHub, 2025-08-04
• RTX 5060 Ti 16GB local LLM benchmarks, Hardware Corner, 2026-03
• RTX 5070 Ti local LLM benchmarks, Hardware Corner, 2026-03
• RTX 5080 local LLM benchmarks, Hardware Corner
• RTX 4060 Ti 16GB local LLM benchmarks, Hardware Corner, 2026-03
• RTX 4080 local LLM benchmarks, Hardware Corner
• GeForce RTX 5060 Ti 16GB price in United States, gpuprix, 2026-11-04
• GeForce RTX 5070 Ti price in United States, gpuprix, 2026-11-04
• GeForce RTX 5080 price in United States, gpuprix, 2026-11-04
• GeForce RTX 4060 Ti 16GB price in United States, gpuprix, 2026-11-04
• GeForce RTX 4080 price in United States, gpuprix, 2026-11-04
• GeForce RTX 5060 Ti 16GB price in Germany, gpuprix, 2026-11-04
• GeForce RTX 5070 Ti price in Germany, gpuprix, 2026-11-04
• GeForce RTX 5080 price in Germany, gpuprix, 2026-11-04
• 96GB VRAM on a single GPU: benchmarking NVIDIA's RTX PRO 6000, Naveen Kulandaivelu, 2025-08-07
• gpt-oss-20b config.json, OpenAI on Hugging Face
• Qwen3.5-9B config.json, Qwen on Hugging Face
• gemma-4-12B-it config.json, Google on Hugging Face
• gemma-4-26B-A4B-it config.json, Google on Hugging Face
• Qwen3.6-35B-A3B config.json, Qwen on Hugging Face
• Qwen3.8-27B config.json, Qwen on Hugging Face
• gpt-oss-20b-GGUF files, ggml-org on Hugging Face
• Qwen3.5-9B-GGUF files, Unsloth on Hugging Face
• gemma-4-12B-it-GGUF files, Unsloth on Hugging Face
• gemma-4-26B-A4B-it-GGUF files, Unsloth on Hugging Face
• Qwen3.6-35B-A3B-GGUF files, Unsloth on Hugging Face
• Qwen3.8-27B-GGUF files, Unsloth on Hugging Face
• gpt-oss-120b-GGUF files, ggml-org on Hugging Face