Best LLMs for an RTX 5090: what fits in 32 GB and what to run, November 2026

For one RTX 5090 in November 2026, I would run Qwen3.8-27B at Q5_K_M for coding, chat and images (18.4 GiB of weights, about 28 GiB with a 128k context), and Qwen3.6-35B-A3B or Nemotron 3.5 Lightning when speed matters (271 and 377 tok/s reported on llama.cpp). By my calculation from each model's config.json, 11 of 15 current open-weight models fit in the card's 31.8 GiB with a 32k context; gpt-oss-120b, Qwen3.8-Flash-Next, Mistral Small 4 and Ornith-1.5-122B need experts in system RAM.

Decode figures from the witcheer/rtx-5090-benchmarks dataset (llama.cpp tg128 and vLLM 0.25.1, May to August 2026), read 7 November 2026. File sizes from the Hugging Face repos, checked the same day. Full table and method below.

How much VRAM does a model need on a 32 GB card?

Add three things: the weight file, the KV cache for your context, and about 1.5 GiB of runtime overhead, and keep the sum under the 31.8 GiB a 5090 actually reports (32,607 MiB in Atomic Chat's tests). The KV term is where 2026 models differ from older ones:

KV bytes per token = 2 (K and V) × attention layers × KV heads × head_dim × 2 bytes (FP16)

Qwen3.8-27B: 64 layers, but only 16 are full attention (full_attention_interval 4)
  2 × 16 × 4 × 256 × 2 = 65,536 bytes = 64 KiB per token
  32k tokens: 2.0 GiB      128k tokens: 8.0 GiB

QwQ-32B (2025, every layer full attention): 64 layers × 8 KV heads × 128
  2 × 64 × 8 × 128 × 2 = 262,144 bytes = 256 KiB per token
  32k tokens: 8.0 GiB      128k tokens: 32.0 GiB (does not fit)

The layer, head and dimension counts are read from each model's config.json on Hugging Face (Qwen3.8-27B); the arithmetic is mine. Three extra terms apply to some models. Gated DeltaNet and Mamba layers keep a fixed recurrent state per sequence instead of a cache, under 0.2 GiB for every model here (the config stores it in float32). Gemma 4's sliding-window layers cache only the last 1,024 tokens, a fixed 0.2 to 0.8 GiB. Vision models need a projector file when you load images, 1.6 GiB for Nemotron Bolt.

The 1.5 GiB overhead is my planning figure for CUDA context and compute buffers, and it checks out against measurements. Atomic Chat logged Gemma 4 31B Q4_K_M at 22,372 MiB (21.85 GiB) with a 32k context on llama.cpp b10988. The formula gives 17.07 GiB of weights, 3.28 GiB of cache and 1.5 GiB overhead: 21.85 GiB. For Gemma 4 26B-A4B and 12B it overestimates by 0.4 to 0.8 GiB, which is the safe side. A desktop session takes its share first: ai.rs saw 5.6 GB in use on an RTX 5090 before loading any model.

Which open models fit in 32 GB?

Eleven of the 15 models fit with a 32k context, and all eleven still fit at 128k, though Gemma 4 31B and the 2-bit Qwen3-Coder-Next leave under 2.5 GiB spare; the four above 46 GiB need their experts in system RAM. Totals include the 1.5 GiB overhead; a verdict of "fits" means under 31.8 GiB.

My calculation, FP16 KV cache, one sequence, 1.5 GiB overhead. Architecture fields from each model's config.json; file sizes from Unsloth, NVIDIA and ornith-ai GGUF repos, checked 7 November 2026. Gemma 4 counts K and V separately on its full-attention layers; its config sets attention_k_eq_v, which Transformers documents as the key projection being reused as the value, so an engine that stores it once needs less. Reported tok/s: witcheer dataset (llama.cpp tg128; offload rows with 64 GB DDR5-5600) except Gemma 4 31B and 26B-A4B (Atomic Chat, llama.cpp b10988). The full speed table is in the RTX 5090 benchmarks.

Two cross-checks against measurements hold up. Hardware Corner's context sweep lists Qwen3 32B at a maximum of 32k on a 5090, Gemma 4 31B at 128k and a Qwen3.5 35B MXFP4 build at 256k, which is what the per-token figures above predict. Ollama is the outlier: kohei_wada measured Qwen3.8-27B at 29.2 GB with the full 262,144-token context, less than my FP16 formula gives, so Ollama is likely storing that cache more compactly; I could not confirm how. A q8_0 cache in llama.cpp (-ctk q8_0 -ctv q8_0) roughly halves every KV figure in the table. The KV cache size article covers the quality cost.

What is the best LLM for coding on an RTX 5090?

Qwen3.8-27B is my pick for coding on one 5090, with Ornith-1.5-35B-A3B as the faster alternative to test against it. Qwen's card reports 73.0 on Terminal-Bench 2.1 (Terminus), 61.7 on SWE-bench Pro and 90.3 on LiveCodeBench v6, all self-reported, under Apache 2.0. At UD-Q5_K_M it leaves room for a 128k context. The catch is speed: about 79 tok/s at 4-bit on llama.cpp. vLLM 0.25.1 with the model's MTP head and NVFP4 weights reached 119.8 tok/s in the witcheer runs, which is the setup I would use for an agent loop.

Ornith-1.5-35B-A3B is a 3B-active MoE under MIT that its card says extends Ornith-1.0, "developed on top of Qwen3.5 and Gemma4". It decodes at 303 tok/s, four times Qwen3.8-27B. Its card claims 79 on SWE-bench Verified, 59.6 on SWE-bench Pro and 67.8 on Terminal-Bench 2.1. NVIDIA's own measurement of the comparable Qwen3.6-35B-A3B on Terminal-Bench 2.1 is 44.38, so a 3B-active model at 67.8 is a large claim from a lab I know little about. Run your own repository tasks on both before choosing. If you need Qwen3-Coder-Next, its 2-bit file (24.92 GiB) fits at 224.9 tok/s, but the 4-bit build at 45.2 GiB does not. Agent setup is in coding agents against your own endpoint.

Which model is best for agents and tool calling?

For agents I would still start with Qwen3.8-27B, and test Nemotron 3.5 Bolt 74B-A9B-24GB next, because it is a 70B-class model NVIDIA built to fit a 24 GB card. NVIDIA's card dates the Hugging Face release to 15 October 2026 and describes a 74B-total, 9B-active Mamba-Transformer hybrid with quantization-aware 2-bit experts, packed into a 22.02 GB GGUF under the OpenMDW-1.1 license. Its card compares it with Qwen3.6-35B-A3B at BF16 under NVIDIA's own runs.

From the Nemotron 3.5 Bolt model card, read 7 November 2026. Vendor-run comparison. The card notes Qwen3.6's own reported MCP-Atlas score is 62.8.

The friction is software. The card's only listed runtime is a specific vLLM nightly image plus a "b12x" backend, with SM120 (RTX 50) and SM121 (GB10) named as supported, and no 5090 speed is published anywhere I could find. Treat it as a model to benchmark, not one to deploy blind. Tool-call parsing is usually what breaks first with a new model; the function-calling guide lists the checks.

Which model is best for vision and for long context?

For images and video, Qwen3.8-27B again, since the card lists native image and video input; Gemma 4 31B and 12B are the alternatives with a vision tower. For long documents, Nemotron 3.5 Lightning: 6 KiB of KV cache per token means its full 256k context costs 1.5 GiB, and the witcheer runs show it keeping 94% of its decode speed at 32k. NVIDIA's card says Lightning supports up to 1M tokens and was validated to 256K on one H100. Qwen3.6-35B-A3B is the second choice, at 20 KiB per token and 27.2 GiB total at 262k. Older full-attention models cannot do this on 32 GB at all. More on running vision models is in the multimodal guide.

What about models bigger than 32 GB?

Mixture-of-experts models above 32 GB run at 35 to 80 tok/s on a 5090 if you keep the shared layers and some experts on the card and the rest in system RAM, but plan on 64 to 128 GB of fast DDR5. llama.cpp's --n-cpu-moe N keeps the expert weights of the first N layers on the CPU. With 64 GB of DDR5-5600, witcheer measured gpt-oss-120b at 46.5 tok/s, Qwen3.8-Flash-Next at 2-bit with --n-cpu-moe 22 at 59.7 tok/s and Ornith-1.5-122B with --n-cpu-moe 24 at 47.6 tok/s. Prefill suffers more than decode: 588 to 896 tok/s against several thousand when resident.

Engines that cache hot experts on the GPU do better. Seppe Gadeyne reports Qwen3.8-Flash-Next at UD-Q5_K_XL running at 78.8 tok/s and Ornith-1.5-122B at 46.4 tok/s on one RTX 5090 with 128 GB of DDR5-5600, using the Strata and TabbyAPI engines, measured 21 to 22 October 2026. Qwen describes Flash-Next as 125B with 6B active plus a 51B n-gram embedding, under a qwen-community-1.0 license whose terms the card does not state. The bandwidth math behind offload is in the sibling article on llama.cpp MoE offload. Dense models do not offload well; for a 70B dense model, more memory on one device is the answer, which is where the 128 GB memory calculator and the DGX Spark vs RTX 5090 comparison come in.

FAQ

Sources

Every figure above is attributed to its source in the sentence or table where it appears. Pages were read on 7 November 2026 unless a date says otherwise.

• Qwen3.8-27B model card and config.json, Qwen on Hugging Face, 2026-08
• Qwen3.6-35B-A3B config.json, Qwen on Hugging Face, 2026-04-15
• gemma-4-31B-it config.json, Google on Hugging Face, 2026-03-11
• Gemma 4 configuration (attention_k_eq_v), Hugging Face Transformers on GitHub
• NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 model card, NVIDIA on Hugging Face, 2026-08-11
• NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB model card, NVIDIA on Hugging Face, 2026-10-15
• Ornith-1.5-35B-A3B model card, ornith-ai on Hugging Face, 2026-08-18
• Qwen3.8-Flash-Next model card, Qwen on Hugging Face, 2026-08
• gpt-oss-20b model card, OpenAI on Hugging Face, 2025-08-04
• gpt-oss-120b model card, OpenAI on Hugging Face, 2025-08-04
• Qwen3.8-27B-GGUF and other Unsloth GGUF repos (file sizes), Unsloth on Hugging Face
• witcheer/rtx-5090-benchmarks, Hugging Face, 2026-10-26
• Best Local LLMs for RTX 5090: Benchmarks, Atomic Chat, 2026-09-18
• RTX 5090 LLM benchmarks (context sweep), Hardware Corner, 2026-03
• Four big local LLMs on one RTX 5090 and 128 GB of RAM, benchmarked side by side, DEV Community (Seppe Gadeyne), 2026-10-21
• Run Qwen3-Coder 30B-A3B on an RTX 5090: 231 tok/s with Ollama, ai.rs, 2026-05-22
• Qwen3.8 27B Measured on an RTX 5090, and Why vLLM Didn't Work, DEV Community (kohei_wada), 2026-11-03