Dual RTX 5090 for LLMs: tensor parallel over PCIe, reported tok/s and cost, November 2026

Two RTX 5090s give you 64 GB of VRAM, not twice the single-user speed: with no NVLink, vLLM tensor parallel lifted one QwQ-32B stream from 65.2 to 79.5 tok/s (1.22x) in DeutscheKI's runs, and a 70B model split across both cards in Ollama decoded at 26.9 tok/s. The pair costs $13,099.98 at 7 November 2026 US prices against $16,399 for one 96 GB RTX PRO 6000, and is rated at 1,150 W against 600 W. Buy the second card for memory and for aggregate throughput.

Does a second RTX 5090 double LLM speed?

For one user it adds 0 to 22% depending on the split, and for many users it can nearly double throughput. The reported dual-card numbers I could find:

DeutscheKI: median output tok/s at 0.1 requests per second, vLLM version not pinned, 4,096-token max length. DatabaseMart reports Ollama 0.6.5 and lists two different CPUs for the same server. CloudRift used vllm/vllm-openai:latest at the time. The adrienbrault repos run on driver 610.57.04 and CUDA 13.3 with stock 600 W and 575 W limits.

The layer split result follows from arithmetic. With a layer split, card one reads its half of the weights, then card two reads the other half, so the decode ceiling equals one card with 64 GB: 1,792 GB/s ÷ 43 GB = 41.7 tok/s. DatabaseMart's 26.9 tok/s is 64% of that, and each GPU sitting about 45% busy is what you would expect when they take turns. CloudRift also reports that two 5090s running separate copies of Qwen3-Coder-30B gave "almost 2x" one card. When a model fits on one card, two copies behind a load balancer beat any split.

The ExLlamaV3 row is the outlier, and it is not a tensor-parallel win: it combines a 6B-active MoE model, its MTP head and n-gram drafting on a layer split. It shows what 64 GB buys for a large sparse model, not what a second card does for a dense one.

Tensor parallel or pipeline parallel over PCIe?

Tensor parallel splits every layer across both cards and cuts single-user latency, but it pays a synchronization cost on every layer; pipeline parallel and layer splits avoid that cost and give no single-stream gain. vLLM's parallelism guide says that if GPUs "do not have NVLINK interconnect", you should use pipeline parallelism for throughput, and NVIDIA's RTX 5090 specifications list no NVLink connector.

The DeutscheKI numbers let you size the tensor-parallel penalty. Perfect scaling would halve the 15.33 ms per token to 7.67 ms; the measured TP=2 figure is 12.58 ms, so about 4.9 ms per token goes to something other than reading weights. QwQ-32B has 64 layers (its config.json), and tensor parallel does two all-reduces per layer, 128 per token. If you assign all of the extra time to them, each costs about 38 microseconds, by my arithmetic. Each all-reduce carries about 10 KiB per token at hidden size 5,120 in 16-bit, a size where latency and synchronization set the cost, so PCIe 5.0's bandwidth does not save you. Llama 3.3 70B has 80 layers, so 160 all-reduces per token.

Pipeline parallel avoids the per-layer synchronization: each card holds whole layers and, under load, works on a different batch than its neighbour, which is why vLLM recommends it without NVLink. CloudRift used pipeline parallel for its 70B result and reports PCIe Gen 4 limiting the 2x4090 run to 467 tok/s against 1,230 on the Gen 5 5090 pair.

Do two RTX 5090s support peer-to-peer transfers?

Not on a stock setup, as far as the evidence goes: GeForce P2P depends on a community-patched kernel module, and without P2P vLLM falls back to NCCL for every all-reduce. tinygrad maintains a fork of NVIDIA's open GPU kernel modules; its 570.148.08-p2p branch README says it "allows using P2P on 4090/5090 GPUs with the 570.148.08 driver version". The newest -p2p branch on the first page of the repo's branch list is that 570.148.08 build from June 2025, far older than the 610-series driver the adrienbrault rigs run.

vLLM checks this itself. Its custom all-reduce code runs a P2P test and, if it fails, logs that custom allreduce is disabled because the platform lacks GPU P2P capability; the same code supports exactly two PCIe-only GPUs and disables itself for three or more. If NCCL hangs on a PCIe-only box, vLLM's troubleshooting page suggests NCCL_P2P_DISABLE=1 as a temporary workaround. Check the topology first:

# how the two cards connect (no NVLink rows on GeForce)
nvidia-smi topo -m

# vLLM 0.33: tensor parallel across both cards
vllm serve Qwen/Qwen3.8-27B-FP8 --tensor-parallel-size 2 --max-model-len 131072

# vLLM: pipeline parallel, the docs' choice without NVLink
vllm serve Qwen/Qwen3.8-27B-FP8 --pipeline-parallel-size 2

# llama.cpp: layer split (default), row split, or experimental tensor split
llama-server -m model.gguf -ngl 99 -sm layer -ts 1,1
llama-server -m model.gguf -ngl 99 -sm row
llama-server -m model.gguf -ngl 99 -sm tensor

The llama.cpp server README describes layer as "split layers and KV across GPUs (pipelined)", row as splitting weights by rows, and tensor as splitting weights and KV, marked experimental. I have not found published dual-5090 numbers for row or tensor mode, so measure before you commit. More flags are in the vLLM and llama.cpp notes.

What does 64 GB fit that 32 GB does not?

The 70B dense class at 4-bit, 80B-class MoE models at 4-bit, and full-length contexts at 8-bit; not the 100B-plus class at 4-bit. Usable memory is about 63.7 GiB (two cards at 31.84 GiB, the readout in Atomic Chat's tests), less 1.5 GiB of overhead per card.

My calculation, FP16 KV cache, one sequence, 3 GiB total overhead. KV per token from config.json: Llama 3.3 70B has 80 layers, 8 KV heads and head dimension 128 (2 x 80 x 8 x 128 x 2 bytes = 327,680 bytes). The 37 GiB Llama weight figure is the AWQ size in GPUwerk's memory calculator; GGUF sizes from the Unsloth and ornith-ai repos, checked 7 November 2026. Splits are never perfectly even, so leave more margin than the spare column shows.

The memory changes which model you run more than how fast it runs. A dense 70B at 4-bit tops out near 42 tok/s on a layer split, as computed above, while a 3B-active MoE at 4-bit (Qwen3-Coder-Next) reads far less per token. For per-model fit on a single card, see the best LLMs for an RTX 5090.

What power supply, board and case do two RTX 5090s need?

By my sizing, plan for about 1,600 W of PSU, two native 12V-2x6 cables, a board that gives each card at least PCIe 5.0 x8, and room for two 2-slot or thicker coolers. NVIDIA rates one RTX 5090 at 575 W and asks for a 1,000 W system for a single card; two cards are 1,150 W before the CPU. The adrienbrault rigs run two cards at PCIe Gen5 x8/x8 on an ASRock X870 Taichi Creator with a Ryzen 7 9800X3D, which shows a desktop AM5 board can carry the pair.

The wall socket matters too, by my arithmetic. A European 230 V, 16 A circuit carries 3,680 W, so a dual-5090 box fits. A North American 120 V, 15 A circuit carries 1,800 W, and 80% of that for continuous load is 1,440 W, which a dual-5090 workstation at full power can exceed. Capping each card is the cheap fix: the sibling article on RTX 5090 power limits shows decode losing about 1.4% at 400 W, which brings the pair to 800 W. The Founders Edition is a 2-slot card (304 x 137 mm per NVIDIA); many partner cards are thicker, so check slot spacing and airflow before buying the board.

Two RTX 5090s or one RTX PRO 6000?

Two 5090s are about 20% cheaper and, in CloudRift's run, about 19% faster in aggregate on a 70B model; one PRO 6000 has 50% more memory, no split overhead and half the rated power.

Prices: NowInStock, cheapest in-stock listings among the retailers it tracks on 7 November 2026, both at Newegg (RTX 5090 tracker, Gigabyte Gaming OC at $6,549.99; RTX PRO 6000 tracker, OEM Workstation Edition at $16,399.00), US dollars before sales tax. Memory and power: NVIDIA's RTX PRO 6000 page (96 GB, 1,792 GB/s, 600 W) and RTX 5090 page. Throughput: CloudRift, 9 October 2025.

The PRO 6000 has the same 1,792 GB/s as one 5090, so a single user on a model that fits both sees similar speed from one PRO 6000 and from two 5090s on a layer split, and up to about 1.2x more from the 5090 pair with tensor parallel. Renting is the third option: Hetzner's GEX131 lists a PRO 6000 Max-Q server at EUR 1,499 a month excluding VAT plus the same again as a setup fee. The Max-Q and Workstation variants are compared in the sibling article on RTX PRO 6000 editions, and the 96 GB card against a 128 GB GB10 machine in DGX Spark vs RTX PRO 6000.

FAQ

Sources

Every figure above is attributed to its source in the sentence or table where it appears. Pages were read on 7 November 2026 unless a date says otherwise.

• llm-performance-tests, DeutscheKI on GitHub
• Benchmarking RTX GPUs for LLM inference, CloudRift, 2025-10-09
• 2xRTX 5090 Ollama Benchmark: Outperforming H100 for 70B LLMs, DatabaseMart, 2026-10-29
• qwen3.8-flash-next-2x-rtx5090, adrienbrault on GitHub, 2026-10-23
• glm5.3-flash-2x-rtx5090, adrienbrault on GitHub, 2026-10-10
• Parallelism and scaling, vLLM documentation, 2026-10-15
• custom_all_reduce.py, vLLM on GitHub
• Troubleshooting, vLLM on GitHub
• vLLM releases (v0.33.0, 2 November 2026), vLLM on GitHub, 2026-11-02
• open-gpu-kernel-modules, branch 570.148.08-p2p, tinygrad on GitHub, 2025-06-06
• llama.cpp server README (split-mode options), ggml-org/llama.cpp on GitHub
• GeForce RTX 5090 graphics card specifications, NVIDIA
• NVIDIA RTX PRO 6000 Blackwell Workstation Edition, NVIDIA
• NVIDIA RTX 5090 in stock tracker, NowInStock, 2026-11-07
• NVIDIA RTX PRO 6000 in stock tracker, NowInStock, 2026-11-07
• GEX131 dedicated GPU server, Hetzner
• Llama-3.3-70B-Instruct config.json, Unsloth mirror on Hugging Face
• QwQ-32B config.json, Qwen on Hugging Face