RTX 5090 power limit for LLM inference: speed, tokens per joule and electricity cost

Capping an RTX 5090 at 400 W instead of its 600 W default cost 1.4% of token-generation speed and 13.8% of prompt-processing speed in RodriMora's llama.cpp test (August 2025: 245.0 against 248.6 tok/s decode on Llama 2 7B), and an RTX PRO 6000 on the same GB202 chip lost 6.1% of decode but 48% of prefill at 300 W on Llama 3.3 70B (Level1Techs, December 2025). Decode is limited by the 1,792 GB/s memory bandwidth, which a power cap leaves largely intact, so for chat and agents a 400 to 450 W limit gives up almost nothing and cuts the worst-case draw by a third. At the EU household average of EUR 0.2832 per kWh, that is up to EUR 41 a month saved on a card running flat out.

How much speed does an RTX 5090 lose at a lower power limit?

About 1% of decode and 14% of prefill at 400 W, in the one controlled LLM test with both settings on the same RTX 5090; a test on the same chip at half power lost about 6% of decode. RodriMora posted it in the llama.cpp CUDA performance thread on 1 and 2 August 2025, running the thread's standard llama-bench test (Llama 2 7B Q4_0, build 9c35706, number 6060) first at a 400 W limit and then, at the thread author's request, at the default.

llama.cpp discussion #15013, RodriMora's comments of 1 and 2 August 2025, read 7 November 2026. RTX PRO 6000 rows: romeozor in the Level1Techs thread of 5 December 2025, llama.cpp build 7278, nvidia-smi -pl 300; the PRO 6000 Workstation uses the same GB202 chip and 1,792 GB/s as the 5090 with a 600 W rating. Percentages are my arithmetic. Another commenter replied in December 2025 that the extra 200 W was probably drawn but bought almost no speed. The RTX 5090 test used a small model; the 70B rows show the same pattern on a large one.

The 3090 row is the useful contrast. A 36% cut on Ampere cost 11% of decode; a 33% cut on the 5090 cost 1.4%. My reading is that the 5090 has enough compute left, even capped, to keep its memory bus busy during decode, which the 3090 at 250 W does not. Gaming behaves similarly: Hardware Corner's power-limit guide measured an RTX 4090 at 80% power drawing 360 W instead of 460 W for 1.9% fewer frames, a different workload with the same shape. Compute-heavy work is where the cap bites: Stable Yogi's RTX 5090 image-generation test (27 September 2026, Qwen Image 2.1, 1024 x 1024, 20 steps) went from 8.39 s per image at 575 W to 9.00 s at 450 W and 10.01 s at 400 W, 19% slower, with mean measured draw of 522.8, 423.3 and 378.5 W.

Why does LLM decode barely slow down under a power cap?

Because generating a token is a memory read, not a compute job: the card streams the active weights from VRAM once per token, and my reading of RodriMora's split (1.4% against 13.8%) is that a cap mostly slows the compute units while the memory bus keeps its speed. NVIDIA's comparison page lists 1,792 GB/s; a dense 17 GB model therefore has a ceiling near 105 tok/s, and the reported 78.9 tok/s for Qwen3.8-27B Q4_K_M sits at 75% of it. The worked roofline numbers are in the RTX 5090 benchmarks.

Prefill is the opposite: thousands of prompt tokens go through the matrix units in parallel, so prefill speed follows the core clock and the cap shows up directly, as the 13.8% loss above. If your workload is long prompts and short answers (document extraction, RAG with large retrieved context), a 13.8% slower prefill makes time to first token about 16% longer.

The memory side is also less elastic than people expect. adrienbrault's dual-5090 rig runs a +4,500 MHz memory offset, which its README puts at +14.3% DRAM bandwidth for only +1.7 to 1.8% decode at one stream, on a sparse model with MTP drafting. Small-active MoE models and drafted decode are partly limited by kernel launches and synchronization, so neither more bandwidth nor more power buys much there. On that evidence, a memory overclock buys little for LLM decode either.

How many tokens per joule does an RTX 5090 deliver?

At least 0.41 tokens per joule at 600 W and at least 0.61 at 400 W on the small reference model, by my calculation, so on these bounds the cap is worth about 48% more tokens per joule. I found no published wattage measured during RTX 5090 decode, so the figures below divide by the power limit. The card cannot exceed its cap for long, which makes these upper bounds on energy per token and lower bounds on tokens per joule. For compute-bound work the cap is close to the truth: the image test above averaged 91% of the 575 W limit, and the 400 W setting used 14% less energy per image (3,789 J against 4,386 J, my arithmetic) while taking 19% longer.

energy per token (J) ≤ power limit (W) ÷ tok/s
kWh per million tokens = J per token × 1,000,000 ÷ 3,600,000

Llama 2 7B Q4_0, decode (RodriMora)
  600 W: 600 ÷ 248.57 = 2.41 J/token → 0.67 kWh per million → 0.414 tok/J
  400 W: 400 ÷ 245.02 = 1.63 J/token → 0.45 kWh per million → 0.613 tok/J

Qwen3.8-27B Q4_K_M, decode 78.9 tok/s (witcheer)
  575 W: 7.28 J/token → 2.02 kWh per million
  400 W, assuming the same 1.4% loss: 5.14 J/token → 1.43 kWh per million

Qwen3.8-27B NVFP4, 32 concurrent, 1,361 tok/s (ComputingForGeeks)
  575 W: 0.42 J/token → 0.12 kWh per million

The last line is the bigger lesson. Serving 32 requests at once cut the energy per token about seventeen-fold against one user, while the power cap cut it by about 30%. If you care about efficiency, batch first and cap second. The 400 W line for the 27B model carries over the 1.4% loss from the 7B test, which is my assumption, not a measurement.

What does the electricity cost per million tokens in the EU?

At most about EUR 0.57 per million generated tokens for a dense 27B model served to one user at the EU average price, EUR 0.40 with a 400 W cap, and about EUR 0.03 when batched. Prices are Eurostat's household figures for the first half of 2026 (all taxes included, 2,500 to 5,000 kWh a year), from the electricity price statistics and dataset nrg_pc_204, updated 19 October 2026.

My arithmetic, card only: CPU, RAM, fans and power-supply losses come on top. A month is 730 hours. Real draw during single-user decode is likely below the cap, so these are ceilings. A country-by-country comparison for several GPUs is in the sibling article on local LLM electricity costs in Europe.

For scale: a DGX Spark draws 60 to 90 W during LLM inference according to the ServeTheHome measurement on the Spark power page, and GPUwerk measured Qwen3-Coder-30B-A3B on one at 80.9 tok/s. At 90 W that is at most 1.1 J per token, against at most 2.5 J for the same model family at 230.8 tok/s on a 575 W RTX 5090 (ai.rs, Ollama 0.19.0). Different quants and engines, so treat it as an order of magnitude; the speed comparison is on DGX Spark vs RTX 5090.

How do I set and keep a power limit on Linux?

Use nvidia-smi as root: enable persistence mode, set the limit in watts, and re-apply both at every boot. NVIDIA's nvidia-smi documentation says -pl takes a value between the card's Min and Max Power Limit and requires root, and that persistence mode "does not persist across reboots". The documented knobs are the power cap and clock locks (-lgc for the GPU clock, supported on Volta and later); there is no voltage-curve editor, so on Linux an "undervolt" is a power cap plus, optionally, a clock lock. On Windows, the same cap is a slider in MSI Afterburner.

# read the allowed range and the current limit
nvidia-smi -q -d POWER

# cap GPU 0 at 400 W (root)
sudo nvidia-smi -pm 1
sudo nvidia-smi -i 0 -pl 400

# watch draw, limit, clocks and temperature every 5 s while a benchmark runs
nvidia-smi --query-gpu=power.draw,power.limit,clocks.sm,clocks.mem,temperature.gpu \
  --format=csv -l 5

# /etc/systemd/system/gpu-power-limit.service
[Unit]
Description=Set RTX 5090 power limit

[Service]
Type=oneshot
ExecStart=/usr/bin/nvidia-smi -pm 1
ExecStart=/usr/bin/nvidia-smi -i 0 -pl 400

[Install]
WantedBy=multi-user.target

Enable the unit with systemctl enable gpu-power-limit.service. Defaults differ by board: the adrienbrault rig reports 600 W on its ASUS card and 575 W on an HP OEM card, against NVIDIA's 575 W rating, so read the range before choosing a number. Then rerun one benchmark at the new limit; the commands are in the benchmarks article.

Is the 12V-2x6 connector safe for 24/7 LLM load?

It has a documented failure mode on the RTX 5090, and a power cap is the simplest way to widen its margin. NVIDIA's RTX 5090 page specifies one 600 W PCIe Gen 5 cable or four 8-pin cables through an adapter, and asks for a 1,000 W system. Wikipedia's 12VHPWR article gives 9.5 A per pin, 684 W across six 12 V pins and a safety factor of 1.14 against 600 W. It also notes that from the RTX 40 series on, the 12 V pins are tied together on the card, so the GPU cannot tell if one pin carries too much. The RTX 50 series article lists melted connectors on the Founders Edition and on partner cards.

By my arithmetic, assuming all board power goes through the cable and splits evenly, six pins carry 8.0 A each at 575 W (84% of the rating), 6.3 A at 450 W and 5.6 A at 400 W (58%). Part of the power comes through the PCIe slot, so the real figures are a little lower; uneven sharing between pins is exactly the failure case, which is why the headroom matters more than the average. The practices that follow from the documented cases: seat the plug fully (partial seating was the main cause of 4090 failures), use a native 12V-2x6 cable from an ATX 3.1 supply, and avoid frequent re-plugging, since the article cites 30 to 40 mating cycles as the limit before contact may degrade. For a card that runs inference around the clock, a cap at 400 to 450 W costs little speed, as shown above, and removes 22 to 30% of the current through those pins against the 575 W rating. The same applies twice over in a dual RTX 5090 build.

FAQ

Sources

Every figure above is attributed to its source in the sentence or table where it appears. Pages were read on 7 November 2026 unless a date says otherwise.

• Performance of llama.cpp on Nvidia CUDA (discussion #15013), RodriMora's 400 W and 600 W runs, ggml-org/llama.cpp on GitHub, 2025-08-02
• GeForce RTX 5090 graphics card specifications, NVIDIA
• Compare GeForce graphics cards, NVIDIA
• nvidia-smi documentation, NVIDIA
• Electricity price statistics, Eurostat, 2026-10
• Electricity prices for household consumers, bi-annual data (nrg_pc_204), API query for EU, DE, CZ, 2026-S1, Eurostat, 2026-10-19
• 16-pin 12VHPWR connector, Wikipedia
• GeForce RTX 50 series, Wikipedia
• qwen3.8-flash-next-2x-rtx5090 (memory clock and power settings), adrienbrault on GitHub, 2026-10-23
• TensorRT-LLM vs vLLM on RTX 5090: Tested, ComputingForGeeks, 2026-10-13
• witcheer/rtx-5090-benchmarks, Hugging Face, 2026-10-26
• Run Qwen3-Coder 30B-A3B on an RTX 5090: 231 tok/s with Ollama, ai.rs, 2026-05-22
• How to Power Limit your GPU with MSI Afterburner, Hardware Corner, 2022-10-18
• RTX Pro 6000 600W vs 300W performance?, Level1Techs Forums, 2025-12-05
• RTX 5090 Qwen Image 2.1 power limit benchmark, Stable Yogi, 2026-09-27