B300 vs B200 vs H200 for inference: MLPerf v6.1 results, rental prices and what fits

In MLPerf Inference v6.1 (results published 16 September 2026), an 8-GPU B300 server produced 12 to 23 percent more tokens per second than an 8-GPU B200 and roughly three times an 8-GPU H200. On 7 November 2026 the three rented for $8.99, $7.99 and $5.29 per GPU-hour at RunPod Secure Cloud, so by my arithmetic the B200 and B300 cost about the same per million tokens and the H200 about twice as much. The B300's real advantage is memory: 288 GB per GPU against 180 GB and 141 GB, at the same 8 TB/s as the B200.

Specs from NVIDIA's HGX, H200 and Blackwell Ultra pages; throughput from the MLPerf v6.1 results file (CoreWeave rows) and Red Hat's v6.0 H200 post; prices read on RunPod's pricing page on 7 November 2026. Each figure is sourced again in the sections below.

How do the B300, B200 and H200 specs compare?

The B300 has more memory and more FP4 compute than the B200, but the same memory bandwidth and the same NVLink. NVIDIA's Blackwell Ultra technical post (22 August 2025) lists 288 GB of HBM3e at 8 TB/s for Blackwell Ultra against 192 GB at 8 TB/s for Blackwell, and 141 GB at 4.8 TB/s for the H200. The HGX platform page gives the eight-GPU board figures I divided by eight below.

Read 7 November 2026. NVIDIA's blog rates Blackwell and Blackwell Ultra at 10 and 15 dense FP4 PFLOPS, with maximum power of up to 1,200 W and 1,400 W; the HGX board figures divided by eight are lower (9 and 13.5), which suggests the HGX parts run at lower power. NVIDIA does not spell this out. MLPerf v6.1 system descriptions list the B300 as "B300-SXM-270GB"; Verda lists it as 268 GB.

Two rows matter for buyers who do more than inference. The B300 gives up almost all FP64 (10 TFLOPS per board against 296 for the B200), so it is the wrong card for simulation codes. And it doubles scale-out bandwidth with ConnectX-8, which counts when a model spans several nodes.

What does MLPerf Inference v6.1 show?

The B300 leads the B200 by 12 to 23 percent on the three large-language-model benchmarks where both have closed-division results, and the B200 leads the H200 by about three times. MLCommons published v6.1 on 16 September 2026 with 30 submitters and 120 systems, per the working-group chairs. I took the numbers from the official results file and picked pairs from the same submitter where possible, so the cloud setup is constant.

Tokens per second, Datacenter Closed, "available" systems. B200 and B300 rows ran TensorRT per their system names. No 8x H200 system submitted these three benchmarks in v6.1's closed division, so the H200 column comes from earlier rounds and, for gpt-oss-120b, a different engine (vLLM); read it as a floor, not a like-for-like figure. The best 8-GPU B300 DeepSeek-R1 Offline row was GigaComputing's 72,445 tokens/s; NVIDIA's own DGX B300 did 70,132.

Within the B300 rows the spread is small. Fifteen 8-GPU B300 systems from 11 submitters (ASUS, CoreWeave, Dell, GigaComputing, HPE, Inventec, NVIDIA, Nebius, Oracle, QCT and Supermicro) landed between 100,183 and 113,685 tokens/s on gpt-oss-120b Server. Who builds the box matters less than which GPU is in it.

Why is the B300 only 12 to 23 percent faster than the B200?

Because much of LLM inference is limited by memory bandwidth, and the B300 has the same 8 TB/s. Each generated token reads the active weights and that sequence's KV cache out of HBM, and at large batch sizes the KV reads grow with every request added. More FP4 compute (13.5 against 9 PFLOPS per GPU on the HGX boards) and twice the attention throughput speed up prefill and the softmax inside attention, which is why the gap is widest on gpt-oss-120b and DeepSeek-R1, two reasoning models with long outputs and heavy attention. Llama 2 70B is a dense model with short sequences, and there the B300 gains 12 percent.

The larger HBM helps in a way MLPerf barely measures. A benchmark run fixes the model and the accuracy target; it does not reward holding a 1M-token context or a thousand concurrent sessions. On a production server, 270 to 288 GB per GPU means more KV cache per GPU, larger batches before you hit memory, and big MoE models on fewer GPUs. That is where the 50 percent memory increase turns into tokens per dollar.

The H200 gap is the reverse case. Against the B200 it has 60 percent of the bandwidth, no FP4 tensor cores and half the NVLink, and NVIDIA's own v5.0 result on Llama 2 70B put 8x B200 at 2.8 times 8x H200. The other Hopper card is covered in the DGX Spark vs H200 comparison, which explains the bandwidth arithmetic for single-user decode.

How much do they cost to rent?

On 7 November 2026, on-demand B300s cost $8.99 to $11.69 per GPU-hour on the pricing pages I read, B200s $7.69 to $9.14 and H200s $5.29 to $6.44. Community and spot tiers are cheaper. Two providers I checked publish no B300 price: CoreWeave asks you to contact sales, and Lambda's page does not list it.

USD per GPU-hour. CoreWeave per-GPU figures are my division of its per-instance price. A wider provider table, with reserved rates and the H100, is in the planned H100, H200 and B200 rental price comparison.

What does a million tokens cost on each?

About $0.17 to $0.19 on a B200 or B300 and $0.34 to $0.41 on an H200, by my calculation. The method: take the 8-GPU MLPerf Offline throughput, multiply by 3,600 seconds, and divide the node's hourly price at RunPod Secure Cloud by that.

cost per million tokens = (price per GPU-hour x 8) / (tokens/s x 3,600 / 1,000,000)

gpt-oss-120b, B300: ($8.99 x 8) / (112,840 x 3,600 / 1e6) = $71.92 / 406.2 = $0.177
gpt-oss-120b, B200: ($7.99 x 8) / (91,487 x 3,600 / 1e6)  = $63.92 / 329.4 = $0.194
gpt-oss-120b, H200: ($5.29 x 8) / (28,680 x 3,600 / 1e6)  = $42.32 / 103.2 = $0.410
Llama 2 70B,  B300: $71.92 / 415.9 = $0.173
Llama 2 70B,  B200: $63.92 / 369.7 = $0.173
Llama 2 70B,  H200: $42.32 / 126.0 = $0.336
DeepSeek-R1,  B300: $71.92 / 250.8 = $0.287
DeepSeek-R1,  B200: $63.92 / 204.9 = $0.312

On these numbers the B300's 12.5 percent price premium over the B200 at RunPod is roughly what its throughput gain is worth: a tie on Llama 2 70B, a small win on gpt-oss-120b and DeepSeek-R1. The H200 is cheaper per hour and more expensive per token.

Which models fit one GPU or one 8-GPU node?

Weights up to about 130 GB fit one GPU of any of the three; at 150 to 250 GB only the B300 holds it on one GPU; and a trillion-parameter-class MoE needs a full node of H200s or half a node of B300s. Weights are parameters times bytes per parameter (FP8 = 1 byte, NVFP4 about 0.56 bytes with its scales), and the leftover memory is KV cache. The table is my arithmetic from the Hugging Face sizes and the per-GPU memory above (141, 180 and 270 GB usable).

DeepSeek sizes are my arithmetic from the tensor counts in the Hugging Face metadata (the Pro repository uses 865 GB of storage). The 86,016-token H200 cap and the 1M-token B200 figure come from NVIDIA's Dynamo recipe, which runs the H200 on the public checkpoint with Marlin MoE kernels and the B200 on NVIDIA's NVFP4 build. The NVFP4 model card reports serving on GB300 with tensor parallelism 4, which is the basis for the four-B300 row. GPU counts in practice round up to a power of two for tensor parallelism.

The DeepSeek-V4-Pro row is the clearest case for the extra memory. On eight H200s the weights leave about 310 GB for everything else (1,128 minus 815, my subtraction), and NVIDIA's recipe caps context at 86,016 tokens, below the 393,216 its own "Think Max" mode needs. Eight B200s leave about 625 GB (assuming NVIDIA's NVFP4 build is no larger) and serve the full 1M. If your model fits in 128 GB, none of this applies: a two-Spark cluster runs DeepSeek-V4-Flash at a fraction of the hourly cost, and the memory calculator does the same subtraction for one box.

Which should you pick for an 8-GPU server?

For most inference buyers in November 2026, I'd pick the B200: it ties the B300 on cost per token in these numbers and is listed by every provider I checked. Pick the B300 when memory is the constraint. Pick the H200 only when the price gap is larger than the throughput gap, or for FP8-only stacks that never use FP4.

• B300 if you serve one very large MoE (DeepSeek-V4-Pro, a 1T model) on half the GPUs, need long context at high concurrency, or span several nodes and want the 1.6 TB/s networking. Skip it for FP64 work.
• B200 for general LLM serving and fine-tuning where 180 GB per GPU is enough. It also keeps FP64, which the B300 drops.
• H200 if you can get it far below $5 per GPU-hour, your stack is FP8 and your models fit in 1,128 GB per node. At RunPod's list prices it costs about twice as much per token.

Two things to check before signing a term. First, the format: the B200 and B300 numbers above rely on FP4 weights, and NVFP4, MXFP4 and FP8 behave differently on accuracy, so test your model in the format you will run. Second, timing: NVIDIA lists an HGX Rubin NVL8 board with 2 TB of HBM4 and 130 TB/s, and what that means for a contract signed now is in the Vera Rubin buyer's guide. GPUwerk quotes 8x B200 and 8x H100 servers per configuration and has none installed today; the B300 numbers here are other people's.

FAQ

Sources

Every figure above is attributed to its source in the sentence or table where it appears. Pages were read on 7 November 2026 unless a date says otherwise.

• NVIDIA HGX Platform: HGX B300 and HGX B200 specifications, NVIDIA
• NVIDIA DGX B300, NVIDIA
• NVIDIA DGX B200, NVIDIA
• NVIDIA H200 Tensor Core GPU, NVIDIA
• Inside NVIDIA Blackwell Ultra: The Chip Powering the AI Factory Era, NVIDIA Technical Blog, 2025-08-22
• MLPerf Inference v6.1 results summary (summary.csv), MLCommons on GitHub, 2026-09-16
• Where the Industry Is Investing: A Look at MLPerf Inference v6.1, MLCommons, 2026-09-17
• NVIDIA Blackwell Delivers Massive Performance Leaps in MLPerf Inference v5.0, NVIDIA Technical Blog, 2025-04-02
• Red Hat AI tops MLPerf Inference v6.0 with vLLM on Qwen3-VL, Whisper, and GPT-OSS-120B, Red Hat, 2026-04-01
• RunPod pricing, RunPod
• Nebius AI Cloud prices, Nebius
• CoreWeave pricing, CoreWeave
• Lambda pricing, Lambda
• Verda pricing, Verda
• DeepSeek V4 Pro recipe, NVIDIA Dynamo docs
• deepseek-ai/DeepSeek-V4-Pro model metadata, Hugging Face
• nvidia/DeepSeek-V4-Pro-NVFP4, Hugging Face, 2026-05-27