Ollama vs vLLM in November 2026: single-user speed, concurrency, formats and hardware

For one user, Ollama and vLLM are close: on an RTX 5090 running Qwen3.8-27B at 4-bit, vLLM 0.31.0 decoded at 85.7 tok/s and Ollama 0.35.1 at 76.8 tok/s in a test published in October 2026. Under load they are not close: at 32 concurrent requests the same card gave 936 tok/s in total on vLLM and 74 tok/s on Ollama, with Ollama's first token arriving after 54 seconds. Red Hat found the same pattern on an A100 (793 against 41 tok/s peak). Pick Ollama for one person on Windows, macOS or a single card, and vLLM for a shared Linux endpoint. Versions current on 7 November 2026: Ollama v0.40.15 and vLLM v0.33.0.

Speed rows: ComputingForGeeks, rented cards, October 2026, vLLM 0.31.0 and Ollama 0.35.1, 474-token prompt, 256 output tokens, thinking off. Other rows from the Ollama and vLLM docs and GitHub releases pages, read 7 November 2026. None of these numbers are GPUwerk measurements.

How much faster is vLLM than Ollama under load?

Between 8 and 28 times in the published tests, depending on card and model, once more than a handful of requests arrive at once. The table collects the two tests that ran both engines on the same hardware with the same model.

ComputingForGeeks ran Ollama with OLLAMA_NUM_PARALLEL=32, llama.cpp with -np 32 and vLLM with --max-num-seqs 32; vLLM used a RedHatAI INT4 (W4A16) checkpoint, the others Q4_K_M GGUF. Red Hat used GuideLLM 0.2.1 with 300-second runs and reports P99 latency at peak of 80 ms for vLLM against 673 ms for Ollama. Third-party figures, read 7 November 2026.

The mechanism is the scheduler. vLLM runs continuous batching: new requests join the running batch at every step, so the cost of reading the weights from memory is shared by everyone in the batch. Ollama gives each loaded model a fixed number of parallel slots, one by default according to the Ollama FAQ, and queues the rest (up to 512 requests before it returns a 503). Red Hat raised Ollama to 32 parallel slots, the highest stable value on its A100, and throughput still plateaued far below vLLM's. GPUwerk's own vLLM 0.27.1 run on one DGX Spark shows the same batching curve: Qwen3-Coder-30B-A3B went from 80.9 tok/s for one request to 3,235 tok/s in total at 256 concurrent requests (benchmarks page, 2 September 2026).

One caveat cuts the 5090 result down to size. The ComputingForGeeks author notes that Ollama 0.35.1 ignored the parallel setting for the hybrid Qwen3.8 architecture and served one request at a time, which is why its total stayed flat at 74 tok/s. Ollama's v0.40.15 release notes (3 November 2026) say recurrent and vision GGUF models can now handle more than one request at a time. I have not seen a re-test, so expect the gap on hybrid models to shrink somewhat. Red Hat's dense Llama result says it will not close.

The same split shows up on a DGX Spark, where Ollama stayed at 42.5 tok/s with 8 users on gpt-oss-120b while vLLM reached 116 to 153; the GB10 engine guide has those numbers and the Spark-specific build steps.

Is Ollama slower for a single user?

Slightly, and the difference is mostly the bundled runtime version, not the design. On the RTX 5090, Ollama decoded at 76.8 tok/s against 77.8 for upstream llama.cpp and 85.7 for vLLM. On the L40S, though, Ollama's bundled runtime gave 20.0 tok/s while upstream llama.cpp with the same flags gave 39.1, so an Ollama release that lags upstream can cost half your speed on some cards.

A quick bandwidth check shows how close all three run to the hardware. Decode reads every weight once per token, so the ceiling is bandwidth divided by model size. By my calculation: 1,792 GB/s (the RTX 5090's bandwidth per NVIDIA's spec table) divided by the 16.5 GB Q4_K_M file gives about 109 tok/s. vLLM reached 79% of that, llama.cpp 72% and Ollama 71%. The vLLM checkpoint is a different 4-bit build (19.5 GB on disk, including parts not read every token), so treat the vLLM percentage as approximate.

Speculative decoding moves single-user numbers more than the engine choice does. Ollama's library build of qwen3.8:27b ships a multi-token-prediction (MTP) draft that is on by default, and with it Ollama reached 129.7 tok/s on the 5090; vLLM with MTP enabled reached 135.3. If you compare engines, compare them with drafting either on in both or off in both.

How do Ollama's concurrency settings change memory?

Each parallel slot gets its own context, so memory for the KV cache grows with slots times context length. The Ollama FAQ states it directly: required memory scales with OLLAMA_NUM_PARALLEL × OLLAMA_CONTEXT_LENGTH. The context-length docs add that the default context depends on VRAM: 4k tokens under 24 GiB, 32k from 24 to 48 GiB, and 256k at 48 GiB or more.

A worked example for a 24 GB RTX 4090 and Qwen3.8-27B. The model's config.json has 64 layers, but only 16 are full attention (the other 48 are linear attention with a small fixed state), with 4 KV heads of dimension 256.

• KV bytes per token at f16: 2 (K and V) × 16 layers × 4 heads × 256 × 2 bytes = 65,536 bytes, or 64 KiB.
• One slot at Ollama's 32k default: 32,768 × 64 KiB = 2 GiB.
• Four slots: 8 GiB. With the 17 GB weights that is about 25 GB, more than the card has, so Ollama would push layers to the CPU.
• Four slots with OLLAMA_KV_CACHE_TYPE=q8_0, which the FAQ says halves KV memory: 4 GiB, about 21 GB in total, which fits with little room to spare.

vLLM handles the same problem differently: it reserves a fraction of GPU memory up front (0.92 by default unless its newer extensible KV cache is on) and fits as many sequences into that pool as their actual lengths allow, so a short request does not hold a full 32k reservation. Run ollama ps after loading: the CONTEXT column shows what Ollama allocated and PROCESSOR should read 100% GPU.

Which formats, APIs and multi-GPU modes does each support?

vLLM supports more formats and parallelism modes; Ollama supports more operating systems and older GPUs. The details as of 7 November 2026:

Read 7 November 2026. vLLM's quantization table has columns up to Hopper and no Blackwell column, so check GitHub issues for RTX 50 kernel status.

The tool-calling row matters more than it looks. Ollama applies the right template and parser for models in its library without flags. vLLM returns tool calls as plain text unless you start it with the parser that matches the model family, which is the most common reason an agent "ignores" its tools on a fresh vLLM install. The sibling article on open models for tool calling lists the parser per family.

Which one runs on a Mac, on Windows, or on a DGX Spark?

On a Mac or a Windows PC, Ollama is the practical choice; on GB10 both run, with vLLM ahead for teams. Ollama moved its Apple silicon backend to MLX: a preview in version 0.19 (30 March 2026) and the default for supported architectures from v0.40.0 (25 September 2026). In the launch post, Qwen3.5-35B-A3B went from 58 to 112 tok/s decode and from 1,154 to 1,810 tok/s prefill when moving from the Q4_K_M llama.cpp path to NVFP4 on MLX, on a Mac the post does not name (it asks for more than 32 GB of memory).

vLLM does not run natively on Windows; its docs point to WSL. On a Mac it needs vllm-metal, a community-maintained plugin with MLX underneath, whose newest named release is v0.28.0 (September 2026) with concurrent serving. It works, but it is not where vLLM's kernel work lands first. The DGX Spark vs Mac Studio page covers the hardware side of that choice.

On a DGX Spark, both engines run on ARM64 with the GB10's sm_121 GPU, but vLLM needs the CUDA 13 container rather than a pip install; the vLLM on DGX Spark page explains why.

When should you pick Ollama, vLLM or llama.cpp?

The number of people sending requests at the same moment decides it, more than the GPU does.

My own rule: if I am the only one sending requests, I run Ollama or llama.cpp and stop thinking about it. The moment a second person or an automated pipeline shares the endpoint, I switch to vLLM, because a 54-second first token is a broken product, whatever the throughput chart says. Neither engine has user accounts, so a team setup also wants Open WebUI for people and a gateway such as LiteLLM for per-user keys.

How do you run the same model on both and compare fairly?

Use the same model at the same bit width, turn speculative decoding off in both or on in both, disable thinking, and measure time to first token as well as tokens per second. Ollama with four slots and a quantized KV cache:

OLLAMA_NUM_PARALLEL=4 OLLAMA_KV_CACHE_TYPE=q8_0 OLLAMA_FLASH_ATTENTION=1 ollama serve
ollama pull qwen3.8:27b
ollama ps

vLLM in a fresh Python environment, following its install docs, with the tool parser on:

uv venv --python 3.12 --seed --managed-python
source .venv/bin/activate
uv pip install vllm --torch-backend=auto
vllm serve <hf-repo-of-a-4-bit-build> \
  --max-model-len 32768 \
  --max-num-seqs 32 \
  --enable-auto-tool-choice --tool-call-parser <parser-for-the-model>

On a 32 GB card the exact checkpoint and flags matter (the vLLM recipe for Qwen3.8-27B on one 5090 needs --enforce-eager to fit), so the full command and the known sm_120 issues are in the separate guide to vLLM on an RTX 5090. Both servers then answer on an OpenAI-style /v1 endpoint (Ollama on port 11434, vLLM on 8000), so the same load script can hit both. On a DGX Spark specifically, install steps and settings for Ollama are in Ollama on a DGX Spark.

FAQ

Sources

Every figure above is attributed to its source in the sentence or table where it appears. Pages were read on 7 November 2026 unless a date says otherwise.

• Releases · ollama/ollama, GitHub, 2026-11-03
• Releases · vllm-project/vllm, GitHub, 2026-11-02
• Ollama vs. vLLM: performance benchmarking (title shortened), Red Hat Developer, 2025-08-08
• Ollama vs vLLM vs llama.cpp: Speed Benchmarked, ComputingForGeeks, 2026-10-06
• FAQ, Ollama docs
• Context length, Ollama docs
• Hardware support, Ollama docs
• OpenAI compatibility, Ollama docs
• Anthropic compatibility, Ollama docs
• Ollama is now powered by MLX on Apple Silicon in preview, Ollama blog, 2026-03-30
• GPU installation, vLLM docs
• Quantization, vLLM docs
• Tool calling, vLLM docs
• Engine arguments, vLLM docs
• vllm-project/vllm-metal, GitHub
• Qwen/Qwen3.8-27B config.json, Hugging Face
• Compare GeForce graphics cards, NVIDIA