Ollama on a DGX Spark: install, the settings that matter, and how fast it is next to llama.cpp and vLLM

Ollama runs on a DGX Spark from its standard arm64 build (v0.40.15, released 3 November 2026) and uses the GB10 GPU with no extra setup, but its defaults were picked for laptops. On one Spark it decodes gpt-oss-120b at about 42 tok/s for one user, against 58.7 tok/s for llama.cpp and 60.7 for vLLM in published runs, and with the default single slot eight concurrent users share those same 42 tok/s. Set the context length, KV cache type, parallel slots and keep-alive yourself, and keep port 11434 off the internet, because Ollama has no authentication.

Defaults read in Ollama's documentation on 7 November 2026. The right-hand column is my recommendation for a 128 GB Spark, explained below.

How do I install Ollama on a DGX Spark?

With Ollama's normal Linux install script, which on a standard DGX OS install also sets Ollama up as a systemd service. From Ollama's Linux docs:

curl -fsSL https://ollama.com/install.sh | sh
ollama -v
nvidia-smi            # the GB10 should be listed
ollama run gpt-oss:20b --verbose "Explain KV cache in two sentences."
ollama ps             # PROCESSOR should read 100% GPU

--verbose prints the prompt and eval rates after the answer, which is the quickest speed check. ollama ps shows where the model sits; the FAQ says "100% GPU" means it is fully in GPU memory, and on a Spark that memory is the same 128 GB pool as system RAM.

Inside a container without systemd, which is what a rented GPUwerk instance is (see the tenant environment), use the tarball instead and keep it under /workspace so it survives a rebuild. This is Ollama's documented arm64 command with a different target directory:

mkdir -p /workspace/ollama
curl -fsSL https://ollama.com/download/ollama-linux-arm64.tar.zst \
    | tar x -C /workspace/ollama        # needs the zstd package
export OLLAMA_MODELS=/workspace/ollama/models
nohup /workspace/ollama/bin/ollama serve >> /workspace/ollama/serve.log 2>&1 &

Put the last two lines in the instance's startup hook to start it after every restart. To update, rerun the same download over the same directory; Ollama's docs give the same advice for /usr.

Which Ollama settings matter on a Spark?

Context length and parallel slots, because together they decide memory use. Ollama's FAQ says "Required RAM will scale by OLLAMA_NUM_PARALLEL * OLLAMA_CONTEXT_LENGTH", and its context-length page says the default is 4k below 24 GiB of VRAM, 32k from 24 to 48 GiB and 256k at 48 GiB or more. A Spark's GPU sees well over 48 GiB, so expect the 256k tier, capped at what the model supports; the FAQ page still says 4,096, so read the CONTEXT column in ollama ps rather than trusting either.

What that costs, by my arithmetic from each model's config.json (2 x full-attention layers x KV heads x head dimension x 2 bytes per token):

gpt-oss-120b: 18 full layers x 8 KV heads x 64 dim  = 36 KiB per token
              131,072 tokens x 1 slot  = 4.5 GiB f16, 2.25 GiB q8_0
Llama 3.3 70B: 80 layers x 8 KV heads x 128 dim     = 320 KiB per token
              131,072 tokens x 1 slot  = 40 GiB f16
              131,072 tokens x 4 slots = 160 GiB f16: does not fit next to 37 GiB of weights

So gpt-oss is cheap to give long context, and a dense 70B is not. The memory calculator does this for other models. My settings for a Spark used by one to three people, applied with sudo systemctl edit ollama:

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=65536"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
Environment="OLLAMA_NUM_PARALLEL=2"
Environment="OLLAMA_KEEP_ALIVE=-1"
Environment="OLLAMA_MAX_LOADED_MODELS=2"

Then sudo systemctl daemon-reload and sudo systemctl restart ollama. The FAQ describes q8_0 as about half the memory of f16 with "a very small loss in precision" and q4_0 as a quarter with "a small-medium loss"; KV quantization needs flash attention, and it is global, so every model gets the same type. 65,536 tokens follows Ollama's own advice of "at least 64000 tokens" for agents and coding tools. KEEP_ALIVE=-1 keeps a model loaded indefinitely: reloading gpt-oss-120b's 65 GB every time the default five minutes pass is wasted time on a box that does nothing else. A single request can override both, through "keep_alive" and "options": {"num_ctx": ...} in the API call.

How fast is Ollama on a DGX Spark compared with llama.cpp and vLLM?

For one user it is close to vLLM on some models and about 30% behind llama.cpp on gpt-oss; under load it falls far behind vLLM. Published single-stream decode on one Spark:

tok/s, all third-party figures. The forum test (author ss121, 400 output tokens, driver 580.173.02, CUDA 13.0) did not state its Ollama version or settings, and its author says the concurrent figures varied by up to 30% between runs. llama.cpp's figures are llama-bench tg32 and llama-batched-bench, a different method, so compare them with care. The Qwen3-30B-A3B llama.cpp row is a Q8_0 build of the Coder variant, which reads twice the bytes of a 4-bit file.

Prefill shows a bigger gap than decode. Ollama's own test put gpt-oss-120b prompt processing at 1,169 tok/s; llama.cpp's table gives 2,444 tok/s at pp2048. The forum test measured time to first token at 449 ms in Ollama against 131 ms in vLLM on gpt-oss-120b. In January, eugr, who maintains a widely used Spark vLLM build, told a new owner on NVIDIA's forum: "forget about Ollama and use llama.cpp instead" (thread 356716). For the engine choice in general, see llama.cpp vs vLLM vs SGLang on a Spark and the planned Ollama vs vLLM comparison.

Can Ollama serve several users on one Spark?

Only a few, and only after you raise OLLAMA_NUM_PARALLEL. With the default of 1, requests queue: the FAQ says the queue holds 512 and returns HTTP 503 beyond that. In the August forum test, Ollama's total at eight concurrent requests equalled its single-stream figure on all four models, and the author's conclusion was that "it does not batch". He did not state his settings, so I read that as the default single slot. Each extra slot reserves its own context window, which is why 2 to 4 slots at 64k is the practical ceiling on 128 GB with a large model. Past four or five regular users, vLLM's continuous batching is the better tool; the llama.cpp guide covers the middle ground.

Which models should I pull on a Spark?

Mixture-of-experts models, because they read few bytes per token and the Spark's 273 GB/s rewards that. From Ollama's library pages, read 7 November 2026:

Skip the -mlx tags, which are Apple-only builds, and the -cloud tags: those models run on Ollama's servers, not on your Spark, so your prompts leave the machine. Ollama's blog notes that its official gpt-oss builds keep attention layers in BF16, while some MXFP4 GGUFs elsewhere quantize them further to q8_0.

Two or three of these fit side by side in 128 GB; gpt-oss:120b plus qwen3.6:35b is 88 GB of weights before KV cache. For model-specific fit and speed, see the gpt-oss-120b page.

How do I expose the Ollama API safely?

Do not bind it to 0.0.0.0 on a machine with a public address. Ollama has no built-in authentication; its FAQ explains how to change OLLAMA_HOST and how to put Nginx, ngrok or Cloudflare Tunnel in front, but not how to secure them. The risk is not theoretical. Cisco Talos counted 1,139 Ollama servers reachable from the internet in a September 2025 scan, 214 of them answering with loaded models (Cisco). Wiz's CVE-2024-37032 showed a path-traversal bug that led to remote code execution, fixed in Ollama 0.1.34, and Wiz's advice for tools like Ollama was that they "should never be exposed externally without protective middleware, such as a reverse proxy with authentication" (Wiz).

For your own use, an SSH tunnel is enough:

ssh -N -L 11434:127.0.0.1:11434 you@your-spark
# on your laptop:
curl http://127.0.0.1:11434/v1/models

The /v1 path is Ollama's OpenAI-compatible API, so editors and agent tools work against http://127.0.0.1:11434/v1. Key setup for the tunnel is in the SSH keys guide. For a team, put a reverse proxy that checks a key or a login in front, or use Open WebUI's accounts, below. If you connect agent tools, the planned guide to MCP with local models covers the tool side.

How do I add Open WebUI?

NVIDIA's Open WebUI and Ollama playbook (updated 14 October 2026, for 64 GB and 128 GB Sparks) uses Open WebUI's image with Ollama built in. Its desktop instructions run:

docker run -d -p 8080:8080 --gpus=all \
  -v open-webui:/app/backend/data \
  -v open-webui-ollama:/root/.ollama \
  --name open-webui ghcr.io/open-webui/open-webui:ollama

That publishes port 8080 on every interface. On a networked Spark I would change it to -p 127.0.0.1:8080:8080 and reach it through a tunnel. You then download models from the web interface. If Ollama is already running as a service, point an existing Open WebUI at http://127.0.0.1:11434 instead of running a second copy; the Open WebUI setup guide covers accounts, signups and HTTPS.

My verdict: Ollama is the right choice for one person who wants to pull a model and chat in five minutes, and for trying many models quickly. If you care about single-user speed, build llama.cpp and get roughly 40% more on gpt-oss-120b. If several people will use the Spark at once, run vLLM. GPUwerk's 128 GB Sparks rent for $0.79 per hour, which is enough to run all three engines on your own prompts in one evening. If you prefer a desktop app, LM Studio on a DGX Spark covers the same box.

FAQ

Sources

Every figure above is attributed to its source in the sentence or table where it appears. Pages were read on 7 November 2026 unless a date says otherwise.

• Linux install, Ollama docs
• FAQ, Ollama docs
• Context length, Ollama docs
• Ollama releases, GitHub, 2026-11-03
• NVIDIA DGX Spark performance, Ollama Blog, 2025-10-23
• Chat with LLMs Using Open WebUI and Ollama, NVIDIA build.nvidia.com, 2026-10-14
• Open WebUI playbook instructions, NVIDIA build.nvidia.com
• Measured inference benchmarks on a single DGX Spark (Ollama, llama.cpp and vLLM, notes and data published), NVIDIA Developer Forums, 2026-08-10
• DGX Spark performance, NVIDIA Developer Forums, 2026-01-07
• llama.cpp DGX Spark benchmarks, ggml-org on GitHub, 2026-02-05
• gpt-oss model library, Ollama
• qwen3.6 model library, Ollama
• gemma4 model library, Ollama
• Detecting Exposed LLM Servers: A Shodan Case Study on Ollama, Cisco Talos, 2025-09-01
• Probllama: Ollama Remote Code Execution Vulnerability (CVE-2024-37032), Wiz, 2024-06-24