Set up Open WebUI on your Spark
An endpoint your colleagues have to curl is an endpoint your colleagues will not use. Open WebUI is the piece that turns a vLLM server into something that looks like ChatGPT, with accounts, chat history, document upload and model switching, running entirely on your hardware.
1, Install it
Open WebUI does no GPU work. It is a frontend that talks to an inference endpoint over HTTP, so it runs happily in a container next to whatever is serving your model.
docker run -d \ --name open-webui \ --network host \ -v /workspace/open-webui:/app/backend/data \ --restart unless-stopped \ ghcr.io/open-webui/open-webui:main
Two choices in there are deliberate. --network host means the container reaches localhost:8000 directly, which is where your vLLM server is, without any bridge networking to configure. And the data volume is a bind mount under /workspace rather than a named Docker volume, because on a GPUwerk instance /workspace is the directory that survives a stop and restore.
With host networking the port is 8080, not the 3000 you will see in the upstream quick start (that is the host side of a -p 3000:8080 mapping). Open http://localhost:8080 through an SSH tunnel and you should get the sign-up screen.
Prefer not to use Docker? pip install open-webui then open-webui serve works, and needs Python 3.11. The container is easier to keep clean, and it is what the project ships first.
2, Connect it to your model
Open WebUI speaks two protocols: the OpenAI-compatible API, and Ollama's native one. For a vLLM backend you want the first.
The lowest-friction way is the admin interface: sign in, then Settings, Admin Settings, Connections, add an OpenAI-compatible connection with the base URL http://localhost:8000/v1 and any non-empty API key string. vLLM does not check the key unless you started it with --api-key, but the field cannot be blank.
To bake the connection into the container instead, so a fresh deployment comes up already configured, set it at run time:
docker run -d \ --name open-webui \ --network host \ -e OPENAI_API_BASE_URLS="http://localhost:8000/v1" \ -e OPENAI_API_KEYS="sk-local" \ -e ENABLE_OLLAMA_API=False \ -v /workspace/open-webui:/app/backend/data \ --restart unless-stopped \ ghcr.io/open-webui/open-webui:main
Note the plural. OPENAI_API_BASE_URLS takes a semicolon-separated list, and Open WebUI will balance across the entries, which is how you would front two Sparks with one interface. ENABLE_OLLAMA_API=False stops it probing for an Ollama server you are not running, which otherwise shows up as a slow page load and a red error toast.
Your models appear in the model picker because Open WebUI calls /v1/models on startup. If the picker is empty, that call is failing: check the endpoint from inside the container with docker exec open-webui curl -s http://localhost:8000/v1/models before touching anything in the UI. The full variable list is in the Open WebUI environment configuration reference, which runs to well over 200 entries.
Pointing Open WebUI at LiteLLM instead of straight at vLLM gives you per-user API keys, spend tracking and fallback routing, while Open WebUI keeps seeing one ordinary OpenAI endpoint. That is the setup we would run for anything beyond a handful of people.
3, Control who gets in
The first account created becomes the admin. That is a race, so create it immediately after starting the container, before the port is reachable by anyone else, and then close the door:
# after your admin account exists
-e ENABLE_SIGNUP=False
If you want colleagues to self-register but not get access on their own, leave signup on and rely on the default: new accounts are created with the pending role and can do nothing until an admin approves them from the users panel. That is a reasonable default for an internal tool, and it is the behaviour you get out of the box.
One variable deserves a warning. WEBUI_AUTH=False disables authentication entirely and turns the instance into a single shared session with no login. It is fine on a laptop; on anything reachable by another human it means every visitor reads everyone's chat history.
If you ever run more than one replica behind a load balancer, set WEBUI_SECRET_KEY explicitly and identically on all of them. Left unset, each instance generates its own key and saves it to .webui_secret_key in the data directory, so a token issued by one replica is rejected by the next and users get logged out at random. Generate one with openssl rand -base64 32.
4, Add your own documents
The feature most people install Open WebUI for is asking questions against their own files. Upload documents through the workspace panel and they are chunked, embedded and stored locally, then retrieved and injected into the prompt at query time. Attach a file to a single chat with the paperclip, or add it to a knowledge collection and reference the collection with # in the message box to make it available across conversations.
Two things are worth knowing before you load a large corpus. The embedding model runs locally too, which is a small extra memory cost alongside your main model, and the default is tuned for size rather than quality, so swapping it for a stronger multilingual embedder in the admin settings is usually the single biggest retrieval improvement available. And retrieval quality depends far more on how your documents are split than on which model answers, so a PDF that is mostly tables will disappoint regardless of the LLM behind it.
The whole point on this hardware is that none of that content leaves the box. The documents, the embeddings and the prompts all stay in the data directory on your instance.
5, Put it behind HTTPS
For your own use, the SSH tunnel is enough and needs no certificates. For a team, you want a real URL.
On a GPUwerk instance, the simplest route is to serve Open WebUI on the instance's public endpoint: our gateway routes https://<name>.gpuwerk.com to port 8888 in the container, so run Open WebUI on 8888 and TLS is handled for you. On your own hardware, put Caddy in front and let it get a certificate:
# /etc/caddy/Caddyfile
chat.yourcompany.com {
reverse_proxy localhost:8080
}
That is the entire config. Caddy provisions and renews the certificate from Let's Encrypt on its own, provided the DNS record points at the machine and ports 80 and 443 are reachable.
6, Back up the one directory that matters
Everything Open WebUI knows lives in /app/backend/data: the SQLite database with accounts and chat history, the vector store, uploaded files, and the secret key. Back up that one directory and a lost instance costs you nothing but the time to start a container.
tar czf openwebui-$(date +%F).tar.gz -C /workspace open-webui
Restoring is the same container command pointed at the extracted directory. Do this before an upgrade, not after one.