Connecting Open WebUI to internal tools and APIs
By the end of this page, your Open WebUI install can call an internal API through a Python tool and answer questions from a knowledge collection of your own documents. It starts from a working install; the Open WebUI setup guide covers install, model connection, accounts and HTTPS.
You need:
- A working Open WebUI connected to a model on your DGX Spark, with an admin account.
- A model that supports function calling (see the first section).
- An internal API the Open WebUI host can reach, if you want to follow the tool example.
- No SSH key to deploy the Spark. GPUwerk configures the machine, and the console's browser terminal works without one. Add your own key only for your own SSH client, scp or an SSH tunnel to the web interface.
Function calling needs the right model
Open WebUI's tools give the model a description of your Python functions and let it decide when to call one, the same function calling agent frameworks use. That depends on the model as well as on Open WebUI. A model not trained to emit structured tool calls either ignores the tools or produces a call in the wrong format.
- Confirm the model you serve on your Spark supports function calling.
- If you run vLLM yourself, start it with tool calling enabled for that model (
--enable-auto-tool-choiceplus the--tool-call-parserfor its chat template). The GPUwerk llama.cpp presets startllama-serverwith--jinja, which enables tool calls; the Qwen3 Coder 30B vLLM preset does not start with--enable-auto-tool-choice, so Open WebUI's native function-calling mode does not work against it. Open WebUI's default mode, which selects tools through a prompt, does not need engine support. - Test with a trivial tool before you write anything larger. You should see a tool-call indicator in the chat.
Writing a tool
A tool in Open WebUI is a Python file with a class whose public methods the model can call. Add one from Workspace, Tools, New Tool and save it; it should then appear in the tools list. This example looks up an internal ticket by ID:
"""
title: Ticket Lookup
description: Look up a support ticket by ID from the internal API.
"""
import requests
class Tools:
def __init__(self):
pass
def get_ticket(self, ticket_id: str) -> str:
"""
Look up a support ticket by its ID.
:param ticket_id: The ticket ID, e.g. TICK-1234
"""
resp = requests.get(
f"http://internal-api.local/tickets/{ticket_id}",
timeout=5,
)
resp.raise_for_status()
return resp.text
The docstring, type hint and parameter comment matter: Open WebUI passes them to the model as the tool's description. A vague description is why a model skips a tool or calls it with the wrong argument. The tool runs as Python inside the Open WebUI backend, so it can reach anything the Open WebUI host can reach: an internal REST API, a database, a wiki search endpoint.
Enabling a tool on a model
- Go to Workspace, Models and select the model.
- Enable the tool under its capabilities and save. To test a tool first, turn it on for one chat from the tool icon in the message input instead.
- Ask something that needs the tool, such as "What is the status of TICK-1234?". You should see a tool-call indicator and an answer built from the API response.
The model decides per message whether to call the tool; it does not run on every message.
RAG over internal documents
The setup guide's document section covers uploading a file to one chat. This section applies the same pipeline to a whole body of internal documents: a wiki export, a folder of PDFs, a knowledge base dump.
The mechanics are the same: documents are chunked, embedded with a local embedding model and stored in a local vector store, and relevant chunks are added to the prompt at query time. None of that leaves the host running Open WebUI. At knowledge-base scale, revisit two defaults that are tuned for single documents. A stronger multilingual or domain-tuned embedding model in Admin Settings, Documents usually helps most, and chunk size matters more once documents are long and varied.
Knowledge collections vs per-chat files
For internal documents, use a knowledge collection rather than per-chat uploads.
- Create a collection in Workspace, Knowledge.
- Upload the document set into it and wait for processing to finish.
- In any chat, type
#collection-nameand ask a question. The answer should cite the documents it used.
Every conversation and every user with access can then use the corpus without uploading it again. Access follows Open WebUI's user and group permissions, so a collection built from an HR policy folder can be limited to the people who should see it.
What actually breaks
- Plausible answers, but no tool-call indicator. Cause: the model answers from its own knowledge instead of calling the tool. Fix: use a model and engine with function calling enabled, and check for the indicator before you trust the output.
- The whole turn ends in an error. Cause: an API timeout or non-200 response raised inside the tool. Fix: catch failures in the tool and return a message the model can work with.
- Answers miss content you know is in the collection. Cause: scanned PDFs or tables stored as images retrieve poorly with any model. Fix: extract or OCR that content before you load it.
- A tool reaches more than it should. Cause: tools run with the network access of the Open WebUI host. Fix: scope credentials and network access for each tool as you would for any internal integration.