Docs/Interfaces
Interfaces

Connecting Open WebUI to internal tools and APIs

By Samuel Seidel · Updated September 9, 2026 · 7 min read

By the end of this page, your Open WebUI install can call an internal API through a Python tool and answer questions from a knowledge collection of your own documents. It starts from a working install; the Open WebUI setup guide covers install, model connection, accounts and HTTPS.

You need:

On this page
  1. Function calling needs the right model
  2. Writing a tool
  3. Enabling a tool on a model
  4. RAG over internal documents
  5. Knowledge collections vs per-chat files
  6. What actually breaks

Function calling needs the right model

Open WebUI's tools give the model a description of your Python functions and let it decide when to call one, the same function calling agent frameworks use. That depends on the model as well as on Open WebUI. A model not trained to emit structured tool calls either ignores the tools or produces a call in the wrong format.

  1. Confirm the model you serve on your Spark supports function calling.
  2. If you run vLLM yourself, start it with tool calling enabled for that model (--enable-auto-tool-choice plus the --tool-call-parser for its chat template). The GPUwerk llama.cpp presets start llama-server with --jinja, which enables tool calls; the Qwen3 Coder 30B vLLM preset does not start with --enable-auto-tool-choice, so Open WebUI's native function-calling mode does not work against it. Open WebUI's default mode, which selects tools through a prompt, does not need engine support.
  3. Test with a trivial tool before you write anything larger. You should see a tool-call indicator in the chat.

Writing a tool

A tool in Open WebUI is a Python file with a class whose public methods the model can call. Add one from Workspace, Tools, New Tool and save it; it should then appear in the tools list. This example looks up an internal ticket by ID:

"""
title: Ticket Lookup
description: Look up a support ticket by ID from the internal API.
"""
import requests

class Tools:
    def __init__(self):
        pass

    def get_ticket(self, ticket_id: str) -> str:
        """
        Look up a support ticket by its ID.
        :param ticket_id: The ticket ID, e.g. TICK-1234
        """
        resp = requests.get(
            f"http://internal-api.local/tickets/{ticket_id}",
            timeout=5,
        )
        resp.raise_for_status()
        return resp.text

The docstring, type hint and parameter comment matter: Open WebUI passes them to the model as the tool's description. A vague description is why a model skips a tool or calls it with the wrong argument. The tool runs as Python inside the Open WebUI backend, so it can reach anything the Open WebUI host can reach: an internal REST API, a database, a wiki search endpoint.

Enabling a tool on a model

  1. Go to Workspace, Models and select the model.
  2. Enable the tool under its capabilities and save. To test a tool first, turn it on for one chat from the tool icon in the message input instead.
  3. Ask something that needs the tool, such as "What is the status of TICK-1234?". You should see a tool-call indicator and an answer built from the API response.

The model decides per message whether to call the tool; it does not run on every message.

RAG over internal documents

The setup guide's document section covers uploading a file to one chat. This section applies the same pipeline to a whole body of internal documents: a wiki export, a folder of PDFs, a knowledge base dump.

The mechanics are the same: documents are chunked, embedded with a local embedding model and stored in a local vector store, and relevant chunks are added to the prompt at query time. None of that leaves the host running Open WebUI. At knowledge-base scale, revisit two defaults that are tuned for single documents. A stronger multilingual or domain-tuned embedding model in Admin Settings, Documents usually helps most, and chunk size matters more once documents are long and varied.

Knowledge collections vs per-chat files

For internal documents, use a knowledge collection rather than per-chat uploads.

  1. Create a collection in Workspace, Knowledge.
  2. Upload the document set into it and wait for processing to finish.
  3. In any chat, type #collection-name and ask a question. The answer should cite the documents it used.

Every conversation and every user with access can then use the corpus without uploading it again. Access follows Open WebUI's user and group permissions, so a collection built from an HR policy folder can be limited to the people who should see it.

What actually breaks

See alsoSet up Open WebUI on your Spark See alsovLLM on a DGX Spark
First top-up: pay $20, get $30 in credit

A chat interface that can act on your own systems.

Deploy a dedicated Spark in Europe, North America or East Asia and run Open WebUI, tools, and your documents on hardware you control.

Create account