Connecting Open WebUI to internal tools and APIs

The Open WebUI setup guide covers install, connecting it to a model, accounts and HTTPS. This page picks up from a working install and covers the part that turns a chat interface into something that can act: function calling through tools, and answering questions against your own internal documents through RAG.

Function calling needs the right model

Open WebUI's tools feature works by giving the model a description of available Python functions and letting it decide when to call one, the same function-calling mechanism used by agent frameworks generally. This depends on the model, not just on Open WebUI: a model that wasn't trained to emit structured tool calls will either ignore the tools entirely or hallucinate a plausible-looking call that doesn't match the expected format. Before building a tool, confirm the model you're serving on your Spark supports function calling, and that vLLM is started with tool-calling support enabled for that model's chat template if the model requires it. A quick manual test, ask the model to use a trivial tool and check the response, is worth doing before writing anything more complex.

Writing a tool

A tool in Open WebUI is a Python file with a class exposing one or more methods; each public method becomes something the model can call. Add one from Workspace, Tools, New Tool. A minimal example that looks up an internal ticket by ID:

"""
title: Ticket Lookup
description: Look up a support ticket by ID from the internal API.
"""
import requests

class Tools:
    def __init__(self):
        pass

    def get_ticket(self, ticket_id: str) -> str:
        """
        Look up a support ticket by its ID.
        :param ticket_id: The ticket ID, e.g. TICK-1234
        """
        resp = requests.get(
            f"http://internal-api.local/tickets/{ticket_id}",
            timeout=5,
        )
        resp.raise_for_status()
        return resp.text

The docstring and the parameter's type hint and comment matter: Open WebUI passes them to the model as the tool's description, and a vague description is why a model skips a tool it should have used, or calls it with the wrong argument. Because this runs as Python inside the Open WebUI backend, a tool can reach anything on your internal network the Open WebUI host can reach, an internal REST API, a database, an internal wiki search endpoint. That's the actual capability this feature buys you over a plain chat interface.

Enabling a tool on a model

Writing a tool doesn't attach it to anything by itself. Go to Workspace, Models, select the model, and enable the tool under its capabilities. Tools can also be turned on per chat from the message input's tool icon, useful for testing one before making it a model default. Once enabled, the model decides on its own, based on the user's message, whether a given turn calls for the tool; it isn't invoked automatically on every message.

RAG over internal documents

Open WebUI's RAG behavior is documented in the base setup guide's document section as a per-chat or per-collection upload. Where this page differs: pointing that same pipeline at a body of internal documents deliberately, a wiki export, a folder of internal PDFs, a knowledge base dump, rather than one file added to a single conversation.

The mechanics are the same either way: uploaded documents are chunked, embedded with a local embedding model, and stored in a local vector store, then relevant chunks are retrieved and injected into the prompt at query time. None of that leaves the host running Open WebUI. What changes at internal-knowledge-base scale is that the default embedding model and default chunking are tuned for casual single-document use, and both are worth revisiting: a stronger multilingual or domain-tuned embedding model in Admin Settings, Documents is usually the highest-leverage change, and chunk size matters more once documents are long and heterogeneous rather than one PDF at a time.

Knowledge collections vs per-chat files

For an internal-tools use case, a knowledge collection is almost always the right shape rather than per-chat uploads. Create one from Workspace, Knowledge, upload the document set into it, and reference the whole collection in any chat with #collection-name. That makes the corpus available across every conversation and every user with access, instead of re-uploading the same files per chat. Access to a collection follows Open WebUI's normal user and group permissions, so a collection built from an HR policy folder can be scoped to the people who should see it.

What actually breaks

• Tool calling silently fails on the wrong model. The failure mode isn't an error, it's the model answering from its own knowledge instead of calling the tool, which looks like a correct response until the answer is wrong or outdated. Verify tool use is actually happening, Open WebUI shows a tool-call indicator in the chat, before trusting the output.
• A tool with no error handling breaks the whole turn. An internal API timeout or a non-200 response inside a tool method that doesn't catch it will raise, and the model's response becomes an error rather than a graceful fallback. Handle failures inside the tool and return a message the model can work with.
• Retrieval quality depends on document structure more than model choice. A folder of scanned PDFs or tables-as-images will retrieve poorly no matter how good the underlying model is; extract or OCR that content before loading it into a collection.
• Tools run with the permissions of the Open WebUI host. A tool that hits an internal API which does not check credentials itself inherits whatever network access that host has. Scope credentials and network access for a tool the same way you would for any other internal service integration.