What is function calling in LLMs?
Function calling is a capability where a language model, given a list of available functions and their expected arguments, can output a structured request to call one of them, rather than only producing free-form text. The model itself doesn't execute anything; it decides when a function is needed and formats the call correctly, and the surrounding application runs the function and returns the result for the model to use in its next response.
How the exchange works
A typical function-calling exchange has a few steps. The application sends the model a prompt along with a schema describing available functions, usually their names, descriptions, and expected parameters in a format like JSON Schema. If the model decides a function is needed to answer the request, it returns a structured object naming the function and the arguments to pass, instead of a plain-text answer. The application executes that function outside the model, then sends the result back as part of the conversation so the model can incorporate it into a final response, or decide to call another function.
What it's used for
Function calling turns a model from something that only generates text into something that can act on live data or systems: looking up a customer record, checking current pricing, running a calculation the model would otherwise get wrong, or writing to a database. It's the mechanism behind most tool use in current chat assistants and coding tools, and it's the foundation that agent loops are built on, where a model repeatedly calls functions and reacts to their results until a task is done.
Reliability is model-dependent
Not every model handles function calling equally well. Some models were explicitly trained with tool-calling examples and produce well-formed calls consistently; others attempt it but produce malformed arguments, hallucinate function names that weren't offered, or call a function when a plain-text answer would have been correct. If tool use matters for your application, test the specific model against your actual function schema rather than assuming general capability claims apply. See evaluating an LLM before deploying it for a structured way to run that check.
How it relates to MCP
Model Context Protocol builds on function calling rather than replacing it. Function calling is the model-level behavior of producing a structured call; MCP standardizes how an application discovers what functions are available across different tools and servers, so the same client can work with many tool sources without custom code for each one.
Running function-calling workloads privately
Because function calling often means passing internal data, customer records, pricing, ticket details, into the prompt and back out through tool results, teams handling sensitive data frequently prefer to keep that traffic off a shared external API. Running the model on a dedicated node keeps the full request, including whatever data flows through function arguments and results, inside infrastructure you control. See what is an OpenAI-compatible API for how self-hosted models expose the same tool-calling interface most client libraries already expect.