Blog/What is an LLM agent?
Definition

What is an LLM agent?

By Samuel Seidel · Published September 9, 2026

An LLM agent is a language model wrapped in a loop: it calls external tools, reads what they return, decides its next action from the result, and repeats until the task is done or it gives up. A plain chat completion takes one input and returns one output, with no chance to act in between.

Three things that make a system "agentic"

The term gets used loosely, but three properties usually define it:

A system with none of these is not an agent, even if it uses an LLM somewhere.

Chat completion vs agent, concretely

A chat completion call sends a prompt and gets back a response: one input, one output, done. An agent built on the same underlying model might receive the same starting instruction, but then call a search tool, read the results, decide it needs to also check a file, read that, and only then produce a final answer, three or more separate model calls chained together with tool calls in between. The model can be identical in both cases; the difference is the scaffolding around it.

Where this matters for deployment

An agent loop calls the model several times per task, many times for a complex job, so cost and latency scale with the number of steps, not the length of one response. Inference speed and cost per call therefore matter more for agents than for one-shot chat: a slow or expensive model is multiplied across every step of the loop.

Agents vs RAG

Agents are often confused with retrieval-augmented generation (RAG), partly because the two are often combined. RAG is about injecting retrieved context into a single generation call; an agent is about a model taking multi-step, tool-using action, which may or may not include retrieval as one of its tools. For the full comparison, including when to use one, the other, or both together, see RAG vs agents: what's the actual difference.

What "autonomy" means in practice

Autonomy in an agent is bounded, not absolute. A developer defines the tools the agent can call, sets limits on how many steps it can take, and often reviews or gates certain actions before they execute. Within those bounds, the model decides the sequence: which tool to call, with what arguments, and when the task looks complete. That's closer to "the model picks its own path through a defined toolset" than to "the model acts without oversight."

Common failure modes

Agent loops can go wrong in ways a single chat completion can't. A model might call a tool with bad arguments, misread a tool's output, or loop on the same failing action without recognizing it isn't making progress. Because errors compound across steps, most production agent systems cap the number of steps, validate tool outputs before feeding them back in, and log each step so a failure can be traced to the call that caused it. Don't count on the loop to correct itself.

Related pages

First top-up: pay $20, get $30 in credit

Run agent loops without paying per call.

A dedicated Spark bills by the minute, $0.79/hour, however many model calls each task makes.

Create account