What is an LLM agent?
An LLM agent is a language model wrapped in a loop: it calls external tools, reads what they return, decides its next action from the result, and repeats until the task is done or it gives up. A plain chat completion takes one input and returns one output, with no chance to act in between.
Three things that make a system "agentic"
The term gets used loosely, but three properties usually define it:
- Tool use. The model can call functions (run code, query a database, hit an API) instead of only generating text.
- Multi-step reasoning. The system loops, feeding each tool result back to the model so it can plan its next move, instead of stopping after one generation.
- Autonomy. Within the boundaries it's given, the system decides which tools to call and when to stop, instead of following a fixed, pre-written sequence.
A system with none of these is not an agent, even if it uses an LLM somewhere.
Chat completion vs agent, concretely
A chat completion call sends a prompt and gets back a response: one input, one output, done. An agent built on the same underlying model might receive the same starting instruction, but then call a search tool, read the results, decide it needs to also check a file, read that, and only then produce a final answer, three or more separate model calls chained together with tool calls in between. The model can be identical in both cases; the difference is the scaffolding around it.
Where this matters for deployment
An agent loop calls the model several times per task, many times for a complex job, so cost and latency scale with the number of steps, not the length of one response. Inference speed and cost per call therefore matter more for agents than for one-shot chat: a slow or expensive model is multiplied across every step of the loop.
Agents vs RAG
Agents are often confused with retrieval-augmented generation (RAG), partly because the two are often combined. RAG is about injecting retrieved context into a single generation call; an agent is about a model taking multi-step, tool-using action, which may or may not include retrieval as one of its tools. For the full comparison, including when to use one, the other, or both together, see RAG vs agents: what's the actual difference.
What "autonomy" means in practice
Autonomy in an agent is bounded, not absolute. A developer defines the tools the agent can call, sets limits on how many steps it can take, and often reviews or gates certain actions before they execute. Within those bounds, the model decides the sequence: which tool to call, with what arguments, and when the task looks complete. That's closer to "the model picks its own path through a defined toolset" than to "the model acts without oversight."
Common failure modes
Agent loops can go wrong in ways a single chat completion can't. A model might call a tool with bad arguments, misread a tool's output, or loop on the same failing action without recognizing it isn't making progress. Because errors compound across steps, most production agent systems cap the number of steps, validate tool outputs before feeding them back in, and log each step so a failure can be traced to the call that caused it. Don't count on the loop to correct itself.