Blog/Self-hosted AI agent infrastructure
AI agents

Self-hosted AI agent infrastructure

By Samuel Seidel · Updated September 9, 2026

Agents are cheaper to budget on your own endpoint because you pay for machine time, not for every call the loop makes. A chat request costs one API call. An agent that reads a file, calls a tool, checks the result and repeats might make thirty or a hundred calls to finish one task, and on a per-token API every one of them lands on your bill.

Why agent loops break per-token budgeting

A single chat message has a predictable shape: one prompt in, one response out. An agent doesn't work that way. It plans a step, calls a tool, reads the tool's output back as new context, decides the next step, and repeats until the task is done or it gives up. Each round trip is a full model call, and a task that looks simple from the outside ("refactor this module and update its tests") can take dozens of iterations if the agent has to re-read files, retry a failed edit, or reason through an error message. On a per-token API the same nominal task can cost very different amounts depending on how many iterations it needed, and you don't know that in advance.

Running the model on a dedicated machine changes the shape of the bill, not the shape of the loop. A Spark at $0.79 an hour, metered per minute, charges for the minutes it runs, not for the tokens the agent burns. The agent can retry, re-read context and take the scenic route without each extra call adding to the invoice. For a team running agents continuously, that is the difference between a budget you can plan around and one that depends on how well-behaved the agent was that day.

The data-control side

An agent with tool access sees more than a chatbot ever does: your file server, your inbox, your internal APIs, sometimes credentials to act on your behalf. Every tool output goes back to the model as context on the next iteration, so an agent on a cloud model API routes your inbox and file contents through that provider's inference servers on every step of every task, not just the first prompt. Serving the model yourself keeps that context away from a model provider at every point in the loop, which matters more for agents than for chat because agents see more.

What the pieces look like on a Spark

vLLM is the inference engine that serves the model itself; it's what turns a GB10 Spark's 128 GB of unified memory into an OpenAI-compatible endpoint an agent framework can call. In front of that, LiteLLM gives you one stable URL, per-user virtual keys, and an optional fallback to a cloud model if the machine is busy (which sends those requests to the cloud provider), useful once more than one agent or team calls the same endpoint. Coding agents against your own endpoint covers wiring an agent framework, CLI tool or editor integration into that endpoint. None of the three pieces is agent-specific; it's the same stack as any self-hosted inference. An agent is a client that calls your endpoint more often, and for longer stretches, than a person typing in a chat window.

Where a cloud API still makes sense

If your agent workload is occasional, a few tasks a week rather than a continuous loop, the hourly cost of a dedicated machine may not pay for itself against a pay-as-you-go API. The crossover point depends on how many hours a month the agent actually runs and how sensitive the data it touches is; a team running agents against internal codebases or customer data for hours a day is a different case from someone running an occasional one-off script.

Related pages

First top-up: pay $20, get $30 in credit

Give your agents a flat-rate endpoint.

vLLM and LiteLLM on a dedicated Spark, billed for machine time, not for each call the loop makes.

A dedicated DGX Spark in Europe, North America or East Asia, $0.79/hour, billed per minute. $30 of credit is about 37 hours of compute.

Create account See the agent setup