Private LLM hosting vs Baseten
We rent DGX Sparks, so weigh that against everything below. Baseten is a model-serving platform: you deploy a model and Baseten handles the GPU provisioning, autoscaling, and endpoint underneath it, a genuinely good fit for teams that want to ship a custom or fine-tuned model without operating infrastructure. A Spark hands you the opposite trade: the machine itself, root access, and none of the deployment tooling, in exchange for a flat hourly rate and full control over what runs on it.
Side by side
| Baseten | Dedicated DGX Spark (GPUwerk) | |
|---|---|---|
| What it is | A model deployment and serving platform: push a model, Baseten provisions GPU capacity and autoscales it behind an API endpoint, per Baseten's documentation. | One dedicated physical machine, rented by the hour, with root access. No deployment platform, no autoscaling layer. |
| Where data is processed | Whichever infrastructure region Baseten provisions your deployment to; check Baseten's own documentation for current region options. | EU-Central (Prague), on one dedicated machine, no other region to configure. |
| Who can see your data | Governed by Baseten's own terms of service and data processing agreement; check Baseten's current legal pages for the specifics that apply to your account and deployment. | Nobody at GPUwerk. Our DPA states GPUwerk "hosts the machine but does not access, read, copy, index, or analyse the content of the controller's instance." |
| Model choice | Open-weight models, custom fine-tunes, and models packaged in Baseten's Truss deployment format, per Baseten's documentation. Broad support for custom and unusual model architectures. | Open-weight models only: gpt-oss-120b, Llama 3.3 70B, Qwen3-Coder 30B, DeepSeek, and anything else you can fit in 128 GB, deployed however you choose since you have root access. |
| Speed and concurrency | Depends on the GPU type Baseten provisions for your deployment and its autoscaling configuration; Baseten does not publish a single tokens/second figure that applies across models and deployments. | A single Spark, single-stream: gpt-oss-120b decodes at 33.5 tok/s and reaches 862.8 tok/s aggregate at 256 concurrent requests (Dendro Logic's concurrency benchmark, run on their own Spark). Raw logs and methodology on our benchmarks page. |
| Pricing model | Usage-based, tied to the compute resources your deployment consumes while serving traffic, with the ability to scale down between requests. See Baseten's own pricing page for current rates. | Per hour, billed per minute: $0.79/hour on-demand, $0.59/hour for a customer-requested stop that holds your reservation. One rate regardless of traffic, per pricing. |
| Worked cost example | See the full arithmetic below the table. | |
| Contracts and DPA | Governed by Baseten's own terms of service and data processing agreement. | A standard GDPR Article 28 DPA published free at /legal/dpa, no negotiation required. Sub-processor list at /legal/sub-processors states none are engaged for instance workloads. |
| What you operate yourself | Your model code and Truss configuration; Baseten manages GPU provisioning, autoscaling, and the serving infrastructure underneath. | Everything, root SSH access to your own container: the model server, any RAG or agent layer, monitoring, and backups. GPUwerk keeps only a recovery copy of /workspace for hardware failures, refreshed roughly every six hours; it is not a backup service, so that responsibility is entirely yours. |
The worked cost example
Take a workload that generates 500 million output tokens a month, roughly what a busy internal support bot or a document-processing pipeline running continuously might produce. Two ways to serve it:
Baseten. Pricing is usage-based, tied to the GPU resources your deployment actually consumes while serving traffic, per Baseten's own pricing page. That figure depends on the model, the GPU type Baseten provisions for it, and how much the deployment scales up under your traffic pattern, so GPUwerk did not attempt to reconstruct a single number here. Run your model and expected volume through Baseten's pricing page or calculator for a figure specific to your deployment.
Dedicated Spark. gpt-oss-120b reaches 862.8 tok/s aggregate at 256 concurrent requests, per Dendro Logic's concurrency benchmark on a single Spark (cited on our benchmarks page). That is 862.8 × 3,600 = 3,106,080 tokens per hour if the node stays saturated at that concurrency the whole time. 500,000,000 ÷ 3,106,080 ≈ 161 hours. At $0.79/hour, 161 × $0.79 = $127.19, before tax.
That Spark figure only holds under the assumption stated: the node is busy at 256 concurrent requests for the full 161 hours, back to back, with no idle time. This is exactly the scenario where Baseten's usage-based model can do better: a workload that autoscales down to near-zero between bursts of traffic pays close to nothing while quiet, where a Spark keeps billing at $0.79/hour regardless. A workload that's steadily busy for most of the day tends to favor the flat rate, since a continuously scaled-up Baseten deployment is paying for GPU time too, just metered differently. The honest comparison is your own traffic shape against both pricing models, not the sticker rate alone.
Migration path
Baseten deployments commonly expose an OpenAI-compatible endpoint for chat-style models, per Baseten's documentation, and a Spark serving a model through vLLM exposes the same shape, so client code that only calls that endpoint moves with a base URL and API key change. What doesn't move is Baseten's autoscaling behavior, its Truss packaging format, and deployment versioning, none of that has an equivalent on a Spark, you'd be trading managed scaling for a fixed machine you scale yourself, if at all.
When Baseten is the right choice
- Your traffic is bursty or unpredictable, and paying only for GPU time actually consumed beats a flat hourly rate that runs whether or not requests are coming in.
- You want Baseten managing GPU provisioning and autoscaling so your team can focus on the model rather than the serving infrastructure.
- You're deploying an unusual or custom model architecture and want the deployment tooling built for that, rather than configuring a serving stack from scratch.
When a dedicated Spark is the right choice
- Your traffic is steady enough to keep a fixed machine busy most of the hours you're paying for.
- You want full root access to the hardware, to run exactly the serving stack, monitoring, and network configuration you choose, not what a platform's deployment format allows.
- You want the shortest possible answer to "who can access our data": one dedicated machine, root access under your own SSH keys, and an operator who states in writing that it does not read your instance's content.
FAQ
What does Baseten actually manage for me?
Baseten is a model deployment platform: you push a model (open-weight, fine-tuned, or custom) and Baseten handles the serving infrastructure, autoscaling, and GPU provisioning behind an API endpoint, per Baseten's own documentation. You don't pick or operate a specific machine, Baseten does that underneath the deployment. A Spark is the opposite: you get the machine directly, root access included, and build the serving layer yourself.
Can I run the same open-weight models on both?
Largely yes. Baseten supports deploying open-weight models including custom fine-tunes through its model library and custom deployment workflow, per Baseten's documentation. A Spark runs the same category of model, gpt-oss-120b, Llama 3.3, Qwen3-Coder, DeepSeek, directly on hardware you control rather than behind Baseten's managed serving layer.
Does Baseten offer EU hosting?
Check Baseten's own documentation for current region and deployment options, this changes as providers add capacity, so state it precisely rather than assume. GPUwerk's Spark fleet runs exclusively from EU-Central (Prague), which is the whole of its regional footprint, not one option among several.
Is a DGX Spark cheaper than Baseten?
It depends on your traffic pattern. In the worked example on this page, 500 million generated tokens a month costs about $127 in Spark compute at $0.79/hour, assuming the node sustains 256 concurrent requests the entire time. Baseten prices usage-based, by the compute resources a deployment consumes while serving traffic, published on Baseten's own pricing page; a bursty workload that autoscales down between requests can look very different in cost from one that keeps a Spark busy around the clock.
What do I give up moving from Baseten to a Spark?
Autoscaling, Baseten's deployment tooling and model versioning, and the ability to scale a deployment down to near-zero cost during quiet periods. A Spark is billed by the hour whether it's serving requests or not, so you take on both the operational work of running the serving stack yourself and the cost of idle capacity, in exchange for full control of the hardware and a flat, predictable rate.
GPUwerk did not find a single flat per-token or per-hour Baseten price that holds across models and GPU types; Baseten prices usage-based deployments on its own pricing page, so that row above reflects that structure rather than a guess. Spark throughput figures come from Dendro Logic's concurrency benchmark, detailed on our benchmarks page.