Private LLM hosting vs Together AI
I run GPUwerk, which rents DGX Sparks, so weigh everything below with that in mind. Together AI makes open models easy to call: a large catalog behind an OpenAI-compatible API, fine-tuning and batch tooling, and no GPU to manage. Its serverless API runs on shared capacity, with a US-only option for some models and none for the EU; dedicated endpoints cost extra. A dedicated Spark is the other shape: one machine, one tenant, in Hradec Králové, and you do the setup yourself.
Side by side
| Together AI | Dedicated DGX Spark (GPUwerk) | |
|---|---|---|
| Tenancy | Shared GPU capacity for the serverless API. Dedicated endpoints are a separate tier, billed per GPU-hour. | Single-tenant by default: one physical Spark, and nobody else's workload runs on it while you rent it. (advantage) |
| Where data is processed | Together's docs offer US-only processing for select serverless models, and no EU serverless option. For residency they point to dedicated endpoints, private networking or VPC deployments, which can include EU regions. | EU-Central (Hradec Králové) on GPUwerk's own tier. No region to choose. (advantage) |
| Who can see your prompts | Together, as the API operator. By default it stores prompts and responses for product improvement; an organisation admin can turn that off for zero data retention. Sharing data for training is opt-in. | No other customer, and GPUwerk only under the DPA. GPUwerk's DPA states GPUwerk "hosts the machine but does not access, read, copy, index, or analyse the content of the controller's instance." (advantage) |
| Model choice | A large, actively maintained catalog of open-weight models, callable at once with no download or setup. (advantage) | Any open-weight model that fits in 128 GB unified memory (gpt-oss-120b, Llama 3.3 70B, Qwen3-Coder 30B, DeepSeek and others), but you download, configure and serve it yourself. |
| Speed and concurrency | Depends on the model and on load. Measure it on your own prompts. | gpt-oss-120b: 33.5 tok/s single-stream and 862.8 tok/s aggregate at 256 concurrent requests (Dendro Logic's benchmark on its own Spark). Qwen3-Coder 30B-A3B AWQ: 80.9 tok/s single-stream (GPUwerk's measurement). Methodology on GPUwerk's benchmarks page. |
| Pricing model | Per token on serverless, by model. On together.ai/pricing (checked 9 November 2026): gpt-oss-120B $0.15 input and $0.60 output per million tokens. Llama 3.3 70B is no longer on the serverless API. Dedicated endpoints: H100 $5.49 and B200 $8.99 per GPU-hour. (advantage) | Per hour, billed per minute: $0.79/hour on-demand, $0.59/hour while stopped and held for you. One rate regardless of model or request volume, per pricing. (advantage) |
| What you operate yourself | Nothing at the infrastructure layer on serverless. You manage prompts and application code. (advantage) | Root access, from the browser terminal or over SSH with your own key, to your own container: the model server, any RAG or agent layer, monitoring and backups. GPUwerk keeps no copy of /workspace, not even for a failed drive, so backups are yours. |
Where the honest difference actually is
Together's catalog is the real draw. You can call dozens of open models the moment you have an API key, with no GPU procurement, driver setup or capacity planning. To try five open models in an afternoon, that is hard to beat, and a Spark needs real setup time per model.
The trade is tenancy and location. Serverless requests run on capacity Together shares across customers, and the only region choice is US-only processing for some models. Together's dedicated endpoints isolate hardware and can include EU regions, which narrows the gap, at $5.49 per H100-hour against a Spark's $0.79. On a Spark, isolation is not an upgrade. It is the only mode: one machine and GPUwerk's written policy of not accessing instance content.
Whether that matters depends on the workload. A prototype with public data probably does not need it. A workload under a contract that says where processing happens is where the difference stops being abstract. On GPUwerk, choosing a location that meets such a contract is your call, and GPUwerk processes the data under its published DPA.
The worked cost example
Take a workload generating 500 million output tokens a month on gpt-oss-120b.
Together AI. 500 million output tokens at $0.60 per million = $300, plus input tokens at $0.15 per million, per Together's pricing page on 9 November 2026.
Dedicated Spark. gpt-oss-120b reaches 862.8 tok/s aggregate at 256 concurrent requests in Dendro Logic's benchmark, cited on GPUwerk's benchmarks page. That's 862.8 × 3,600 = 3,106,080 tokens per hour if the node stays saturated at that concurrency. 500,000,000 ÷ 3,106,080 ≈ 161 hours. At $0.79/hour, 161 × $0.79 = $127.19, before tax, with input tokens included.
The Spark figure assumes 256 requests in flight for all 161 hours, at about 3.6 tok/s per request. Real traffic rarely looks like that. Together's per-token pricing follows actual usage with no commitment, so at low or bursty volume, or when users need fast single responses, Together can cost less even before your setup time. The Spark wins as steady volume climbs and the machine stays busy.
Migration path
Together's API and a Spark serving a model through vLLM both expose an OpenAI-compatible chat completions API, so client code moves with a base URL and API key change. What does not move by itself is tuning done against a particular Together-hosted checkpoint. If the Spark uses a different quantization or serving stack, check output quality before you switch.
When Together AI is the right choice
- You want to try or run many open models without setting up infrastructure for each.
- Your volume is variable enough that per-token pricing beats paying for a machine around the clock.
- You need fast single responses, or you can pay for a dedicated endpoint in an EU region when isolation is required.
When a dedicated Spark is the right choice
- Single-tenant hardware is a requirement, and you want it as the default, not a paid upgrade.
- You have settled on one or two models and do not need a large ready-made catalog.
- Your traffic is steady enough to keep the machine busy for most of the hours you pay for.
FAQ
Is Together AI multi-tenant?
Its serverless API serves requests from GPU capacity shared across customers, the usual shape for a hosted inference API. Together also sells dedicated endpoints, priced per GPU-hour (H100 at $5.49 on its pricing page, checked 9 November 2026).
Does Together AI train on my API data?
Not by default. Together's privacy documentation says sharing data for training other models is opt-in. It does store prompts and responses by default for product improvement; an organisation admin can switch that off for zero data retention.
Is a Spark more private than Together AI's shared API?
On tenancy, yes, compared with the serverless tier: a Spark is one dedicated machine, and GPUwerk's DPA states GPUwerk does not access, read, copy, index, or analyse instance content. Together's dedicated endpoints narrow that gap because they isolate hardware per customer, so compare the two directly if you are considering that tier.
Does Together AI have more models than GPUwerk?
Yes. Together hosts a large, actively updated catalog of open-weight models you can call with no setup. On a Spark you can run any open-weight model that fits in 128 GB unified memory, but you download, configure and serve it yourself.
Together AI claims checked on 9 November 2026 against together.ai/pricing (serverless and dedicated endpoint prices) and Together's privacy and security docs (training opt-in, prompt storage, region selection). Prices change; check them before you decide. Spark figures come from GPUwerk's benchmarks page and GPUwerk's pricing page.