Private AI/vs OpenAI API
Comparison

Private LLM hosting vs the OpenAI API

By Samuel Seidel · Published September 9, 2026 · 10 min read

I run GPUwerk, which rents DGX Sparks, so weigh everything below with that in mind. If your task needs a specific OpenAI model by name, a Spark cannot run it: a Spark runs open-weight models only, and that is the shape of the product. If an open model such as gpt-oss-120b meets your quality bar and your question is where the data goes, a dedicated Spark in Hradec Králové gives you one machine assigned to you alone instead of a shared API. OpenAI now offers European data residency too, so the difference is tenancy, operator and pricing model more than location.

Side by side

OpenAI APIDedicated DGX Spark (GPUwerk)
Where data is processed OpenAI's infrastructure. Regional processing is available in Europe (EEA and Switzerland), the US and the UAE; models released on or after 5 March 2026 cost 10% more on those endpoints. OpenAI is a US company. EU-Central (Hradec Králové) on GPUwerk's own tier, on one dedicated machine. No shared endpoint, no other tenant, no surcharge for the location. (advantage)
Who can see your prompts OpenAI's data controls guide says API data is not used to train its models unless you opt in, and abuse-monitoring logs are kept for up to 30 days by default. Zero Data Retention needs OpenAI's prior approval. No other customer, and GPUwerk only under the DPA. GPUwerk's DPA states GPUwerk "hosts the machine but does not access, read, copy, index, or analyse the content of the controller's instance." There is no abuse-monitoring log, because GPUwerk runs infrastructure, not a model API. (advantage)
Model choice OpenAI's closed models (the GPT-6 family on its pricing page in October 2026), plus reasoning, image and audio endpoints and the Responses API's built-in tools. (advantage) Open-weight models only: gpt-oss-120b, Llama 3.3 70B, Qwen3-Coder 30B, DeepSeek, and anything else that fits in 128 GB. OpenAI's closed models are not available on a Spark under any arrangement.
Speed and concurrency OpenAI publishes no tokens/second figure; usage limits are tiers of tokens and requests per minute. gpt-oss-120b: 33.5 tok/s single-stream and 862.8 tok/s aggregate at 256 concurrent requests (Dendro Logic's benchmark on its own Spark). Qwen3-Coder 30B-A3B AWQ: 80.9 tok/s single-stream on GPUwerk's own fleet. Methodology on GPUwerk's benchmarks page.
Pricing model Per token, input and output priced separately, by model. On OpenAI's pricing page (checked 9 November 2026): gpt-6.1-sol $2 input and $10 output per million tokens; gpt-6-luna $0.10 and $0.50. Per hour, billed per minute: $0.79/hour on-demand, $0.59/hour while stopped and held for you. One rate regardless of model or how many requests you send. USD, tax extra, per pricing. (advantage)
Worked cost example See the arithmetic below the table. See the arithmetic below the table.
Contracts and DPA OpenAI's business terms and data processing addendum for API customers. Enterprise agreements at volume. A standard GDPR Article 28 DPA published free at /legal/dpa, no negotiation required. The sub-processor list names none for instance workloads on GPUwerk's own tiers. (advantage)
Lock-in and portability The chat completions API is widely copied, so basic client code ports to other OpenAI-compatible endpoints, a Spark's included, with a base URL and key change. Prompts tuned to one closed model, and platform features such as the Responses API's built-in tools, do not move with you. Open weights mean you can move the model file to another Spark, another GPU machine or your own hardware. You give up OpenAI's managed platform and build what you need yourself. (advantage)
What you operate yourself Nothing at the infrastructure layer. OpenAI handles capacity, scaling and serving; you manage prompts, code and rate limits. (advantage) Root access, from the browser terminal or over SSH with your own key, to your own container: the model server, any RAG or agent layer, monitoring and backups. GPUwerk keeps no copy of /workspace, not even for a failed drive, so backups are yours.

The worked cost example

Take a workload that generates 500 million output tokens a month, about what a busy internal support bot or a document pipeline running all day might produce.

OpenAI API. At the prices on OpenAI's pricing page on 9 November 2026, 500 million output tokens cost $5,000 on gpt-6.1-sol ($10 per million) or $250 on gpt-6-luna ($0.50 per million), plus input tokens, plus 10% on a European regional processing endpoint.

Dedicated Spark. gpt-oss-120b reaches 862.8 tok/s aggregate at 256 concurrent requests, per Dendro Logic's concurrency benchmark on a single Spark (cited on GPUwerk's benchmarks page). That is 862.8 × 3,600 = 3,106,080 tokens per hour if the node stays saturated at that concurrency the whole time. 500,000,000 ÷ 3,106,080 ≈ 161 hours. At $0.79/hour, 161 × $0.79 = $127.19, before tax, input tokens included.

The Spark figure holds only if the node is busy at 256 concurrent requests for all 161 hours. Real traffic comes in bursts, and a half-idle Spark still costs $0.79 for every hour it runs. OpenAI's per-token price follows actual usage. It also buys a closed model that may answer better than gpt-oss-120b on your task, so compare quality on your own evaluation before you compare price.

Migration path

Both sides speak the same API at the client level. A Spark serving a model through vLLM exposes the OpenAI chat completions API, so an application that only calls that endpoint moves with a base URL and API key change. Put LiteLLM in front and you get one OpenAI-compatible gateway with request logging, key management and rate limits, which lets you route some tasks to OpenAI and others to an open model on a Spark. What does not move is prompt tuning for one closed model and OpenAI platform features such as the Responses API's built-in tools. OpenAI shut down the older Assistants API on 26 August 2026, so code still using it needs migrating either way.

When the OpenAI API is the right choice

When a dedicated Spark is the right choice

FAQ

Can I run OpenAI's closed models on a DGX Spark?

No. OpenAI's closed models run only on OpenAI's infrastructure, not on hardware you rent or own. A Spark runs open-weight models: gpt-oss-120b (OpenAI's own open-weight model), Llama 3.3, Qwen3-Coder, DeepSeek and similar. If your task needs a specific closed OpenAI model, the OpenAI API is the place to get it.

Does OpenAI train on my API data?

Not by default. OpenAI's data controls guide says data sent to the API has not been used to train its models since 1 March 2023 unless you opt in. Abuse-monitoring logs are kept for up to 30 days by default, and Zero Data Retention needs OpenAI's approval. On a Spark, GPUwerk's DPA states GPUwerk hosts the machine but does not access, read, copy, index, or analyse the content of your instance.

Where does the OpenAI API process my data?

By default on OpenAI's infrastructure, operated by a US company. OpenAI offers regional processing in Europe (EEA and Switzerland), with a 10% surcharge on models released on or after 5 March 2026, and some options need OpenAI's approval. A dedicated Spark on GPUwerk's own tier runs in one place, Hradec Králové, on a single machine assigned to you alone.

Is a DGX Spark cheaper than the OpenAI API?

It depends on the OpenAI model, your volume and how busy you keep the Spark. In the worked example on this page, 500 million generated tokens a month cost about $127 in Spark compute at $0.79/hour, assuming 256 concurrent requests the whole time, against $250 on gpt-6-luna or $5,000 on gpt-6.1-sol in output tokens alone. A lightly loaded Spark, billed for every running minute, can cost more per token than a per-token API.

Can I switch between the OpenAI API and a Spark without rewriting my application?

Mostly. A Spark serving a model through vLLM exposes an OpenAI-compatible chat completions API, so client code that only calls that endpoint ports with a base URL and API key change. What does not port is tuning for one closed model's behaviour, and OpenAI platform features such as the Responses API's built-in tools, which you would build yourself.

How does this compare to Azure OpenAI or Anthropic's API?

The core trade, closed models on someone else's shared infrastructure against open-weight models on a dedicated machine, is the same across OpenAI, Azure OpenAI and Anthropic. The separate comparisons against Azure OpenAI and the Anthropic API cover each vendor's data location and prompt handling.

OpenAI claims checked on 9 November 2026 against OpenAI's API pricing page (model prices, 10% regional processing uplift), its data controls guide (training, 30-day abuse monitoring, Zero Data Retention, data residency regions) and its deprecations page (Assistants API shut down on 26 August 2026). Prices change with each model release, so check them for your own model and volume. OpenAI's data-usage statements are its own policy, not something GPUwerk can verify.

Related pages

First top-up: pay $20, get $30 in credit

Run your own numbers before you commit either way.

Deploy a dedicated DGX Spark and test your actual prompts against an open model before comparing quotes.

Deploy now Read the benchmarks