Open-weight vs closed-weight models: how to choose
Choose a closed API to prototype fast or to get the strongest frontier model. Choose a self-hosted open-weight model when the data must stay away from an AI vendor, when steady high volume makes a flat hourly rate cheaper than per-token billing, or when you need to fine-tune or quantize the model. Those three questions (data, cost at scale, customization) decide most projects, and the framework at the end puts them in order.
What "open-weight" means here
An open-weight model ships its trained parameters for anyone to download and run. You can host it yourself, on your own hardware or a rented GPU, and requests go only to the machine you run it on. Qwen, Llama, gpt-oss, and DeepSeek's open releases are all open-weight in this sense, and they vary a lot in size and capability, from models that fit on a laptop to ones that need a rack.
A closed-weight model is one you can only reach through an API the provider runs. You send a request over the network, their servers run it, and you get a response back. You never hold the weights, can't run the model anywhere else, and every request is subject to whatever data-handling and retention policy the provider has published (or hasn't).
When the API wins: convenience and frontier capability
A closed API is the right default when you're building something and don't yet know if it'll work. There's no hardware to provision, no engine to configure, and no capacity planning; you get an API key and start sending requests. If the project doesn't work out, you've spent only the tokens you used. For a prototype, a lightly used internal tool, or a product without a usage pattern yet, that flexibility beats anything self-hosting offers.
APIs also tend to front-run open weights on frontier capability. The labs that publish the strongest closed models generally don't open-weight their top tier, If your task needs the best available reasoning, the newest multimodal capability or the largest context window on the market, a closed API is often the only way to get it, at least for a while after release.
The trade is that every request goes through the AI provider's systems, under its retention and training-use policy, whatever that currently says. For many workloads that's fine. For anything involving customer data, regulated information, or material you don't want anywhere near another company's training pipeline, it's the reason to look elsewhere.
When self-hosted open-weight wins: control, cost at scale, customization
Data control is the clearest case. On hardware you operate yourself, there is no AI vendor's retention policy to trust and no question whether its terms changed since you last read them. For legal, healthcare or financial data that can't leave the building, an open-weight model on your own hardware is often the only option that meets the requirement at all.
Rented dedicated hardware sits in between. No AI vendor sees your prompts and no training-use policy applies, but the hosting provider operates the machine and keeps administrator access, so you still need its data processing agreement. On a rented GPUwerk Spark, that is GPUwerk's published DPA, which applies to every customer.
Cost is the second case, and it only favors self-hosting past a certain volume. An API has no fixed cost: you pay per token and nothing when you're idle. Self-hosting has a fixed hourly cost whether you use it or not, so at low or spiky volume the API is cheaper. Past a high enough steady volume the math flips: the hourly rate stays fixed while the per-token bill keeps climbing. Where that crossover sits depends on your token volume and the API pricing you compare against, so calculate it for your workload instead of assuming either direction.
Customization is the third case. Fine-tuning, quantizing to a format that fits your hardware, and changing the serving stack all need the weights, which a closed API never gives you. If you need a model tuned on your own data, or a latency or memory footprint only a custom quantization reaches, open-weight is the only path.
A short framework
Ask, in order:
- Must the data stay away from an AI vendor's API? If yes, self-host an open-weight model.
- Do you need to fine-tune, quantize or otherwise modify the model? If yes, same answer.
- Is your volume high and steady enough that a fixed hourly rate beats a per-token bill, and have you run that calculation? If yes, self-hosting saves money too.
If none apply, and you mainly want to prototype quickly or need the strongest model for a hard task, start with a closed API. You can move to open-weight once the project's shape and volume are clearer.
Most teams that end up self-hosting started on an API, learned their usage pattern and data sensitivity, and moved once one of the three questions decided it. That order works, as long as you build against a portable interface such as the OpenAI-compatible API rather than one provider's SDK, so moving doesn't mean a rewrite.