Docs/Inference API
Inference API

Call open models with one API key

By Samuel Seidel · Updated November 20, 2026 · 5 min read

The GPUwerk Inference API serves open-weight models from DGX Sparks, priced per token and paid from your account credit. One API key works for every model, in the OpenAI format and in the Anthropic format, so most clients need only a new base URL and key. There is no machine to rent and nothing to deploy.

You need:

On this page
  1. Models and prices
  2. OpenAI format
  3. Anthropic format
  4. Use it from your tools
  5. Limits and errors
  6. Where requests run and what is kept

Models and prices

Prices are in US dollars per million tokens. The console's AI inference page always shows the current list and prices; this table is from November 20, 2026.

Model IDPromptCached promptCompletionContextMax output
qwen3.6-35b-a3b (Qwen3.6 35B A3B, NVFP4)$0.05$0.02$0.70131,0728,192

The model is quantized to NVFP4 and served with vLLM. Its reasoning is returned separately from the answer (reasoning_content in the OpenAI format), not inside it. Prompt caching is automatic: a repeated prompt prefix is charged at the cached price. Native tool calling (the tools parameter) is not supported yet. GET /v1/models lists the models available to your key.

OpenAI format

Base URL https://api.gpuwerk.com/inference/v1, your key as a bearer token. Chat completions and completions work, streamed or not.

curl https://api.gpuwerk.com/inference/v1/chat/completions \
  -H "Authorization: Bearer $GPUWERK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen3.6-35b-a3b",
       "messages": [{"role": "user", "content": "Summarise GDPR Article 28 in three bullets."}]}'

With the OpenAI Python SDK:

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.gpuwerk.com/inference/v1",
                api_key=os.environ["GPUWERK_API_KEY"])

resp = client.chat.completions.create(
    model="qwen3.6-35b-a3b",
    messages=[{"role": "user", "content": "Summarise GDPR Article 28 in three bullets."}],
)
print(resp.choices[0].message.content)

Anthropic format

For clients that speak the Anthropic Messages API, the base URL is https://api.gpuwerk.com/inference (without /v1) and the key goes in x-api-key.

import os
import anthropic

client = anthropic.Anthropic(base_url="https://api.gpuwerk.com/inference",
                             api_key=os.environ["GPUWERK_API_KEY"])

msg = client.messages.create(
    model="qwen3.6-35b-a3b",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarise GDPR Article 28 in three bullets."}],
)
print(msg.content[0].text)

Use it from your tools

Any tool with an "OpenAI-compatible" provider setting works with three values: base URL https://api.gpuwerk.com/inference/v1, your API key, and the model ID qwen3.6-35b-a3b.

Limits and errors

Each account has limits, shown in the console: today 60 requests a minute, 2 requests in flight and $25 of spending per UTC day. Errors you may see:

The API runs on Sparks while no rental needs them, and rentals always take priority, so capacity varies. There is no service level agreement. Batch jobs that can retry later suit it best. The terms are in section 21 of the Terms.

Where requests run and what is kept

A request runs on a DGX Spark that is free at that moment: one GPUwerk owns in the Czech Republic, or one owned by a hosting partner, which may be in another country. The console's AI inference page shows the countries of the machines serving requests now.

To keep a key's requests on GPUwerk-owned machines in the Czech Republic, tick "GPUwerk-owned machines in Czechia only" when you create the key in the console, or send "own_hardware_only": true when you create it with POST /v1/api-keys; PATCH /v1/api-keys/<id> changes it later (see the API reference). It is off by default. Fewer machines can take such a key's requests, so it may get 429 more often.

Prompts and completions are held in memory while a request runs. They are not stored, logged or used for training. Billing records hold the time, model and token counts, never content. The Inference API data policy describes where requests run, what protects them on a partner's machine and which sub-processors are involved.

NextPut LiteLLM in front of it Back toAll docs

One key for open models, paid per token.

Create an account, add credit, and create an inference key in the console.

Create an API key