DGX Spark/Guides/API key rotation best practices for a private LLM deployment
Security

API key rotation best practices for a private LLM deployment

By Samuel Seidel · Published September 9, 2026

This guide covers how to scope, track, rotate and revoke the API keys for a self-hosted inference endpoint without causing an outage. The rotation mechanics are the easy part. The hard part is knowing which keys exist: a private endpoint collects them faster than anyone plans for (one for the console, one per teammate, one per script, one for the internal tool that started calling the model last month), and nobody rotates a key they don't know about.

Scope before you rotate

A key with narrow, specific scope is one you can rotate without a scramble; a key with broad, undocumented access is one nobody wants to touch because nobody's sure what breaks. GPUwerk's own console keys carry an explicit scope: an inference-only key, for example, can call the model endpoint and nothing else in the account (see the vLLM verification steps). Apply the same discipline to keys you issue yourself:

LiteLLM's key management, if that's your gateway

If you're running LiteLLM in front of your model, per-key controls are built in rather than something you bolt on. Keys are issued through its key management API or admin UI, each with its own rpm_limit and tpm_limit (see the rate limiting guide). The same issuance flow is where you set an expiration or a scoped model list.

Rotation with LiteLLM is four steps: issue a new key with the same scope, update the consumer to use it, confirm traffic has moved, then revoke the old one. Revocation takes effect immediately. If you're unsure whether a key is still in use, leave it alive one more cycle rather than guess wrong and break something silently.

A rotation schedule that matches actual risk

Rotating on a calendar for its own sake is a compliance checkbox more than a security measure. What matters is a cadence that matches how exposed each key is.

Whatever cadence you pick, write it down. A rotation policy that lives only in someone's memory doesn't survive that person being on vacation when a key needs to turn over.

Rotating without an outage

The failure mode to avoid is revoking the old key before every consumer has switched to the new one. A safe rotation is sequential, not simultaneous:

  1. Issue the new key with identical scope to the one it's replacing.
  2. Update the consumer's configuration or environment variable to the new key. Then check the gateway's usage or spend logs for that key to confirm it's actually in use, not just deployed.
  3. Leave the old key active for a short overlap window, long enough to confirm nothing is still calling with it.
  4. Revoke the old key, and confirm the endpoint still answers using only the new one.

A key shared across several independent consumers (scripts, CI and a person) is the argument for not sharing keys in the first place: it turns step 2 into a coordination problem across everyone at once, instead of one script at a time.

Practical checklist

See the security hardening guide for the instance-level side of this, or the team access guide for setting up per-teammate keys in the first place. A first engagement can help design a key policy for a specific team's tooling.

First top-up: pay $20, get $30 in credit

Scoped keys from the start.

Rent a dedicated Spark from $0.79/h, billed by the minute, and issue a narrow, trackable API key for every teammate and script from day one.

Create account Read the security guide