DGX Spark/Guides/Audit logging for a compliance-sensitive self-hosted LLM deployment
Security

Audit logging for a compliance-sensitive self-hosted LLM deployment

By Samuel Seidel · Published September 9, 2026

This guide covers what to capture in an LLM audit log, where to store it, and how to keep the log from becoming a liability of its own. If a compliance program requires a record of who asked a model what and what it answered, that record has to come from inside your own container. GPUwerk gives you a dedicated Spark with root access, not a managed platform with built-in audit trails, so the logging is yours to set up and own.

Where the responsibility line sits

GPUwerk provisions the machine and the network path to it. What runs inside your container, including any audit logging, is yours to configure and yours to secure. GPUwerk does not inspect, log, or retain the contents of requests sent to a model running in your instance, and it does not check whether audit logging is set up at all. If an auditor asks for evidence of request logging, that evidence has to come from your own instance, not from GPUwerk. This is deliberate: a self-hosted deployment is attractive precisely because the operator controls this layer end to end, and that control comes with the responsibility attached to it.

What to actually capture

An audit log that tries to capture everything usually ends up too large to query and too sensitive to keep. Start narrower and add fields only when a specific requirement calls for them.

Whether prompt and response text itself belongs in the log is a decision to make deliberately, not by default. Full-content logging is far more useful for an actual audit, but it also means the log becomes the largest concentration of sensitive data on the box. If regulatory or contractual language requires content retention, store it; if it doesn't, hashing plus metadata is usually the safer default.

Getting the logs out of the request path

If you run a gateway like LiteLLM in front of the model (see the LiteLLM setup doc), you can point its request logging at a structured sink rather than a local file: a syslog target, an S3-compatible bucket, or a database it writes to over the network. The general pattern for whatever gateway or reverse proxy is in front of your model:

  1. Log at the gateway or proxy layer, not inside application code for every service that happens to call the model. One logging point is easier to audit than several.
  2. Write logs off the instance as they're generated, or on a short interval, rather than accumulating them only on local disk. Terminating a Spark deletes its workspace permanently, so logs held only in /workspace disappear with the instance.
  3. Set retention centrally, at the sink, so a compliance-mandated retention period (say, one year of logs) is enforced by policy rather than by remembering to run a cleanup script.

Access control on the log itself

An audit log that anyone with SSH access can read or edit undermines its own purpose; the whole point is a record that's trustworthy after the fact. Two practical steps matter more than the storage technology chosen:

This overlaps directly with instance-level hardening in general, covered in the security hardening guide: running services under a non-root user and separating credentials by purpose reduces the blast radius of any one compromised process, audit logging included.

What compliance frameworks actually tend to ask for

Requirements vary by framework and by auditor. Treat this as a starting checklist, not legal advice, and confirm the actual requirement with whoever owns the compliance program before you build to it.

A few things come up often enough to plan for from the start: a documented retention period rather than an indefinite one, a way to produce logs for a specific user or time window on request rather than only a full export, and a written description of what the logging captures and what it deliberately omits. Write the description while you build the logging, not after an auditor asks for it.

For the network and instance side of a compliance-sensitive deployment, see the security hardening guide. For getting per-user identity into the logs in the first place, see the team and multi-user access guide. A first engagement can help scope logging requirements against a specific framework before deployment.

First top-up: pay $20, get $30 in credit

A dedicated Spark, root access, your own logging policy.

Rent a dedicated Spark from $0.79/h, billed by the minute, and build audit logging that matches your actual requirement, not a vendor's default.

Create account Read the security guide