Private AI for technical documentation
An LLM that reads a codebase can draft new docs and flag stale ones, which helps with the thing a team lets slip first and a new hire misses most. Reading the code means having access to it, though, and for proprietary code that access shouldn't mean feeding the source to a hosted coding assistant. A model on a dedicated machine does the same reading without that.
What this actually saves
Writing documentation from scratch is slow because it requires reading code that's often undocumented in the first place, then writing a clear explanation of what it does and why. A model can draft that first pass: reading a function, module, or service and writing a description of its behavior, its inputs and outputs, and any non-obvious constraints it can infer from the code itself. It also helps with stale docs: comparing an existing doc against the current code and flagging where they've diverged, a task most teams skip because nobody wants to diff prose against source by hand.
None of this removes the need for a human editor. A model reading code can describe what the code does, but it can't reliably explain why a decision was made or which alternative was rejected. That context usually lives in someone's head or an old pull request discussion. Docs generated this way work best as a draft that an engineer who knows the system corrects and approves, not as a final artifact.
Why proprietary code is the wrong thing to hand a cloud tool
The parts of a codebase most worth documenting well tend to be the parts a company has the least interest in exposing: core business logic, architecture decisions that encode competitive advantage, authentication and authorization code where a documentation error could also reveal a security gap. Running a hosted coding assistant against that code for documentation purposes means the same source that goes into a code review tool goes here too, under the same third-party terms of service and the same uncertainty about training and retention. Reviewing diffs with AI has the same exposure; the coding agents doc shows how to point one at a model on your own machine.
A self-hosted model reads the same code without sending it to a coding-assistant vendor. That matters more for documentation than for review: a documentation pass touches every file in a codebase over time, not just the files changed this week, so running it continuously against a hosted service exposes the whole repository eventually.
How teams set this up
The common pattern is a coding model with retrieval over the repository: rather than loading the whole codebase into context at once, the system pulls in the specific file being documented plus related files (callers, dependencies, existing docs) as needed, the same retrieval approach covered in the guide to on-premise RAG. A CI step that runs on merged pull requests keeps docs from drifting far behind the code: it flags a docstring that no longer matches a changed function signature without anyone having to remember.
If you already run a self-hosted coding assistant day to day, the comparison of self-hosted coding assistants against Copilot covers the infrastructure question; documentation generation is usually one more task on the same running model, not a separate deployment. A DGX Spark node's 128 GB of unified memory holds a capable coding model with enough context budget to read a file and its immediate dependencies in one pass, and at $0.79/hour for a single-GPU node, running this as a periodic CI job costs a fraction of what continuous usage would.
FAQ
Can a model keep documentation updated automatically as code changes?
It can flag when a function's docstring no longer matches its signature or behavior, and draft an updated version for a human to approve, typically as a step in CI triggered by a merged pull request. It can't reliably judge whether a doc change should also update a higher-level architecture doc elsewhere; that still needs a person to decide.
Why not just use a hosted coding assistant that already reads the whole repo?
Some teams do, and for a codebase with nothing sensitive in it, that's a reasonable choice. The concern is specific to proprietary code: architecture, business logic, unreleased features and security-relevant implementation details are usually the parts most worth documenting well, and those are also the parts a company has the strongest reason to keep off third-party infrastructure.
Does the model need to read the entire codebase at once?
No. Most documentation tasks work on a file or module at a time, with retrieval pulling in related files as needed rather than loading the whole repo into context. That keeps response times reasonable even on a large codebase.