Building a private AI internal knowledge base
A self-hosted LLM can turn a half-stale wiki, a shared drive nobody can search, and years of Slack threads into one knowledge base that answers questions with sources. The usual alternative, a hosted AI knowledge base product, means uploading the wiki, the drive and the Slack export to someone else's servers. For engineering runbooks, HR policies and internal decisions, that's a lot of company detail to hand to a third party in one batch.
The mechanics (embeddings, a vector database, an LLM for synthesis) are covered in self-hosted semantic search. This post covers what changes when the corpus is a company wiki plus its Slack history: what to index, what to leave out, and how to keep answers honest about what's current.
Wikis, docs, and chat are different shapes of knowledge
A wiki page is written to be read later, so it tends to be structured and reasonably self-contained. A Slack thread is the opposite: the useful answer is often buried three replies deep, in reaction to a question, with context that lives in the messages above it. Indexing Slack the same way you'd index a wiki, one message as one chunk, loses that context and returns fragments that don't make sense on their own.
The better unit for chat is the thread, not the message. Pull each thread as a whole, flatten it into a single block of text with speaker turns preserved, and embed that. It's more tokens per chunk than a wiki section, but a thread usually holds a resolved question and answer, which is what someone searching six months later wants.
What to include, and what to leave out
Not every channel belongs in the index. A #random channel or a channel used mostly for social chat will dilute retrieval quality without adding useful answers, since the embedding model has more noise to search through for the same signal. Start narrow: engineering channels, an incident-response channel, HR and onboarding, and the wiki spaces people really use. Expand from there based on what questions the tool fails to answer well.
DMs and private channels are a harder call. Decide it explicitly instead of defaulting to "index everything reachable by the service account." A knowledge base that can surface something someone said in a private channel, out of context, to whoever asks the right question, is a different product than one scoped to shared team spaces. Scope the indexing to what's meant to be searchable by the whole company, and treat access control on the source as the boundary the search tool has to respect too.
Staleness is the failure mode that matters most here
A wiki page from two years ago that nobody's touched since is either still correct or quietly wrong, and a search tool that can't tell the difference will confidently cite it either way. Two things help. First, surface the source's last-edited date next to any answer, so a reader can judge freshness themselves rather than trusting the tool's tone. Second, weight retrieval toward recency for anything document-shaped that changes over time, like policies or runbooks, while leaving older architectural decisions and postmortems unweighted, since those stay useful long after the date on them.
Re-indexing needs to be continuous, not a one-off project. A wiki that gets updated and a knowledge base that doesn't pick up the change within a day or two starts answering from the old version, which is worse than a search box that admits it doesn't know. A scheduled job that watches for edits and new Slack threads, then re-embeds only what changed, keeps the index from drifting without re-processing the whole corpus on every run.
Answering with attribution
The synthesis step, an LLM reading the retrieved wiki sections and Slack threads and writing a direct answer, is what makes this feel like asking a knowledgeable coworker rather than searching a database. The instruction that matters most is telling the model to answer only from what was retrieved and to name its sources: "according to the deployment runbook, updated March" reads very differently from an unsourced answer, and it lets someone check the source themselves before acting on it.
Link back to the original wiki page or Slack thread in every answer. That's the citation a careful reader wants, and it's how gaps get found: a reader who clicks through and finds the source thin or outdated has just told you which page needs attention.
One Spark runs the whole stack
The footprint matches the general semantic search setup: embeddings model, vector database and a mid-size instruct model for synthesis, all in one Spark's 128 GB of unified memory. A wiki plus a few years of relevant Slack history from a mid-size team usually lands well under a million chunks, which a single Qdrant instance on the same box handles without tuning. If the corpus grows past that, or you want a larger synthesis model, a linked pair at $1.79/hour adds headroom (256 GB across two nodes, not one pool).
On a rented Spark, GPUwerk processes the indexed wiki and Slack export under its published DPA.