Private AI for HR and recruiting

Resume screening is one of the most common AI use cases in HR, and one of the most sensitive. A stack of resumes contains names, addresses, employment history, sometimes photos, and, once you read between the lines, signals about age, nationality, disability, or family status that a candidate never explicitly disclosed but that a model can pick up on anyway. Run that stack through a third-party AI API and all of it, plus every performance review and disciplinary note an HR tool later touches, passes through that vendor.

Not legal advice. Automated decision-making in hiring is a live regulatory area in multiple jurisdictions, and the specific rules depend on where you operate and where your candidates are. This post describes an infrastructure choice, not a compliance opinion. Check with your own employment counsel before using AI in any hiring decision.

Why HR data is a different category

Most business documents are sensitive because they're confidential: a contract, a financial forecast. HR data is sensitive for a different reason, it's about identifiable people who didn't choose to be your customer or counterparty, and in many jurisdictions employment data sits closer to the categories that get extra regulatory attention than ordinary business records do. A resume alone can reveal a candidate's age (graduation years), possible disability (an employment gap explained honestly), or national origin, none of which the candidate necessarily intended to disclose as a factor in the decision. Feeding that into a general-purpose AI product, with unclear retention and no HR-specific handling, means those inferences exist somewhere outside your organization.

Employee data compounds the exposure further. Performance reviews, disciplinary records, and compensation history are the kind of information HR treats as need-to-know internally, restricted to the employee's manager and HR itself. Running an AI tool over that data through a shared third-party endpoint puts it through infrastructure with a much wider blast radius than the access controls your HR system already enforces.

What running the model yourself changes

The setup looks like any self-hosted assistant: an instruct model on your own hardware, wired to whatever workflow needs it, whether that's summarizing a resume against a job description, drafting interview notes into a structured format, or answering a manager's question against an employee's review history. Resumes, reviews, and disciplinary notes go to your model, on your infrastructure, and nowhere else. There's no vendor data processing agreement to evaluate for this specific use case, no question about whether the plan tier your recruiter happens to be logged into opts out of training on submitted content.

That doesn't resolve the harder question, which is whether AI should be part of the hiring decision at all, and if so, how. Screening resumes for keyword match against a job description is a different act than ranking or rejecting candidates automatically, and the second one is exactly where automated decision-making rules in hiring start to apply in various jurisdictions. Keep a human reviewing and making the actual decision; use the model to summarize and surface, not to decide.

A reasonable starting scope

Start with the lower-stakes half of the workload: summarizing resumes into a consistent format for a recruiter to read faster, or answering an HR generalist's question against policy documents (this overlaps with a general internal knowledge base, if HR policy is already indexed there). Save anything that touches a ranking or pass/fail judgment on a specific candidate for later, once you've worked out with counsel what role AI is allowed to play in that decision under the rules that apply to you.

Wherever the model touches employee-identifiable data, apply the same access boundary the underlying HR system already has. A self-hosted model doesn't inherit your HR platform's permission model automatically, someone has to wire that access control into whatever front end sits in front of the model, or it becomes a shortcut around restrictions that took real effort to set up in the first place.

Retention is part of the design, not an afterthought

A cloud AI product's default retention window is built for the vendor's general use case, not for HR's specific obligations around candidate and employee records, which often carry their own retention and deletion rules independent of anything AI-related. Running the model yourself doesn't automatically solve retention either, someone still has to decide how long transcripts, summaries, and chat logs from the HR tool itself are kept, and delete them on that schedule. The advantage of self-hosting here is that the retention policy is entirely yours to set and enforce; there's no vendor default to override or a support ticket to file when a candidate asks what happened to their data.

Rejected-candidate data is worth deciding on explicitly. Some organizations keep resumes and screening notes for a defined period in case a role reopens; others delete on rejection. Whatever your policy already is for the underlying HR system, apply the same schedule to anything the AI tool generated from that data, summaries, extracted skills lists, screening notes, rather than letting those artifacts persist past the point the source record was supposed to be deleted.

Where the hardware fits

An instruct model in the 20B-70B parameter range handles resume summarization and internal HR Q&A comfortably within a single Spark's 128GB of unified memory, alongside whatever retrieval layer sits in front of policy documents. HR data volumes are typically modest compared to a company-wide knowledge base, so this rarely needs more than one node; a $0.79/hour Spark in EU-Central covers most HR teams' actual workload.