What is an LLM hallucination?
A hallucination, in the context of a large language model, is an output that is fluent and confident in tone but factually wrong, fabricated, or unsupported by the source material it was supposed to be grounded in. A model can invent a citation that doesn't exist, misstate a date, cite a court case that was never decided, or describe a function argument that no library actually accepts, all while phrasing it the same way it would phrase a correct answer.
Why it happens
A language model generates text by predicting the next most likely token given everything before it, based on patterns learned from training data. It has no built-in mechanism for checking whether a specific claim is true; it produces the sequence of words that statistically resembles a correct answer, and most of the time that sequence is correct because true statements dominated its training data. When the pattern of a plausible-sounding answer diverges from an actually correct one, the model has no internal signal telling it to stop, because fluency and correctness aren't the same thing it's optimizing for.
Where it shows up most
Hallucination risk rises with how far a question sits from well-represented training data: obscure facts, recent events past the model's training cutoff, specific numbers, and exact citations are common failure points. It's also common when a model is asked to reason beyond what it retrieved or was told, filling gaps with plausible-sounding invention rather than saying it doesn't know. Models tend to hallucinate less on tasks with strong patterns in training data, like common code syntax, and more on tasks requiring precise, verifiable facts.
What reduces it, and what doesn't fully fix it
Retrieval-augmented generation grounds a model's answer in retrieved source documents rather than its training memory alone, which meaningfully reduces fabrication on questions the retrieved documents actually cover. Lower sampling temperature makes output more deterministic and can reduce some invention, though it doesn't address the underlying cause. Neither eliminates the problem: a model can still misread a retrieved passage or state something confidently that the passage doesn't actually support. Systems that rely on LLM output for consequential decisions generally need a human review step or automated fact-checking against a trusted source, not just a better prompt.
Why it matters for how you deploy
Hallucination is a property of the model's generation process, not something that changes based on where the model runs. What does change with deployment is your ability to test, log, and iterate on a specific model against your actual workload, comparing outputs across model versions and prompt changes to see which configuration hallucinates least on your data. See evaluating an LLM before deploying it for a structured way to test this before committing to a model, and logging and log retention for LLM APIs for keeping the record needed to catch hallucinations after deployment.