
A hallucination is when an AI confidently gives you false information: a fake fact, a made-up refund rule, a library that doesn't exist. It doesn't look like an error. It looks exactly like a correct answer, which is what makes it dangerous.

Why it happens
An LLM is a very good next-word predictor. It doesn't look facts up. It writes the most likely next word, then the next, until the answer feels complete.
If the real answer isn't in its training data or in the context you gave it, it still finishes the pattern smoothly. The result reads well and is wrong. The model isn't lying on purpose. It just has no built-in sense of "I don't actually know this."
How I reduce it in production
- Ground it with RAG. Search our own documents first and pass the best chunks in, so the model answers from real text instead of memory.
- Make it cite. Every claim must point to a source ID. If it can't cite it, it shouldn't say it, and a missing citation is easy to check in code.
- Let it say "I don't know". If the search scores are too low, don't call the model at all, or tell it to say it doesn't have enough information and hand off to a person.
- Measure it. Keep a test set of tricky questions and check faithfulness on every prompt or model change.
The part people miss
Most "hallucinations" I've debugged were not the model inventing things. The model answered faithfully from the wrong three chunks. That is a search problem, and you fix it in retrieval, not by rewriting the prompt. More on that in how I fix a broken RAG pipeline.
Key takeaways
- 01LLMs predict words; they don't verify facts.
- 02Grounding, citations and an abstain threshold do most of the work.
- 03Check retrieval before blaming the model.