Hallucination
Also called: confabulation, model hallucination
When a language model produces something fluent, plausible and false — usually because it was asked a question it had no grounds to answer.
The model produces an answer that reads correctly and is wrong. A citation that does not exist, a policy clause that was never written, a figure that looks like the others.
The fluency is the problem. A confidently wrong answer is harder to catch than an obviously wrong one.
Why it happens
A language model predicts plausible continuations. Asked something it has no grounded basis for, the most plausible continuation is a well-formed answer — not an admission of ignorance. Without design intervention, the default behaviour is to produce something.
What actually reduces it
Ground the answer. Supply the relevant source material at question time and instruct the model to answer only from it. This is what RAG is for.
Cite sources. So a user can verify. This also changes model behaviour: a system required to point at its evidence produces fewer ungrounded claims.
Constrain the task. Narrow, well-specified questions hallucinate far less than open ones.
Permit refusal. Explicitly instruct and reward "I do not know." Systems that cannot refuse will always invent.
Measure it. An evaluation set turns "it sometimes makes things up" into a number you can move.
What does not work
Asking the model to be accurate. Instructions like "do not hallucinate" have close to no effect, because the model is not knowingly fabricating — it has no separate signal telling it which of its outputs are grounded.