Hallucination

What is a Hallucination?

A hallucination occurs when a language model generates text that is linguistically coherent but factually incorrect—invented facts, names, sources, or numbers that sound just as confident as correct statements.

How it works

For every token, a language model calculates which continuation is statistically most likely, regardless of whether the existing context is factually correct. There is no internal mechanism that recognizes and corrects a statement once it has already begun but is incorrect. Once generation has committed to a false fact, every subsequent token continues that line consistently because it is calculated from the already generated—but incorrect—context.

Why an incorrect answer can still sound convincing

Linguistic confidence and factual correctness are two separate properties of generated text. A language model continuously optimizes for linguistic coherence, not for factual accuracy. A confident, authoritative tone is therefore not a reliable indicator that the information is correct.

A hallucination is not a defect or software bug but a direct consequence of how language models fundamentally work: they generate probabilities, not verified facts.

Why hallucinations have practical implications

• Model size: Under comparable conditions, smaller models tend to hallucinate more often because their more limited model capacity may represent subtle relationships less effectively. Training data, architecture, Fine-Tuning, and prompting also play important roles.
• Verification: Facts, numbers, quotations, and references generated by language models should always be verified independently whenever factual accuracy or safety is important.
• Topic dependency: Hallucinations occur more frequently in niche subjects, rare details, or questions that fall outside the model’s training data.

Typical example

A model invents a plausible-sounding book title together with an author and publication year, even though the book does not exist. The response sounds just as confident as a correct one.

Further reading

Fundamentals

→ How a Language Model Works

Related terms

→ Inference
→ Parameter