distiliaa Learn
Module 5 12 min

Hallucinations

Why models make things up — and what you can do about it.

Ask an LLM for the biography of a person you invent, and it may deliver a confident, detailed, entirely fictional life story — complete with plausible dates and a university it picked out of thin air. This is a hallucination, and after everything you've learned in this module, you're ready for the punchline: it isn't a bug in the code. It's the training objective doing exactly what it was designed to do.

Why does this exist?

This lesson exists because hallucination is the number-one thing that breaks real LLM products — fabricated case law in legal filings, invented API functions in generated code, fake citations in research summaries. You cannot fix what you misdiagnose. Understanding why models make things up (they predict plausible text, they don't retrieve facts) is what separates engineers who ship reliable systems from those who ship lawsuits.

The mechanics of making things up

Recall the loop: the model always outputs a probability distribution over next tokens, and a sampler always picks one. There is no step where the model checks a database, and no built-in "I don't know that" escape hatch. When you ask about something real and well-represented in training data, the most probable continuation happens to be true. When you ask about something obscure or nonexistent, the machinery keeps producing the most plausible-shaped continuation:

Prompt:  "Dr. Elara Voss, the marine biologist, is best known for"
Model:   "her groundbreaking research on bioluminescent
          communication in deep-sea cephalopods, published
          in her 2011 book 'Signals in the Dark'..."

Dr. Voss doesn't exist. But thousands of real biographies shaped the model's sense of what biography-text looks like, so it generates a statistically perfect one. Fluency and truth come from the same mechanism — that's why hallucinations are so convincing.

Why the model can't just say "I don't know"

Several forces stack up:

  • No knowledge/plausibility distinction. Facts are stored as statistical associations in weights, not as entries that can be present or absent. There's no lookup that can miss.
  • Compression is lossy. Trillions of tokens squeezed into billions of parameters (sound familiar from the autoencoder module?) keeps common patterns and blurs rare details — precisely the details you ask about when questions get specific.
  • Training rewards answering. During next-token training, hedging text isn't especially probable; during human-feedback tuning, raters historically preferred confident, complete answers. Both push toward fluent commitment.
  • Sampling adds spice. Higher temperature raises the odds of drifting into a low-probability (often wrong) token, and then the autoregressive loop happily builds on the error.

High-risk zones

Hallucination rates spike for: specific citations and URLs, niche people and companies, exact numbers and dates, recent events past the training cutoff, and API/library details ("plausible function names that don't exist" is a classic). Treat model output in these zones as unverified drafts, always.

Engineering defenses

You can't eliminate hallucination with a clever prompt, but you can engineer around it in layers:

  1. Ground the model (RAG)Retrieve real documents and instruct: answer only from the provided context, else say you don't know. Turns recall into reading comprehension — the single most effective fix.
  2. Lower the temperature
  3. Give an exit ramp
  4. Verify downstream
  5. Design for review

A minimal grounded prompt looks like this:

Answer the question using ONLY the context below.
If the context does not contain the answer, reply exactly:
"I don't have enough information to answer that."

Context:
{retrieved documents}

Question: {user question}

This pattern is the seed of Retrieval-Augmented Generation, which gets its own module soon.

Reframe: creativity and hallucination are the same feature

The mechanism that invents a fake citation is the one that writes a novel poem or brainstorms product names: generating plausible text that was never in the training data. "Hallucination" is what we call that ability when we wanted a fact. The engineering question is never "how do I turn it off" — it's "does my use case want plausible text or verified truth, and if truth, where does verification come from?"

Build it yourself

Build a small hallucination probe: write 10 questions about entities you invented (fake papers, fake people) and 10 about well-known ones, ask a model at temperature 0 and 1.2, and score fabrications. Then repeat with the grounded prompt above and empty context. Measuring the difference yourself will teach you more than any benchmark chart.

Summary

  • Hallucination is the training objective working as designed: plausible ≠ true, and the model only optimizes plausible.
  • There's no internal lookup that can fail — facts are lossy statistical associations in weights.
  • Risk peaks on citations, niche entities, precise numbers, recent events, and API details.
  • Defend in layers: ground with retrieved context, lower temperature for factual work, allow "I don't know", verify mechanically, keep humans in the loop for stakes.
  • The same machinery powers creativity — your job is deciding when plausibility suffices and when verification is mandatory.