Handbook-style interview prep: why LLMs hallucinate, hallucination types, and production defenses (RAG, guardrails, citations, eval) — asked at OpenAI, Google, Meta, Anthropic, Microsoft, and Amazon.
Why do Large Language Models hallucinate? This is one of the most common AI interview questions because it tests whether you understand how LLMs actually work — not just how to call an API.
Many candidates treat ChatGPT, Claude, or Gemini like databases that "know" facts. In reality, LLMs are probabilistic sequence models: they predict the next token, not verify truth. Interviewers want you to connect that theory to production engineering — RAG, grounding, citations, guardrails, evaluation, and when to admit uncertainty.
This guide is written as a premium AI engineering chapter. Part 1 covers the interview framing and root causes. Part 2 covers hallucination types and why RAG alone is not enough. Part 3 is the production architecture: layered defenses, fine-tuning vs RAG, confidence scoring, and eval frameworks.
Work through the three questions in order. Pair with our RAG hallucination mitigation question for debugging drills.
Deep explanations with architecture diagrams for every question below.
Advanced scenario question on why LLMs hallucinate — next-token prediction vs fact verification, root causes, and production defenses asked at OpenAI, Google, Meta, Anthropic, Microsoft, and Amazon.
Read full explanationPart 2 of the LLM hallucination handbook — factual, citation, reasoning, math, code, temporal, context, and tool-use hallucinations; decoding risks; RAG failure modes; production case study.
Read full explanation