LLM Fundamentals for Production Interviews: Tokens, Context, and Failure Behavior

LLM fundamentals interviews in production roles focus on behavior under constraints, not architecture history. Interviewers want to know how tokenization affects billing and context limits, why a JSON schema request fails on certain unicode inputs, and how temperature and top-p interact with structured output requirements in customer-facing APIs.

Context window management is a systems problem. Candidates should explain prefill versus decode costs, why long contexts increase latency disproportionately, and how to prioritize which documents, tool results, and conversation history survive when you hit token limits. Truncation strategies have direct product impact—dropping the system prompt is not equivalent to dropping oldest user turns.

Hallucination is not a single failure type. Confident fabrication, subtle numeric drift, and correct-sounding but wrong citations each need different mitigations. Interviewers probe whether you distinguish parametric knowledge from retrieved knowledge and design systems that refuse when evidence is insufficient rather than filling gaps plausibly.

Decoding choices affect reproducibility, safety, and eval stability. Discuss when greedy decoding is appropriate, when self-consistency helps, and how logprobs enable confidence-based routing to smaller models or human review. Production teams care about variance across model versions as much as mean quality.

Reasoning models and tool-augmented workflows change the interview bar. Expect questions on extended thinking budgets, when to externalize reasoning to scratchpads, and how to detect when a model is reasoning past its reliable depth. Tie every concept back to observability: what you log, what you alert on, and what you measure per route.