Temperature, Top-p, and Top-k: Decoding Strategies Explained (SOLVED)
Decoding parameters interview question — how sampling controls randomness, creativity, and hallucination risk in production.
TL;DR — Quick Answer
LLMs output a probability distribution over next tokens. Temperature scales that distribution — lower (0–0.3) = more deterministic for factual tasks; higher (0.7–1.0) = more creative for brainstorming. Top-k keeps only the k highest-probability tokens; top-p keeps the smallest set whose cumulative probability exceeds p (nucleus sampling). Production factual apps often use low temperature + top-p ~0.9; creative writing uses higher temperature. Lower randomness reduces but does not eliminate factual errors.
The Interview Question
Explain temperature, top-p (nucleus), and top-k sampling. When would you use each in production LLM applications?
Deep Explanation
From logits to text
Sign in to unlock full answer
Get deep explanations, PDF export & all LLMs questions
- 13 more sections of deep explanation
- Real-world examples
- Common mistakes
- Interviewer expectations
- Follow-up questions