What GenAI onsite whiteboards look like now

A GenAI onsite whiteboard round often looks less like a classic distributed-systems exercise and more like a conversation about a product that uses a language model. This post describes a common format and how to practice for it. It is general guidance, not a description of any named company's interview.

The prompt is often short and open: design a support assistant over a help center, an internal search tool over company documents, an agent that handles a simple workflow, or a summarizer for long meetings. You get a fixed block of time, a whiteboard or shared document, and an interviewer who will push on whatever you skip.

The format moves through a recognizable set of stages, and practicing each one is the best preparation.

The first stage is requirements. Who are the users, how much traffic is expected, where does the data come from, how fresh must it be, and what does a wrong answer cost? Spend the first few minutes here. These answers decide whether you need retrieval, tools, human review, or a smaller model.

The second stage is the data flow. Draw the path from the user's request to the final answer: input handling, retrieval or tool calls, the model call, and output handling. Keep it simple at first and add detail only where the interviewer leans in.

The third stage is the deep part, usually retrieval or tools. For retrieval, expect questions on chunking, indexing, access control, and how you check what came back. For tools, expect questions on schemas, permissions, retries, and stopping rules. The RAG interview guide at /blog/rag-interview-guide, the vector database system design guide at /blog/vector-database-system-design, and the LangGraph agent system design guide at /blog/langgraph-agent-system-design-interview-guide cover these.

The fourth stage is evaluation. How will you know it works, and how will you know a change made it better? Describe a fixed test set, separate checks for retrieval and answer quality, and a review of failures. The LLM evaluation guide at /blog/llm-evaluation-interview-guide gives you the language.

The fifth stage is failure paths and safety. What happens when retrieval finds nothing, a tool fails, or a user tries to make the system do something it should not? Draw the fallback, the hand-off to a human, and where authorization is enforced. The MCP security guide at /blog/mcp-security-interview-questions is useful here.

The sixth stage is cost and latency. Name where time and money go, and the levers you would pull: caching, trimming context, streaming, and routing simple requests to a smaller model. The LLM cost and latency guide at /blog/llm-cost-latency-interview-guide covers this.

To practice, pick three prompts and run each one on a timer, out loud, using the same six stages. Have a friend interrupt you, or record yourself and listen back for places where you said "and then the model handles it" without saying how. After each run, write down the one stage you rushed and start there next time. The composite story at /blog/failed-llm-system-design-round shows how this round goes when the stages are skipped.

The whiteboard is not grading a perfect architecture. It is grading whether you can move from a vague prompt to a system you can explain, measure, and defend when someone pushes on it.