Architecture AI Interview Questions

500M vectors at sub-100ms p99 is a staff-level vector search design question from Uber, Airbnb, and large-scale ML platform teams. Learn sharding strategies, index tuning, and the operational trade-offs that separate senior from principal engineers.

Cursor rules are context engineering in disguise.

A production AI coding agent typically includes:

**Session model** WebSocket/WebRTC streaming multimodal input/output.

**Proxy layer** Backend holds keys; assigns budgets per customer/plan.

**Session** Persistent realtime connection with session.update for instructions/tools; handle reconnect.

Multi-server hosts are production reality — GitHub + Jira + DB + docs + internal APIs. Poor design creates tool soup.

Enterprise MCP rarely connects laptops directly to SaaS MCP servers. The gateway is the control plane interviewers want you to design.

Building a custom MCP server is a common system-design follow-up to "what is MCP?" Strong candidates describe end-to-end architecture, not just SDK boilerplate.

Horizontal scaling patterns: hash sharding, semantic partitions, routing embeddings, and merge/rerank at query time.

Protect instructions, secrets, policy, and tool policies in system prompts against leakage, override, and indirect injection.

Medium AI Agents interview question on tool schema design and versioning — architecture, trade-offs, eval, and production patterns.

Hard AI Agents interview question on customer support agent system design — architecture, trade-offs, eval, and production patterns.

Medium AI Agents interview question on swarm vs supervisor multi-agent design — architecture, trade-offs, eval, and production patterns.

Medium AI Agents interview question on idempotent tool execution — architecture, trade-offs, eval, and production patterns.

Medium AI Agents interview question on tool calling design best practices — architecture, trade-offs, eval, and production patterns.

Hard RAG interview question on serverless rag architecture on aws/azure — architecture, trade-offs, eval, and production patterns.

Enterprise RAG interviews test system design at scale: ACL-aware retrieval, audit logging, and ingestion pipelines for millions of documents. This is a staff-level question appearing at Microsoft, Salesforce, and Fortune 500 AI teams. Walk through a complete architecture with security boundaries and operational concerns.

Multi-agent orchestration is a senior/staff-level system design question gaining traction at Google DeepMind and Microsoft. Learn the supervisor pattern, shared state management, and how to avoid the 'too many agents' anti-pattern that sinks most candidate answers.

The Model Context Protocol is Anthropic's open standard reshaping how LLMs connect to tools and data. If you're interviewing for Claude ecosystem roles or AI platform engineering, expect MCP questions. Understand host vs server architecture and how MCP differs from ad-hoc function calling.

Structured JSON output from LLMs is a production necessity at Stripe, Vercel, and every AI-native startup. Interviewers test your reliability engineering — schema enforcement, retry logic, streaming edge cases — not just 'use JSON mode.' Master the full production playbook.

OpenAI's model lineup changes fast. GPT-4 vs GPT-4o is a model selection question that tests whether you understand latency, cost, multimodal capabilities, and when reasoning depth matters. Critical for any role touching OpenAI APIs in production.

A production Claude Code rollout is an agent platform problem, not a 'give everyone a CLI' problem.

This is an enterprise platform design question spanning agents, MCP, governance, and observability.