Fine-Tuning Architecture Interview Questions

500M vectors at sub-100ms p99 is a staff-level vector search design question from Uber, Airbnb, and large-scale ML platform teams. Learn sharding strategies, index tuning, and the operational trade-offs that separate senior from principal engineers.

Cursor rules are context engineering in disguise.

A production AI coding agent typically includes:

Part 3 of the hallucination handbook — production defense layers from retrieval and prompting through guardrails, citations, confidence scoring, eval frameworks, and enterprise architecture. Asked at OpenAI, Google, Meta, Anthropic, Microsoft, and Amazon.

**Training** Refusal datasets, toxicity filters on training data.

**3.1** 128k context, stronger instruction following — update inference stack for long context KV cache.

**Session model** WebSocket/WebRTC streaming multimodal input/output.

**Proxy layer** Backend holds keys; assigns budgets per customer/plan.

**Session** Persistent realtime connection with session.update for instructions/tools; handle reconnect.

Multi-server hosts are production reality — GitHub + Jira + DB + docs + internal APIs. Poor design creates tool soup.

Enterprise MCP rarely connects laptops directly to SaaS MCP servers. The gateway is the control plane interviewers want you to design.

Building a custom MCP server is a common system-design follow-up to "what is MCP?" Strong candidates describe end-to-end architecture, not just SDK boilerplate.

Horizontal scaling patterns: hash sharding, semantic partitions, routing embeddings, and merge/rerank at query time.

Protect instructions, secrets, policy, and tool policies in system prompts against leakage, override, and indirect injection.

Medium AI Agents interview question on tool schema design and versioning — architecture, trade-offs, eval, and production patterns.

Hard AI Agents interview question on customer support agent system design — architecture, trade-offs, eval, and production patterns.

Medium AI Agents interview question on swarm vs supervisor multi-agent design — architecture, trade-offs, eval, and production patterns.

Medium AI Agents interview question on idempotent tool execution — architecture, trade-offs, eval, and production patterns.

Medium AI Agents interview question on tool calling design best practices — architecture, trade-offs, eval, and production patterns.

Hard RAG interview question on serverless rag architecture on aws/azure — architecture, trade-offs, eval, and production patterns.

Model question on RLHF — SFT, reward model, PPO alignment, and how human preferences change helpfulness vs hallucination trade-offs.

Hard scenario on LLM cost control — token budgeting, routing, caching, batch APIs, and unit economics at millions of queries.

RAG has become the foundational architecture for production GenAI applications at companies like Notion, Duolingo, and Morgan Stanley. Interviewers expect you to explain the full retrieval pipeline — not just define the acronym. Follow along to master what RAG is, when to use it over fine-tuning, and how to articulate trade-offs that separate junior from senior candidates.

Enterprise RAG interviews test system design at scale: ACL-aware retrieval, audit logging, and ingestion pipelines for millions of documents. This is a staff-level question appearing at Microsoft, Salesforce, and Fortune 500 AI teams. Walk through a complete architecture with security boundaries and operational concerns.