AI Observability Interview Questions

Hard scenario on PII in LLM systems — detection, redaction, DPA compliance, log minimization, and regional deployment.

Agents fail opaquely without MCP observability. Treat every tool call as a microservice span.

Medium AI Agents interview question on agent tracing and replay debugging — architecture, trade-offs, eval, and production patterns.

Enterprise RAG interviews test system design at scale: ACL-aware retrieval, audit logging, and ingestion pipelines for millions of documents. This is a staff-level question appearing at Microsoft, Salesforce, and Fortune 500 AI teams. Walk through a complete architecture with security boundaries and operational concerns.

Product teams shouldn't each reinvent agent security.

AI observability extends classic APM.

Treat agent models/prompts/tools like dependencies.

**Infrastructure** Managed endpoints vs batch prediction; GPU quotas.

**Proxy layer** Backend holds keys; assigns budgets per customer/plan.

**Contractual** Enterprise DPA, ZDR endpoints, prohibition on training customer data.

**Tracing** Span per node: inputs hash, duration, LLM/tool child spans. thread_id correlates user session.

**Serving pattern** Stateless API pods; thread state in checkpointer DB; long runs may use queue workers resuming checkpoints.

**State schema** ticket_id, customer_tier, messages, retrieved_docs, proposed_actions, sentiment, escalation_reason, resolution_code.

**Retry patterns** Tenacity-style retries inside node or dedicated retry wrapper node incrementing state.retry_count. Exponential backoff for transient MCP errors.

**LangChain agents** Higher-level executors (ReAct, tool calling) with less boilerplate. Good for demos and simple loops. Harder to debug complex branching, human interrupts, and precise persistence semantics.

Enterprise MCP rarely connects laptops directly to SaaS MCP servers. The gateway is the control plane interviewers want you to design.

Latency, recall proxies, ingest lag, index size, filter rates, error budgets, and RAG downstream signals.

Hard AI Agents interview question on startup interview: build an agent mvp in 4 weeks — architecture, trade-offs, eval, and production patterns.

Hard AI Agents interview question on building an internal agent marketplace — architecture, trade-offs, eval, and production patterns.

Medium AI Agents interview question on openai swarm / agents sdk patterns — architecture, trade-offs, eval, and production patterns.

Medium RAG interview question on end-to-end rag observability — architecture, trade-offs, eval, and production patterns.

Hard project question on LLM gateway — auth, routing, rate limits, logging, key management, and multi-provider abstraction.

Hard Amazon company question — model cascades, unit economics, SageMaker/Bedrock routing, and SLO-aware cost optimization.

Hard Meta company question — Llama serving, GPU clustering, quantization, batching, and open-weight ops.