Google AI MLOps Engineer Interview Questions

**Infrastructure** Managed endpoints vs batch prediction; GPU quotas.

AI observability extends classic APM.

Advanced scenario question on why LLMs hallucinate — next-token prediction vs fact verification, root causes, and production defenses asked at OpenAI, Google, Meta, Anthropic, Microsoft, and Amazon.

Part 2 of the LLM hallucination handbook — factual, citation, reasoning, math, code, temporal, context, and tool-use hallucinations; decoding risks; RAG failure modes; production case study.

Part 3 of the hallucination handbook — production defense layers from retrieval and prompting through guardrails, citations, confidence scoring, eval frameworks, and enterprise architecture. Asked at OpenAI, Google, Meta, Anthropic, Microsoft, and Amazon.

Scenario interview on defining and communicating hallucination metrics to executives.

Hard agents ops question on timeouts, circuit breakers, and budget protection.

LLM interview topic on distillation — teacher/student training, latency/cost wins, and quality regression risks.

**Use cases** Full codebase questions, hours-long media, multi-doc merger analysis.

Rate limiting for agents differs from human API usage — bursts are larger and less predictable.

Deployment topology shapes MCP architecture — interviewers test practical product sense.

Long-running tools expose gaps between synchronous tool calling and real workloads.

Agents fail opaquely without MCP observability. Treat every tool call as a microservice span.

MCP auth interviews test whether you treat agents as privileged automation — because they are.

Cross-language prompt design covering translation drift, code-switching, locale formatting, and eval gaps in multilingual products.

Scenario question comparing CoT prompting with calculators, code interpreters, and search tools for reliable reasoning.

Project question on synthetic data — distillation datasets, diversity, contamination, quality filters, and human review.

Scenario question on abstention — retrieval scores, calibration, UX of uncertainty, and avoiding confident wrong answers.

Hard project question on use-case benchmarking — golden sets, task metrics, regression gates, and leaderboard limitations.

Project question on prefix/prompt caching and KV cache reuse — TTFT optimization and provider billing implications.

Model question on multimodal LLMs — image/audio tokenization, document QA, OCR pipelines, and latency/cost trade-offs.

Hard project question on LLM-as-judge — rubric design, position bias, human calibration, and production eval pipelines.

Scenario question on context windows — token budgets, lost-in-the-middle bias, and why RAG still matters despite 1M-token claims.

Foundational model question on BPE/tokenizers — why token counts differ from words, and production impact on cost, limits, and RAG chunking.