OpenAI MLOps Engineer Interview Questions

MLOps for prompts: semantic versioning, eval gates, traffic splitting, and rollback when prompt changes regress quality or cost.

Programmatic prompt tuning with labeled datasets, composable modules, and optimizers that search instructions and demonstrations.

Silent quality decay from model updates, data shift, registry bypass, and stale few-shot banks without version discipline.

Company/scenario question on open vs closed LLMs — self-hosting economics, compliance, upgrade velocity, and hybrid strategies.

Advanced scenario question on why LLMs hallucinate — next-token prediction vs fact verification, root causes, and production defenses asked at OpenAI, Google, Meta, Anthropic, Microsoft, and Amazon.

Part 2 of the LLM hallucination handbook — factual, citation, reasoning, math, code, temporal, context, and tool-use hallucinations; decoding risks; RAG failure modes; production case study.

Part 3 of the hallucination handbook — production defense layers from retrieval and prompting through guardrails, citations, confidence scoring, eval frameworks, and enterprise architecture. Asked at OpenAI, Google, Meta, Anthropic, Microsoft, and Amazon.

Scenario interview on defining and communicating hallucination metrics to executives.

Medium prompt engineering question on prompt unit tests and CI gates.

Scenario interview on safe GenAI rollouts — flags, shadow eval, canaries, and kill switches.

LLM interview topic on distillation — teacher/student training, latency/cost wins, and quality regression risks.

**Azure benefits** VNet, Azure AD auth, Microsoft compliance certifications, existing EA billing.

**Pipeline** Pre-filter user input → model → post-filter output before display; async moderation for streaming with revoke.

**Model selection** Benchmark retrieval on your corpus — smaller models cheaper if recall OK.

**Common failures** Model calls nonexistent tool; args wrong type; parallel tool_calls partial failure; streaming partial JSON; user content tricks model into exfil tool args.

Rate limiting for agents differs from human API usage — bursts are larger and less predictable.

Company-based system design prompt for tool-using assistants with scope limits, confirmation gates, and structured tool policies.

Embedding similarity, k-NN example banks, MMR diversity, and eval-driven example curation for few-shot prompts.

Build golden sets, error taxonomy, automated graders, and human review loops to iterate prompts like production software.

Evaluate persona and role prompts for tone, domain framing, and when they add noise without improving task accuracy.

Foundational prompt engineering question on instruction placement, example selection, and when demonstrations beat bare instructions.

Medium AI Agents interview question on openai swarm / agents sdk patterns — architecture, trade-offs, eval, and production patterns.

Hard AI Agents interview question on browser agents and computer use risks — architecture, trade-offs, eval, and production patterns.

Easy AI Agents interview question on the ReAct pattern — Thought/Action/Observation loops, trade-offs, and production guardrails.