From ML engineer to GenAI platform engineer: an 8-week bridge

This is an eight-week bridge for an ML engineer who wants to move into GenAI platform engineering. It assumes you already train and evaluate models, work with data pipelines, and think in terms of metrics and experiments. The shift is from building one model well to building the shared platform other teams use to ship language-model features safely. Plan for focused weekday hours and one longer build each weekend, and stretch the weeks if you need to.

The plan builds one project across all eight weeks: a small internal platform that lets a team call models through one gateway, run retrieval, evaluate changes, and see what everything costs. Each week adds a layer you can explain in an interview.

Week 1 is the model-serving view. Learn how hosted model APIs and self-hosted models differ in practice: tokens, context windows, sampling settings, rate limits, and streaming. Build a thin gateway that sits in front of model calls, logs every request and response, and records token counts and timing. Your ML background helps here, because you already know to measure before you optimize.

Week 2 is prompts as managed assets. Move prompts out of code into versioned templates, with a fixed set of test inputs for each. Practice structured output that downstream code can parse. The platform question to answer is how a team changes a prompt without breaking someone else's feature.

Week 3 is retrieval as a service. Build ingestion, chunking, embeddings, a vector store, and metadata filters, with access control applied at query time. Check what retrieval returns before reading any generated answer. The RAG interview guide at /blog/rag-interview-guide covers the questions you should be able to answer here.

Week 4 is evaluation infrastructure. Your ML background gives you a head start here, because offline evaluation is familiar. The new part is that outputs are free text and there may be no single right answer. Build an evaluation harness that runs a fixed set of cases on every change, with separate checks for retrieval, faithfulness to the retrieved context, and whether the answer resolved the question. The LLM evaluation guide at /blog/llm-evaluation-interview-guide is the companion for this week.

Week 5 is agents and tools on the platform. Add a way for teams to register tools with typed inputs, clear descriptions, permission checks, and step limits. The MCP tools and server failure modes guide at /blog/mcp-tools-servers-failure-modes-interview and the LangGraph agent system design guide at /blog/langgraph-agent-system-design-interview-guide cover the design questions you will face.

Week 6 is safety and guardrails. Add input and output checks, rules for keeping sensitive data out of logs, redaction where needed, and a clear path for human review. The platform view is that these checks should be shared building blocks, not something every team reinvents.

Week 7 is cost, latency, and reliability. Add caching, timeouts, retries, fallbacks to a second model, and per-team usage reporting. Measure where time and money go, and practice explaining what you would change when traffic grows.

Week 8 is rehearsal. Write a five-minute story of your platform: the problem, the first version, what broke, how you measured it, and what you would build next. Do at least two mock interviews out loud, one on platform system design and one on debugging a failed request end to end.

The habit that carries ML engineers through this switch is one you already have: treat every change as an experiment with a measurement. GenAI platform interviews add questions about shared infrastructure, safety, and cost, but they still reward someone who can say what they measured and why they trust it. For a story-style look at a similar switch from the backend side, read the 90-day story at /blog/backend-dev-to-ai-engineer-90-days.