Deep explanation
AI System Built failure in an enterprise RAG assistant: how would you respond?
Junior scenario question: investigate a groundedness drop in an internal RAG assistant you built, from checking ingestion and the index to prompt and model changes, with low-cost fixes and a clear interview story.
Quick answer
I would collect examples of the bad answers and, for each, check whether the right document was retrieved at all, because that tells me if the problem is in search or in the model's answer. Common causes are an ingestion job that failed so new documents are missing, a changed chunking or embedding setting, or a prompt or model change. I would fix the specific cause, re-run a small test set of questions with known source documents to confirm, and add a daily check so it is caught earlier next time.
The interview question
You helped build an internal enterprise RAG assistant that answers employee questions from company documents. While you are on call, users report that answers are less grounded: they cite the wrong documents or say things that are not in the sources. As a junior engineer, how would you investigate and fix this without breaking the latency SLA or the budget, and how would you describe it as "an AI system I built"?
Deep explanation
Simple explanation
Sign in to unlock the full answer
Free accounts include 5 full deep answers. Sign in to start unlocking.
- 20 more sections of deep explanation
- Real-world examples
- Common mistakes
- Interviewer expectations
- Follow-up questions