Mistral failure in an enterprise RAG assistant: how would you respond?

Question

You are the senior engineer on call for an enterprise RAG assistant that uses a Mistral model (via La Plateforme or self-hosted open weights) to answer from internal documents. Over the past week users report answers are less grounded: fewer citations, more confident statements that are not in the sources. Nothing obvious was deployed. How do you investigate and fix it without breaking the latency SLA or the token budget?

Accepted Answer

First I split the problem into "the right text never reached the model" versus "the model had the text and ignored it", because the fixes are completely different. I would replay a fixed set of failing questions, log the exact retrieved chunks and final prompt, and check for silent changes such as a `-latest` model alias moving to a new version, an index rebuild, or prompt truncation. Then I fix the specific cause, pin versions, and add a groundedness check to the eval gate so it cannot regress silently again.

Mistral failure in an enterprise RAG assistant: how would you respond?

The interview question

Deep explanation

Simple explanation