Deep explanation
Debug Azure AI regression in a multi-tenant AI support platform
Mid-level debugging question: isolate an Azure OpenAI latency regression in a multi-tenant app using per-stage timings, 429/Retry-After behavior, token counts, deployment quotas, and Azure Monitor metrics.
Quick answer
First I split the request into stages and time each one, so I know whether the slow part is retrieval, our own code, or the Azure OpenAI call. If it is the model call, I check three things: are we getting 429s and silently retrying, did the prompt or output get longer, or is the deployment out of quota capacity. Then I fix the specific cause, for example by separating noisy tenants onto their own deployment or quota, trimming the prompt, or honoring Retry-After instead of retrying blindly.
The interview question
After a release, your multi-tenant support platform shows p95 latency spikes on chat replies while answer quality stays flat. The app calls Azure OpenAI through a shared deployment, and the same release changed the prompt template, the retrieval step, and the retry library. How would you isolate whether Azure OpenAI is the cause, and what would you fix?
Deep explanation
Sign in to unlock the full answer
Free accounts include 5 full deep answers. Sign in to start unlocking.
- 27 more sections of deep explanation
- Real-world examples
- Common mistakes
- Interviewer expectations
- Follow-up questions