System Design: Enterprise RAG Platform for Multi-Tenant Knowledge Assistants

Enterprise RAG platform design interviews start with tenants, SLAs, and compliance—not vector database logos. Interviewers want an end-to-end architecture: connectors, parsing pipelines, chunking services, embedding workers, index management, query API, generation layer, and feedback loops.

Ingestion at enterprise scale means heterogeneous sources—SharePoint, Confluence, tickets, PDFs with OCR—and continuous updates. Discuss idempotent pipelines, dead-letter queues, document versioning, and backpressure when a customer uploads a million-page archive.

ACL-aware retrieval is non-negotiable. Filters must enforce permissions at search time using authoritative identity systems, not cached approximations. Candidates explain failure modes when ACL sync lags and how to prevent cross-tenant leakage in shared infrastructure.

Platform APIs should expose observability to customers: query traces, citation previews, index health, and eval dashboards. Internal teams need multi-tenant cost attribution, per-customer rate limits, and noisy-neighbor isolation on shared GPU pools.

Disaster recovery includes index rebuild strategies, embedding model migrations, and RPO/RTO targets for search availability versus generation. Interviewers probe whether you design for regional failover or accept read-only degradation.

Rollout strategy matters. Describe blue-green indexes, shadow queries, and customer-specific canaries. Platform engineers think in migrations and backwards compatibility, not big-bang launches.