Project 4 — Production RAG application

Requirements

Multi-tenant retrieval, incremental updates/deletions, policy versions, hybrid search, evaluated reranking, and source-aware caching.

Architecture

• Source sync → versioned ingestion → published index; authorized question → hybrid retrieval → rerank → grounded output.
• Lifecycle and permissions drive eligibility.

Technologies

Existing app stack, queue, object storage, vector/lexical search, reranker where justified, telemetry/eval platform.

Database design

ACL and corpus versions, processing manifests, failure reasons, chunk provenance, publication markers, and safe cache dependencies.

API design

Admin ingestion/reindex endpoints and status, query endpoint with cited response, secure source open endpoint, deletion/revocation flow.

AI design

Tune chunking and retrieval on a golden set; introduce parent expansion or query rewrite only when measurements justify it.

Security

Apply eligibility before evidence reaches the model, propagate revocation/deletion to vectors/caches, and test injected documents.

Evaluation

Release gates for retrieval recall, applicable correctness, unsupported answers, citations, tenant isolation, p95 latency, and cost.

Observability

Source freshness, sync lag, failed ingestion, filtered retrieval metrics, route/version tags, and feedback linked to traces.

Deployment

Versioned index rollout with validation and rollback; capacity-test real filters; protect backups and caches.

Interview questions

• “Why hybrid?” Answer: combine paraphrase recall with exact terminology.
• “How do you update safely?” Answer: publish completed validated versions and invalidate dependent results.

Acceptance: A changed or revoked document is reflected consistently in search, generated answers, and cached output.