Project 4 — Production RAG application
Words made simple
A few useful meanings before you read the explanation.
- AI
- Software techniques for tasks such as recognizing patterns, understanding language or generating content.
- API
- A way for one piece of software to ask another for data or an action. A claims API might return the current status of a claim.
- Vector
- An ordered list of numbers. An embedding is a vector used to represent content for tasks such as similarity search.
More words from this lesson
- Retrieval
- Finding and fetching information that is useful for the current question.
- Grounding
- Connecting an answer to supplied evidence, such as a policy document or a verified database result.
- Authorization
- Checking what that identity is allowed to read or do. Being logged in does not grant access to every record.
- Tenant
- A customer or organization whose data must stay separate from other customers in a shared application.
- Evaluation
- Checking results against examples and clear criteria to find whether the system works well and where it fails.
Requirements
Multi-tenant retrieval, incremental updates/deletions, policy versions, hybrid search, evaluated reranking, and source-aware caching.
Architecture
Source sync → versioned ingestion → published index;authorized question → hybrid retrieval → rerank → grounded output.- Lifecycle and permissions drive eligibility.
Technologies
Existing app stack, queue, object storage, vector/lexical search, reranker where justified, telemetry/eval platform.
Database design
ACL and corpus versions, processing manifests, failure reasons, chunk provenance, publication markers, and safe cache dependencies.
API design
Admin ingestion/reindex endpoints and status, query endpoint with cited response, secure source open endpoint, deletion/revocation flow.
AI design
Tune chunking and retrieval on a golden set; introduce parent expansion or query rewrite only when measurements justify it.
Security
Apply eligibility before evidence reaches the model, propagate revocation/deletion to vectors/caches, and test injected documents.
Evaluation
Release gates for retrieval recall, applicable correctness, unsupported answers, citations, tenant isolation, p95 latency, and cost.
Observability
Source freshness, sync lag, failed ingestion, filtered retrieval metrics, route/version tags, and feedback linked to traces.
Deployment
Versioned index rollout with validation and rollback; capacity-test real filters; protect backups and caches.
Interview questions
- “Why hybrid?” Answer: combine paraphrase recall with exact terminology.
- “How do you update safely?” Answer: publish completed validated versions and invalidate dependent results.
Acceptance: A changed or revoked document is reflected consistently in search, generated answers, and cached output.
Can you explain it without the notes?
Project 4 — Production RAG application
Use your own words. Explain an example and one limitation before checking the explanation.
Check your explanation
Requirements: Multi-tenant retrieval, incremental updates/deletions, policy versions, hybrid search, evaluated reranking, and source-aware caching.
Turn this concept into a clear answer.
Practice this conceptAdapted from 13_AI.MD · “Project 4 — Production RAG application”. Examples are study fragments; framework behavior can depend on your version.