Project 3 — PDF RAG assistant
Words made simple
A few useful meanings before you read the explanation.
- AI
- Software techniques for tasks such as recognizing patterns, understanding language or generating content.
- API
- A way for one piece of software to ask another for data or an action. A claims API might return the current status of a claim.
- Token
- A small piece of text processed by a model. It may be a whole word, part of a word, or punctuation.
More words from this lesson
- Embedding
- A list of numbers that represents the meaning of some content. Similar meanings can produce nearby lists of numbers.
- Vector
- An ordered list of numbers. An embedding is a vector used to represent content for tasks such as similarity search.
- Retrieval
- Finding and fetching information that is useful for the current question.
- Authorization
- Checking what that identity is allowed to read or do. Being logged in does not grant access to every record.
- Tenant
- A customer or organization whose data must stay separate from other customers in a shared application.
Requirements
Upload approved PDFs, track ingestion, ask questions, return source references, and abstain when evidence is missing.
Architecture
Upload → parse/chunk/embed → index; question → retrieve → model → validated cited answer. Parsing preserves source coordinates.
Technologies
Node API, queue, PDF parser or Python worker, embedding API/model, one vector store, Zod/Pydantic.
Database design
Documents with version/hash/status; chunks with source text/page/model metadata; ingestion jobs and conversation ownership.
API design
POST /documents, GET /jobs/:id, POST /questions; response includes answerability and authorized source IDs.
AI design
Section-aware chunking, matching query/document embeddings, bounded top-k context, clear abstention instructions.
Security
File size/type limits, isolated parsing, tenant filters, secure original-file storage, and source-link authorization.
Evaluation
Known clause questions, scans, tables, missing answers, outdated versions, and cross-tenant documents. Measure retrieval and answer quality separately.
Observability
Stage timings, parse failures, index readiness, retrieved IDs, answer support, tokens, and cost.
Deployment
Queue workers independently; provide a staging corpus; verify object storage and database backups.
Interview questions
- “What if retrieval fails?” Answer: distinguish no evidence from system failure and avoid inventing an answer.
- “How do citations work?” Answer: map model references to real retrieved source IDs.
Acceptance: Every material answer claim has applicable evidence; missing evidence leads to a clear fallback.
Can you explain it without the notes?
Project 3 — PDF RAG assistant
Use your own words. Explain an example and one limitation before checking the explanation.
Check your explanation
Requirements: Upload approved PDFs, track ingestion, ask questions, return source references, and abstain when evidence is missing.
Turn this concept into a clear answer.
Practice this conceptAdapted from 13_AI.MD · “Project 3 — PDF RAG assistant”. Examples are study fragments; framework behavior can depend on your version.