RAG chatbot

Overview

A RAG chatbot answers conversations using retrieved knowledge. The dialogue interface and retrieval pipeline are separate responsibilities.

Why this matters

Private or changing information cannot be safely answered from model knowledge alone.

An everyday example

A support employee consults the correct handbook while talking.

A technical example

/chat retrieves eligible policy passages and returns answer plus source IDs.

How it works

1. Authenticate
2. Resolve follow-up
3. Retrieve
4. Build context
5. Generate
6. Validate citations.

Architecture and flow

Chat API → retrieval → grounded model → cited answer.

The insurance assistant example

Explain reporting instructions from the customer's active policy.

When to use it

Document-grounded conversation.

When to avoid it

Account status belongs to an exact authorized service lookup.

Interview answer

“I combine conversation handling with eligible evidence retrieval and separate their evaluation.”

Benefits, trade-offs, and production considerations

Manage ingestion freshness, source versions, abstention, and follow-up rewriting; failures should identify whether search or generation failed.