Context, short-term memory, and conversation memory

🟠 SHOULD KNOW · Memory is an application data design, not proof that a model remembers accurately.

Overview

• Context is what the model sees now.
• Short-term memory is task/session information retained across nearby steps.
• Conversation memory is selected dialogue history or a summary used to maintain continuity.

Why this matters

Users refer to earlier information and dislike repeating details every turn.

An everyday example

A support employee keeps notes during your call.

A technical example

Retain a verified claim ID, recent messages, and pending question in a session record.

How it works

1. Persist scoped messages/events
2. Select recent relevant items
3. Refresh authoritative facts
4. Construct bounded context
5. Update approved session state.

Architecture and flow

Session store → history selection + verified facts → current model context.

The insurance assistant example

“What document did you say was missing?” uses previous dialogue but rechecks the claim's current document status.

When to use it

Multi-turn chat and active task assistance.

When to avoid it

Do not resend every conversation forever or treat old dialogue as current backend truth.

Interview answer

“I persist conversation state but send only relevant context, and I refresh business facts through authorized tools.”

Benefits, trade-offs, and production considerations

• Scope conversations by tenant and user, handle deleted messages, and retain tool-call/result pairs correctly.
• Unbounded history increases privacy exposure, cost, and confusion.