Reranking

Overview

• Reranking is a second ranking stage that examines a short candidate list more carefully.
• A reranker often reads the query and each candidate together, allowing richer comparison than separate embeddings.
• It changes candidate order; it cannot recover a relevant document absent from the candidate list.

Why this matters

Initial retrieval prioritizes speed and recall, while the answer needs a small set of highly relevant evidence.

An everyday example

A recruiter first finds 50 possible candidates, then carefully reviews their applications before selecting five.

A technical example

Retrieve 50 chunks, rerank them, and send the best five nonduplicate passages to the LLM.

How it works

1. Retrieve broadly
2. Score query/candidate pairs
3. Sort
4. Diversify/deduplicate
5. Construct bounded context.

Architecture and flow

Initial retrieval → reranker → selected evidence → generation.

The insurance assistant example

Prefer the applicable claim reporting clause over a superficially similar marketing article.

When to use it

When candidate recall is good but the final context includes weak or distracting matches.

When to avoid it

It will not solve missing ingestion, wrong permissions, or an unrelated embedding model.

Interview answer

“Reranking improves precision among retrieved candidates. I fix retrieval recall first and justify its additional latency and cost through evaluation.”

Benefits, trade-offs, and production considerations

• Cap candidate count and batch requests.
• Rerank scores are model-specific.
• Treat all candidate text as untrusted input and keep tenant boundaries in place before sending it to an external reranker.