Reranking
A second-pass step in retrieval that re-scores a shortlist of candidate documents with a more accurate (and more expensive) model, so the best matches rise to the top.
Definition
First-stage retrieval, whether keyword, vector or hybrid, is built for speed. It compares a query with millions of documents using pre-computed representations, which makes it fast but approximate. Reranking adds a second, slower stage: it takes the top 50 or 100 candidates and scores each one against the query with a model that reads both together. The reordered top few are what the LLM finally sees.
Most rerankers are cross-encoders. Unlike the bi-encoder used for embeddings, which encodes query and document separately, a cross-encoder reads the pair in one pass and can weigh exact wording, negation and context. That accuracy is why it is kept for the shortlist only. Rerankers are available as hosted APIs and as open models that can be self-hosted.
Example: a query about "termination for convenience by the customer" retrieves 60 contract passages. Vector search ranks a generic termination clause first. The reranker notices that one passage specifically grants the customer that right and moves it to the top.
Why it matters for your business
- Better passages in, better answers out: reranking is one of the highest-return tweaks to a RAG system.
- Lets you send fewer tokens to the LLM with more confidence, cutting cost.
- Adds latency and a dependency; both must be measured against the quality gain.
- Works with any first stage, so it can be added to an existing search without re-indexing.
How Teksmobile uses it
Reranking is part of the standard retrieval stack in our enterprise RAG builds and our hybrid search service. We choose between hosted and self-hosted rerankers based on data-residency rules and measure the gain on the client's own query set.
Related terms
Part of the Teksmobile AI Glossary. For longer reads, see our guides.
Frequently asked questions
Why not use the reranker for everything?
Because it scores each query–document pair individually, it cannot be pre-computed into an index. Running it over millions of documents per query is far too slow, so it only sees a shortlist.
How much does reranking help?
It depends on the data, but it is one of the most reliable improvements to RAG answer quality for the effort involved. Measure it with your own retrieval evals before and after.
Which rerankers exist?
Hosted rerank APIs from several providers, open-source cross-encoder models that can be self-hosted, and LLM-based reranking for small candidate sets. Latency and data residency usually decide.