A reranker is a cross-encoder that reads your query and each candidate document together, so it judges relevance far more closely than the embedding search that found them. The best reranking APIs are hosted endpoints you call on your top results; pick an open-weight model instead when data can't leave your servers or volume makes per-token billing painful.
Here's the practical difference. An embedding model encodes the query and the documents separately, which is why it can search millions of chunks in milliseconds. A reranker reads each query and document as a pair, which is far more accurate and far too slow to run over the whole index. So you don't choose a reranker instead of retrieval. You bolt it on after.
Where reranking sits in a retrieval pipeline
A typical two-stage setup looks like this. Stage one pulls a generous candidate set, say the top fifty or a hundred chunks, using embeddings, keyword search or both. Stage two sends the query and those candidates to the reranker, which scores every pair and hands back a new order. You keep the top handful and pass them to your LLM or show them to the user. If you haven't settled stage one yet, start with embedding models, because a reranker can only reorder what retrieval found.
Three forks decide which reranker fits that second stage.
Hosted API or a model you run. A hosted endpoint is one HTTP call and someone else's GPUs. An open-weight model or a library runs in your own process or cluster, which keeps documents on your side and turns a per-call bill into a hardware bill. Half the tools below are hosted only; the other half give you weights, a library or a container to run yourself.
How you're billed. Hosted rerankers meter in different units: per search, per token, or as part of a wider platform's usage. The unit matters more than the headline rate, because it decides whether long documents or big candidate sets blow up the cost.
What the model can do. Some rerankers are tuned for many languages. Some take a long context, so you can rerank whole documents rather than small chunks. At least one takes plain-language instructions about how to rank, such as favouring recent material or a particular source. If your corpus is multilingual or your ranking rules aren't pure relevance, this fork outranks the other two.
Hosted rerank endpoints
Cohere Rerank
A hosted reranker from a large model company: send a query plus a candidate list and get it back reordered, billed by search unit.
Voyage AI rerankers
Cross-encoder rerankers sold by the token, with a free token allowance per account that covers a lot of prototyping.
Contextual AI reranker
Your ranking rules aren't just relevance? This hosted reranker accepts plain-language instructions, such as weighting recency or trusted sources.
Together AI Rerank
Teams already running inference on Together can add this rerank endpoint, which scores documents against a query on the same platform.
Pinecone hosted reranking
A vector database feature rather than a stand-alone API: it reranks search results inside Pinecone using a hosted or external model.
Models you can run yourself, or call by API
Jina Reranker
A multilingual listwise reranker with a very long context window, offered both through an API and as downloadable models.
Mixedbread rerank
Open mxbai rerank models in two sizes, built for over a hundred languages, which you can self-host or call through Mixedbread's API.
ZeroEntropy zerank
From a small retrieval company, a flagship reranker you can call by API or run from weights published on HuggingFace.
FlashRank
Python developers who want reranking without a network call use this light open-source library, which runs cross-encoders and LLM rankers in-process.
NVIDIA NeMo Retriever reranking NIM
Aimed at companies with their own NVIDIA hardware, this GPU-accelerated microservice runs reranking inside your own infrastructure.
What changed this year
Three shifts are visible on the vendors' own sites this month. Hosted model lines have moved on, with at least one vendor now announcing a fifth-generation reranker, so a tutorial written against an older version may not match what you get today. Long context has arrived, with at least one listwise reranker reading a context window big enough for whole reports. And instruction-following rerankers have appeared, which let you describe ranking rules in words rather than writing them as post-processing code.
Open weights have also become normal rather than exceptional. Four of the ten tools here either publish model weights or are open-source code, and one more is a container you run on your own GPUs. That changes the build-or-buy maths for anyone with steady, high volume.
Run a side-by-side on your own queries
Public leaderboards won't tell you how a reranker handles your documents. An afternoon will. Pull thirty real queries from your logs or your support inbox, including a few that your current search gets wrong. For each, save the top fifty candidates your stage one returns today, so every reranker sees the same input. Mark the one or two chunks that really answer each query.
Now send those identical candidate sets to two hosted rerankers and one model you can run locally. For each query, note where the right chunk lands in the top five. Count how often it moves into first place, how often it drops out of the top five, and how long each call takes. Then do one more pass with longer chunks, because that's where chunking rules and token billing start to bite.
Want AI doing the heavy lifting in your marketing?
I build the systems that handle the boring 80 percent, so you get your week back. Done properly, with the human kept in.
You'll usually find two of the three are close on quality, and the decision comes down to latency, billing unit and where the data is allowed to go. That's a far better reason to pick than a benchmark number someone else ran on someone else's corpus. If you also want to compare whole pipelines rather than one stage, this guide to RAG tools covers the wider stack.
How the bill is worked out
Each billing unit punishes a different habit, so read the definition, not just the rate.
Per search. Cohere defines one search unit as a query with up to a hundred documents, and any document over five hundred tokens is split into chunks that each count as a document. Long documents quietly multiply your searches.
Per token. Voyage counts query tokens times the number of documents, plus all document tokens. A long query against a big candidate set is expensive, so trim the query and the candidate count. Its free allowance per account is large enough to run a full side-by-side before paying anything.
Inside a platform. If you rerank inside your vector database, it's billed as part of that platform's usage, and on Pinecone's usage-based plan there's a monthly minimum whether you rerank or not. Handy if you're already paying it; an odd reason to sign up if you aren't.
Self-run. No meter at all, just GPU time, or CPU time for the lighter libraries. It's the cheapest route at steady high volume and the most work at low volume.
Two of the ten publish a per-token reranking rate, and on their own pricing pages this month it sits between roughly two and five cents per million tokens. That's cheap per call, and it's why the counting rule matters more than the rate: fifty candidates of four hundred tokens each is twenty thousand tokens a query before the query itself is multiplied in. The other eight either bill in another unit or don't publish a figure, so price those on your own query volume.
Mistakes that waste a reranker
A pool that's too small. If stage one only returns the top five, the reranker can only shuffle five, and the right chunk was probably number twelve. Give it fifty or more.
Chunks that are too big. Long chunks cost more on every billing model and get split anyway on per-search pricing, so chunk sensibly before you rerank.
No answer key. Without a few dozen queries where you've marked the right chunk, you can't tell whether the reranker helped, hurt or did nothing, and you'll end up judging it on vibes.
Latency nobody measured. A reranker adds a round trip to every query, and in a chat interface that delay is felt. Time it in your side-by-side, not after launch.
Which reranker to pick
Prototyping a RAG app: a hosted endpoint with a free allowance, so you can measure the lift before you commit to anything.
Documents that can't leave your servers: an open-weight reranker or an in-process library. For regulated teams that's often the end of the discussion.
Multilingual or long-document corpora: shortlist the multilingual, long-context models and test them on your hardest languages first.
Ranking rules beyond relevance: an instruction-following reranker saves you writing and maintaining the rules yourself.
Already all-in on one vector database or inference provider: try its built-in rerank first. One less vendor is worth a lot, as long as it holds up in your side-by-side.
Do you make reranking APIs or models? Tool makers can ask to be considered for a featured place on this list.
Want your tool on this list? Featured spots and reviews here are paid partnerships. Here's how to partner with me.


