0:00Last time, we changed the question before searching, and counted the right passages that came back.
0:06This time, the right passage is already in the results, and we fix where it ranks.
0:11First, a search that finds the answer and ranks a less useful passage above it.
0:16Then a model called a cross-encoder, which reads the question and each passage together and scores them again.
0:23We'll time that model, because it does more work for every passage it scores.
0:28Then the one thing it can't do, which is bring back a passage the search never returned.
0:33Finally, a simple check of whether reranking put the answer where we need it. Let's start with the search itself.
0:39This script is the first stage of our search, a semantic search like the ones we've built so far. It ranks four short passages against one question.
0:49The user wants to reset their password, but they can't get into their email account any more.
0:54The passages cover a reset link sent by email, notification settings, getting help from support, and keeping an account safe.
1:02Only the support passage works for this user. The reset link is sent to the email they can't open.
1:08The embedding model turns the question into a vector, and the passages into vectors, in separate calls. Cosine similarity compares them, and the passages are sorted by that score.
1:20We run the first stage.
1:23It prints each passage's similarity score and the start of its text, highest first.
1:28The reset link comes first. It's about the same thing as the question, resetting a password by email.
1:34The support passage comes second. So the search found the answer, but ranked it below a passage this user can't use.
1:41Why did the reset link score higher?
1:44It comes from how the first stage scores a passage.
1:48The embedding model turns each text into its own vector.
1:52The question is encoded without seeing any passage, and each passage without seeing the question.
1:58Since a passage's vector doesn't depend on the question, it's computed once, ahead of time. That's what makes the first stage fast.
2:06But each vector is a summary made without the other text. It can show that two texts are about the same thing, which isn't the same as one answering the other.
2:18A reranker scores the pair a different way. It uses a cross-encoder, a model that takes the question and the passage together, as one input.
2:27Both texts go through the model at the same time, so it can compare the passage's words directly with the question's.
2:34It outputs one number for the pair, a prediction of how well this passage answers this question.
2:39Nothing here can be computed ahead of time, because every score depends on the question. Let's run a cross-encoder on our four passages.
2:50This script is the second stage. It imports the same question and the same four passages from the first one.
2:57It loads a small cross-encoder, trained to score how well a passage answers a search query.
3:03Each pair puts the question next to one candidate, so we have four pairs.
3:07The model returns one score for each pair, and the candidates are sorted by that score, highest first.
3:13The first stage's ranking is still above. Now we run the reranker.
3:18The same four passages print in a new order, with the cross-encoder's scores.
3:23The support passage is now first, and the reset link second. Reading each passage with the question, the model scored the one about lost email access higher.
3:33These scores aren't on the cosine scale, and two of them are negative. They're only useful for ordering the candidates for this question.
3:41The gap between the top two is small, though. With a slightly different wording of the question, the reset link stays first, so treat this as one example rather than a rule.
3:53Reading every pair takes more work than comparing stored vectors. Let's measure how much.
4:00This script times the cross-encoder as the number of pairs grows.
4:04It builds our four pairs and runs the model once without timing it, so the setup isn't counted.
4:10Then it repeats the pairs to stand in for longer shortlists, and times one call to the model for each size.
4:17We run the timing script.
4:19Each line is one call to the model, timed on this laptop's GPU.
4:24Forty pairs take about two and a half times as long as four, because the GPU reads the pairs in batches.
4:31From four hundred up, ten times the pairs take about ten times as long. Four thousand pairs take almost two seconds, for a single question.
4:40The exact times depend on the hardware and the length of the passages. But a collection of a million passages would mean a million pairs for every question, so a reranker only reads a shortlist.
4:54Systems that rerank usually work in two stages.
4:57The first stage searches everything, with passage vectors computed ahead of time and an index, so it stays fast as the collection grows.
5:06How many passages it keeps is a setting, often somewhere between twenty and fifty. Our example kept all four.
5:14The cross-encoder reads only those, so its cost depends on the shortlist's length, not on the size of the collection.
5:21The best few after reranking go on to whatever uses them, usually a language model writing the answer.
5:28The reranker can only reorder what it's given. Let's see what that means for the shortlist's length.
5:34This sketch shows the question's embedding as a point.
5:38Our four passages are placed by similarity, so the reset link is nearest, then the support passage, as in the first stage's ranking.
5:46Suppose the first stage passed on only its closest passage.
5:50The support passage is still in the collection, but it isn't in the shortlist. The reranker never reads it, so no reordering can put it first.
5:58Reranking changes the order of what the first stage retrieved, and can't add to it. So the first stage's recall is a ceiling on what reranking can do.
6:08The usual fix is a longer shortlist.
6:11With two candidates, the support passage is in, and the reranker can move it up.
6:16Each extra candidate is one more pair to score, so a longer shortlist costs time. And if the answer sits far down the first stage's list, the first stage needs fixing.
6:26Now let's measure whether reranking helped.
6:31This script checks whether the passage that answers the question is near the top of each ranking.
6:37We know the support passage answers this question, so it's marked by hand.
6:41This function keeps the first k passages of a ranking and checks whether the answer is among them. The check is called hit at k.
6:49The script ranks the passages with the first stage alone and with both stages, then runs the check with k of one and four.
6:57We run the check.
6:59It prints one line for each k.
7:01For the top one, the answer is missing before reranking and present after, which is the change we saw in the two rankings.
7:08For the top four, it's present both times. With four candidates, reranking only changes their order, so a system that passes on four passages gets the same four either way.
7:19So pick k to match how many passages your system actually uses. And one question isn't enough to decide whether a reranker is worth adding.
7:29Here's how to test a reranker on your own data.
7:32Start with ten to twenty questions people really ask, where you know which passage holds each answer.
7:38Search for each one with the first stage alone and with both stages, and count how often the answer lands in the top k.
7:45Measure the reranker's time on the number of candidates you'd really send it, since every question adds that time.
7:51Keep the reranker if it puts answers in the top k noticeably more often, and the extra time fits what your system can afford.
7:59The first stage finds candidates quickly by comparing stored vectors. A cross-encoder then reads each candidate with the question, and puts them in a better order.
8:10You can now add that second stage, measure what it costs, and check with hit at k whether it helps on your questions.
8:17So far, each video has looked at one step of retrieval. Next, we follow a wrong answer back through those steps to find the one that caused it.

Reranking Search Results

Created by Abhyuday DesaiPublished 4 Oct, v1Runs 8 min 32 sec

0 views

About this video

In Episode 05 of Retrieval Explained, a user wants to reset their password but can no longer open their email. The first stage of the search finds the support passage that would help them, and ranks the emailed reset link above it. A cross-encoder reads the question and each passage together and puts the support passage first. We time it as the number of pairs grows, see why a reranker only reads a shortlist from the first stage, and check with hit at k whether reranking put the answer where the system needs it.

You'll learn how to:

Tell embeddings computed ahead of time from a cross-encoder that reads the question and the passage as one input
Rerank a shortlist with a small cross-encoder, and read its scores as an order rather than a probability
Measure the reranker's time on the shortlist sizes you would really send it
See why the first stage's recall is a ceiling that reranking cannot raise
Check a reranker with hit at k on questions where you know the answer

This video is for developers whose search finds the right passage but ranks it too low. The code runs from the companion repo.

What changed

v104 Oct 2026abhydesai8 min 32 secb9e8014

First published version.

0 views0 likes on this version