0:00Last time, we compared keyword and semantic search, always with the question exactly as the user typed it.
0:07This time, we change the question before it reaches the search.
0:10First, a baseline: a casual question, searched as it is, against a small collection of engineering notes.
0:17Then we ask a language model to turn it into a cleaner query.
0:21Next, several rewordings of the question, each searched on its own, with the results combined.
0:27Then a method called HyDE, which has the model write a possible answer and searches with that instead of the question.
0:34Finally, we count how many of the right passages each method retrieves. Let's start with the question and the collection.
0:42This file holds the question we'll work with for the whole video, and the collection we search.
0:47The user's service slows down the more time it spends running, and they ask about it casually, the way people type into a chat.
0:54Ten short notes follow, and two of them answer the question. The first is about memory leaks, where memory grows for hours until the service slows down.
1:03The second is about connection pools. Connections that are never given back make requests wait, so response times climb while the process runs.
1:12The other eight cover other performance topics, like profiling a slow function or speeding up startup. The retriever is at the bottom of the file.
1:22It's semantic search. Every note is embedded once, and a search returns the five notes whose embeddings are most similar to the query's.
1:30Let's search with the question exactly as the user typed it.
1:35The baseline script passes the question to search unchanged, then prints each result's ID and its first few words.
1:42We run the baseline.
1:44Five passages come back, the most similar first.
1:47The top result is about making a service start faster. It shares words like Python and service with the question, but it's about a different problem.
1:57Second is the memory leak passage, so one of the two answers made it in.
2:01The connection pool passage isn't among the five. So the baseline finds one of the two answers, even though the other one is in the collection. Why didn't the search find it?
2:13A sketch of the embeddings helps. The question's embedding is a point, placed according to what its words mean.
2:19Each of the ten notes has an embedding too, placed the same way.
2:23Search returns the five notes nearest the question. These five are the baseline's results.
2:29The memory leak passage is close enough to get in. The connection pool passage talks about database connections and response times, so it lands further away.
2:38Different wording would put the question somewhere else in the space. So the methods in this video change what we search with.
2:45The first one rewrites the question, so let's open its script.
2:51This function sends the question to a language model, with a short instruction in front of it.
2:56It asks for a single query that makes sense on its own, keeps what the user meant, and doesn't guess at a cause.
3:03The script prints the rewrite, then searches with it, using the same retriever and the same five results.
3:10The baseline's results are still above. Now we run the rewrite script.
3:15After a few seconds for the model call, the rewrite prints, then the five passages it retrieves.
3:21The model dropped the greeting and the question at the end, and kept the user's own description of the problem.
3:27So the search returns the same five passages as before, in the same order. It's still one of the two answers.
3:34A rewrite helps most when a question is vague, or leans on earlier messages. This one was already clear, and the problem is the words it uses.
3:45So the next method searches with several different wordings at once.
3:50The expansion script asks the model for several alternative queries instead of one.
3:55The prompt asks for three, one per line, worded differently but asking for the same thing.
4:01The list of queries starts with the rewrite we just ran, and the three alternatives follow it.
4:07Each query gets its own search. The merge keeps one copy of each passage, in the order it first appears.
4:14Then the merged list is cut back to five passages.
4:19We run the expansion script.
4:21After the model call, the four queries print, then the final five passages.
4:27The alternatives do change the vocabulary. One talks about performance degrading over time, and another about diagnosing a service that runs for a long time.
4:36But the final five are exactly the passages the rewrite found. Its results came first in the merge, so they took every slot.
4:44As written, this merge can't change the result. And all four are still questions about a slow service, while the connection pool passage is written about database connections.
4:55The third method stops searching with questions, and searches with an answer instead.
5:01HyDE stands for hypothetical document embeddings. The model writes a short passage that could answer the question, and we search with that passage.
5:09The prompt asks for a few sentences written like documentation, because that's how the notes in our collection are written.
5:16It also tells the model the passage doesn't have to be right. Search only uses its wording, and nothing checks what it claims.
5:24Then the script searches with the generated passage instead of the question.
5:30We run the HyDE script.
5:32The model takes a few seconds to write the passage, and the script prints it before the results.
5:37It names unbounded caches, objects kept in memory, and unreleased connections, written in the same style as our notes.
5:45This time the memory leak passage is first and the connection pool passage is second. Both answers are in the five.
5:51Once the search has returned, the generated passage has done its job.
5:56The script's last comments state the rule. The hypothetical is thrown away, and only the real passages the search returned go on to be quoted or given to the model.
6:07If it were passed along as a source, an unverified guess would look like evidence. Let's go back to the sketch to see why searching with it worked.
6:17This is the sketch from the baseline search. Searching with the question returned the memory leak passage and missed the connection pool passage.
6:25The hypothetical is written like an answer, so its embedding lands near the passages that answer the question.
6:32Search returns the five notes nearest to it, and this time both answer passages are among them.
6:39That matches the run, where both answer passages came back in the top five.
6:43This can work against you too. A hypothetical about the wrong cause would move the search toward the wrong passages just as easily.
6:51So for every method, we need to count what it actually retrieves.
6:57This script does that count for every input we've tried.
7:00First, the two passages that answer the question are labeled by hand, by their IDs.
7:05This helper searches with a query and counts how many labeled passages come back in the top five. That count is called hits at five.
7:13We also score one bad rewrite on purpose. It assumes the problem is a memory leak, which the user never said.
7:21The loop scores four inputs: the raw question, the saved rewrite, the saved hypothetical, and the bad rewrite.
7:30We run the evaluation.
7:33It prints one count per input.
7:35The raw question and the rewrite each find one of the two answers, which is what we saw in their runs.
7:41The hypothetical finds both.
7:43The bad rewrite also finds one. It gets the memory leak passage and loses the connection pool passage, because it narrowed the question to a single cause.
7:52Expansion isn't listed, but its final five were the rewrite's five, so it would score one as well.
7:59All of these queries read well, and only the count shows which ones found the answers. Which method should you try first?
8:08It depends on why the search is missing.
8:11If the question is chatty, vague, or depends on earlier messages, a rewrite is the cheapest thing to try.
8:18If documents use other words for the same thing, try expansion, with a merge that lets each query's results into the final list.
8:26If questions are phrased very differently from the documents that answer them, try HyDE, and never treat the generated passage as a source.
8:35Whichever you try, count the labeled answers it retrieves across a set of real questions, not just one. Then weigh that against the time and cost of the extra model calls.
8:46We changed what goes into the search in three ways, and kept the collection and the retriever the same.
8:51A transformed query can read better and still return the same passages, or narrow the question. You can now try each method, and check it by counting the answers it brings back.
9:03Everything so far decides which passages come back. Next, we look at reranking, which takes those passages and orders them again with a more careful model.

Query Transformation

Created by Abhyuday DesaiPublished 4 Oct, v1Runs 9 min 20 sec

0 views

About this video

In Episode 04 of Retrieval Explained, a user asks casually why their service slows down the longer it runs. Searched as typed, the question finds one of the two notes that answer it. A language model rewrites it into a cleaner query, then into three rewordings searched separately and merged, and both return the same five passages. HyDE has the model write a short passage that could answer the question, and searches with that instead; both answers come back. Counting the labeled answers in the top five, including for a bad rewrite that guesses at a cause, shows which methods actually helped.

You'll learn how to:

See how a question's wording places it among the embeddings, and why an answer written in other words can land too far away
Rewrite a question with a language model without letting it guess at a cause
Expand a question into several queries, and merge their results so that each query can contribute
Search with a hypothetical document, and never pass it along as a source
Measure each method with hits at five on labeled answers

This video is for developers whose search misses answers that are in the collection. It calls a language model through any OpenAI-compatible provider; the companion repo shows how to set one up.

What changed

v104 Oct 2026abhydesai9 min 20 secb9e8014

First published version.

0 views0 likes on this version