0:00A retrieval system finds the passages that match a question's meaning. This video shows how, with code we run and the numbers it prints.
0:09First, a small model running on this machine turns a question into a list of numbers, called a vector.
0:15Then we compare two of those vectors by the angle between them, and use that angle as a score.
0:21With that score we rank passages for a question, first by checking every passage, and then with an index that checks only some of them.
0:30Finally, we look at what this kind of search cannot do on its own. Let's start with the model.
0:37This script turns one question into a vector, using the sentence transformers library to run a small embedding model on this machine.
0:46The model is all-MiniLM-L6-v2. It downloads once and then runs locally, so no API key is involved.
0:53The encode call turns the question into a vector.
0:56Then we print the vector's shape, and its first four numbers.
1:00Let's run it.
1:02This runs the model on our question about resetting a password.
1:06The shape says the vector holds three hundred eighty four numbers.
1:10Every text gets a vector of that length, a short question or a long paragraph. Text past the model's input limit is cut off, not compressed.
1:20These are the first four. No single number stands for a topic or a word. The meaning is in the whole pattern, so we always compare whole vectors.
1:30So how do we compare two of them?
1:33To compare two embeddings, think of each one as an arrow from a shared starting point. This is the query's arrow, drawn in two dimensions instead of three hundred eighty four, and the picture is the same.
1:46A passage whose arrow points the same way as the query agrees with it completely. The cosine of the angle between them is one, the highest the score can be.
1:57If the passage swings a quarter turn away, the two have nothing in common.
2:01The angle between them is a right angle now, and that angle is what the score measures.
2:06The cosine of a right angle is zero, so the score is zero as well.
2:11Pointing the opposite way is complete disagreement.
2:14The cosine is minus one. So a smaller angle means a higher score, and that is the whole scoring rule.
2:21Real vectors also differ in length.
2:24At this angle the cosine is about point five seven, and length should not change that number, because direction is what carries the meaning.
2:31So we normalize. This circle marks length one from the origin, and that is where both arrows are going.
2:39Both lengths become one, and the angle between them does not change. Neither does the cosine. Length drops out of the comparison entirely.
2:49Written out, that is the cosine formula. The dot product of the two vectors, divided by the product of their lengths. The denominator is the normalization we just watched.
3:01Next, let's see that formula in code.
3:05This script uses that formula to score three passages against one question.
3:09Two of the passages are about passwords, and one is about office holidays.
3:14This is the formula from the drawing: the dot product of the two vectors, divided by the product of their lengths.
3:21Then it encodes the question and the passages with the same model, and prints each passage's score.
3:28Let's run it.
3:29It prints one score per passage, in the order the passages were listed.
3:33Both password passages score well above the holiday passage, which barely scores at all.
3:39The login credentials passage shares almost no words with the question, and it still scores point five three. The model matched the meaning.
3:47One caution. Point six three is not a sixty three percent chance of being relevant, and different models produce different ranges. Use these scores to rank passages from one model.
3:58Ranking every passage by this score is already a complete search. Let's write it that way.
4:06This version scores every passage in one step.
4:09The matrix product computes all the dot products together, and the division normalizes them, exactly like the cosine function.
4:16Then it sorts the passages by score, highest first, and keeps the top two.
4:22Let's run it.
4:23This time it prints only the two passages that are kept.
4:26The two password passages, highest score first, with the same scores as before.
4:32This is exact nearest neighbor search: every passage is scored, and the best ones are kept. In a real store the passages are embedded ahead of time, so each search only has to embed the question.
4:44Checking every passage works for three of them. It works less well for millions.
4:51Suppose the collection holds ten million vectors instead of three.
4:54Exact search scores all ten million for every question. That is ten million dot products per query, which is often too slow to feel instant.
5:03An approximate index organizes the vectors ahead of time, so that each question checks a small fraction of the collection.
5:10The most common index for this is called HNSW. Let's build one.
5:16This script builds an HNSW index with the hnswlib library.
5:21The index measures cosine on vectors of three hundred eighty four numbers. Adding the passages links each vector to its nearest neighbors, before any question arrives.
5:31A query moves along those links toward the question. This sets how many candidates it keeps as it goes. A wider search finds the true neighbors more often, and takes longer.
5:41The library reports cosine distance, so one minus that distance gives back the similarity we have been using.
5:48Let's run it.
5:50It asks the index for the two nearest passages.
5:53The same two passages come back, with the same scores as exact search.
5:58On three vectors, that only shows the code works. With millions of vectors the index is much faster than exact search, and it can miss a true nearest neighbor.
6:09It can miss one because it does not look everywhere. How often it finds the true neighbors is called recall.
6:16Suppose exact search would return two passages, and the index finds only one of them. Its recall at two is fifty percent.
6:25Raising the search width usually raises recall, and makes every search slower.
6:30So measure recall and speed on the questions your system really gets, before settling on a setting.
6:36Even an exact search can return the wrong kind of passage, and a question about a community pool shows how.
6:45The new question asks whether the community pool is open in winter. To answer it, we need evidence about winter specifically.
6:53One passage gives the pool's summer hours, one closes outdoor facilities from November to March, and one is about yoga.
7:02Let's rank them.
7:03It uses the same model and the same cosine function as before.
7:07The summer hours passage ranks first by a wide margin, point seven six against point four six.
7:14But it only says the pool is open in summer. It is about the right pool, and it does not answer the question.
7:20The closure rule covers the winter months, but only for outdoor facilities, and none of these passages says whether the pool is outdoors.
7:29Similarity measures how close two texts are in meaning. It does not check whether a passage answers the question.
7:37That is a complete vector search, built one step at a time.
7:42A model turns text into vectors, cosine similarity scores them, and an index keeps search fast at scale. What comes back is a ranked list of candidates. Checking whether they answer the question is a separate step.
7:56Next, we look at where passages come from in the first place: how documents are cut into chunks.

Embeddings and Vector Search

Created by Abhyuday DesaiPublished 4 Oct, v1Runs 8 min 8 sec

0 views

About this video

In Episode 01 of Retrieval Explained, we build a vector search one step at a time, with code we run and the numbers it prints. A small model running on the laptop turns a question about resetting a password into a vector of 384 numbers. We compare two vectors by the angle between them, then rank passages by that score, first by checking every passage and then with an HNSW index that checks only some of them. A last question, whether a community pool is open in winter, shows what this kind of search cannot do on its own.

You'll learn how to:

Turn text into an embedding with a local model, and see why the meaning is in the whole vector rather than in any one number
Read cosine similarity as the angle between two arrows, and see why normalizing takes length out of the score
Write exact nearest neighbor search as one matrix product and a sort
Build an HNSW index, and measure recall: how often it finds the true neighbors, and what a wider search costs
See why the most similar passage is not always one that answers the question

This video is for developers starting on retrieval for AI applications who are comfortable reading Python. The code runs from the companion repo, with no API key.

What changed

v104 Oct 2026abhydesai8 min 8 secb9e8014

First published version.

0 views0 likes on this version