Once everything is a vector, search becomes geometry: find the stored vectors closest to the question's vector.
4 minute read
In short
Vector search returns the stored vectors closest to the question's vector, usually the top k by cosine similarity.
Exact search checks everything; approximate search checks a small part using an index like HNSW or IVF.
Approximate search trades a little recall for a lot of speed.
In plain words
Keyword search finds documents that share words with your question. Vector search finds documents that share meaning, even with no words in common. It works in three moves: turn every document into a vector once, turn the question into a vector when it arrives, and return the documents whose vectors are closest.
'Closest' usually means highest cosine similarity. The few best matches are called the nearest neighbours, and the number you ask for is k, as in 'top-k'.
Worked example: something hot to drink
Use the four foods from the last lesson as our documents. A user asks for 'something hot to drink'. Suppose the embedding model turns that into [0.9, 0.8, 0.0]: hot, a drink, not solid.
1
Score every document against the question
Cosine similarity between the question vector and each stored vector.
With k = 2 the search returns chai and coffee. Samosa is hot but not a drink, so it falls well behind.
top-2: chai, coffee
Notice the question never contained the words chai or coffee. That is the whole point of searching by meaning.
Your turn
You have 10 million documents. Comparing the question with every one, as we just did, is called exact search. What's the main problem with it?
How real systems make it fast
Real systems use approximate nearest neighbour search. Instead of checking everything, they build an index that lets them check a small, promising part. The most common index, HNSW, links each vector to its neighbours so a search can hop towards the answer. Another, IVF, groups vectors into clusters and only searches the closest few.
The trade is always the same: much faster, slightly less accurate. Now and then the true best match is missed. The share of true top results that the index actually finds is called recall, and you tune the index to balance recall against speed.
One practical detail. If every vector is scaled to length 1, the dot product and cosine similarity give the same ranking, and the dot product is cheaper. Many embedding models already return length-1 vectors for this reason.
Common mistakes
Using vector search alone for exact terms like product codes or error IDs, where keyword search is much better.
Forgetting that approximate search can miss the true nearest neighbour; check recall against exact search.
Re-embedding the query with a different model from the one that embedded the documents.
Check yourself
Answer in your own words, out loud or on paper, before you open the model answer.
How does vector search differ from keyword search?
Keyword search, like BM25, matches the exact words in the query against documents, weighting rare words more. It's precise for names, codes and exact phrases but misses documents that use different words for the same idea. Vector search embeds the query and documents into the same space and returns the documents whose vectors are closest, usually by cosine similarity, so it finds matches by meaning even with no shared words. At scale we use approximate nearest neighbour indexes like HNSW, trading a little recall for big speed gains. In practice the best systems combine both, often with a reranker, because each catches what the other misses.
More to think about
What is approximate nearest neighbour search, and what do you give up by using it?
Your vector search misses obvious matches. What would you check?