epochs

Learn · Module 01 · Python and maths toolkit · Lesson 1 of 5

Vectors and dot products

Almost everything in machine learning is a list of numbers, and the most common thing done to two lists is the dot product.

4 minute read

In short
  • A vector is a list of numbers describing something.
  • The dot product multiplies matching numbers and adds them; bigger means the vectors agree more.
  • Cosine similarity is the dot product divided by both lengths, so it measures direction only.

In plain words

A vector is a list of numbers that describes something. A movie might be [how much action, how much comedy]. A word, a picture or a user can all be turned into vectors, which is what lets a computer compare them.

You can add two vectors (add matching numbers) and scale one (multiply every number). The most important operation is the dot product: multiply matching numbers and add up the results. It measures how much two vectors agree.

Worked example: recommending a movie

A user's taste is [3, 1]: they love action (3) and mildly like comedy (1). Two movies are described the same way.

  1. 1
    Movie A is mostly action: [0.9, 0.2]

    Multiply matching numbers and add.

    3×0.9 + 1×0.2 = 2.9
  2. 2
    Movie B is mostly comedy: [0.1, 0.8]
    3×0.1 + 1×0.8 = 1.1
  3. 3
    Recommend the higher score

    Movie A fits this user's taste better, so it goes first.

    A: 2.9   B: 1.1

Worked example: length and angle

A vector's length is the square root of its dot product with itself. Dividing the dot product by both lengths gives the cosine of the angle between them, which ignores size and keeps only direction.

  1. 1
    Two vectors
    a = [2, 1]   b = [1, 3]
  2. 2
    Dot product
    2×1 + 1×3 = 5
  3. 3
    Lengths
    |a| = √(4 + 1) = 2.236
    |b| = √(1 + 9) = 3.162
  4. 4
    Cosine

    0.707 is the cosine of 45°, so the two vectors point 45° apart.

    5 ÷ (2.236 × 3.162) = 0.707
Your turn

What is the dot product of [1, 2] and [−1, −2]?

Where you'll see this

A neural network layer is mostly dot products: each output is the dot product of the input with a row of learned weights. Embedding search compares a question with documents using the dot product or cosine. Attention scores are dot products between queries and keys.

Common mistakes

  • Comparing raw dot products between vectors of very different lengths. Long vectors win regardless of direction; normalise first or use cosine.
  • Saying a dot product of zero means the vectors are 'unrelated' in every sense. It only means they point at right angles.
  • Forgetting that cosine similarity ignores length, so a short and a long document on the same topic score the same.
Check yourself

Answer in your own words, out loud or on paper, before you open the model answer.

What does the dot product of two vectors tell you?

The dot product multiplies matching entries and adds them up. Geometrically it equals the two lengths multiplied by the cosine of the angle between them, so it measures how much the vectors point the same way, scaled by their sizes. Positive means broadly the same direction, zero means at right angles, negative means opposite. If I divide by both lengths I get cosine similarity, which keeps only the direction, and that's what we use to compare embeddings. It shows up everywhere in ML: a neuron's weighted sum is a dot product, attention scores are dot products between queries and keys, and recommendation scores are often a user vector dotted with an item vector.

More to think about
  • When would you use cosine similarity instead of the dot product?
  • What does a negative dot product mean?

Go further

Was this lesson clear?