The Dot Product — A One-Page Primer

What It Is

The dot product is a way to multiply two lists of numbers together and get a single number back. That’s it.

Say you have two lists (mathematicians call them vectors):

A = [1, 2, 3]
B = [4, 5, 6]

The dot product is: (1 × 4) + (2 × 5) + (3 × 6) = 4 + 10 + 18 = 32

Multiply each pair of matching entries, then add them all up. One number out.

Why It Matters in AI

In AI, every word, image, and concept gets turned into a list of numbers (a vector). The dot product is how the system measures similarity between two things. If two vectors point in roughly the same direction, their dot product is large. If they’re unrelated, it’s close to zero. If they point in opposite directions, it’s negative.

When you read about attention in a Transformer — the mechanism that lets AI figure out which words in a sentence are relevant to each other — here’s what’s actually happening under the hood:

Each word gets turned into a Query vector (“what am I looking for?”) and a Key vector (“what do I contain?”).
The system computes the dot product between every Query and every Key.
High dot product = these two words are relevant to each other. Low = they’re not.

That’s the core of it. The dot product is the operation that lets a Transformer decide that in “The cat sat on the mat because it was tired,” the word “it” is most similar to “cat” and not “mat.”

The Formal Definition

For two vectors of length $n$ :

$\mathbf{A} \cdot \mathbf{B} = A_1 B_1 + A_2 B_2 + \cdots + A_n B_n$

Or more compactly: the sum of the element-wise products.

In the Transformer paper, this shows up in the attention formula:

$\text{Attention}(Q, K, V) = \text{softmax}\!\left(\frac{QK^T}{\sqrt{d_k}}\right)V$

That $QK^T$ in the middle? That’s a dot product between every Query and every Key. The rest is just scaling and normalization.

Want the Deep Dive?

If you want to really understand linear algebra — dot products, vectors, matrices, and all the math that underpins AI — pick up No Bullshit Guide to Linear Algebra by Ivan Savov. It’s the clearest explanation of the subject out there, written for people who want to understand, not just pass an exam.

Primer by Darrell Thomas, 2026