The dot product

The dot product

Explanation

We have added vectors and scaled them. There is also a way to multiply two vectors together, and the answer is a single number. It is called the dot product.

The dot product

Definition

For $\vec{v}, \vec{w} \in \mathbb{R}^n$ , the dot product $\vec{v} \cdot \vec{w}$ is

\vec{v} \cdot \vec{w} = v_1 w_1 + v_2 w_2 + \cdots + v_n w_n.

Each component of $\vec{v}$ is multiplied by the component of $\vec{w}$ in the same position, and the $n$ products are added.

In more general settings, where the objects being multiplied need not be columns of numbers, a product of this kind is called an inner product.

Example

Let $\vec{v} = \begin{bmatrix} 3 \\ -1 \\ 4 \\ -2 \end{bmatrix}$ and $\vec{w} = \begin{bmatrix} 5 \\ 6 \\ -2 \\ 3 \end{bmatrix}$ .

Calculate $\vec{v} \cdot \vec{w}$ .

Solution

Three of the four products are negative, since in those positions the paired components have opposite signs:

\begin{align*} \vec{v} \cdot \vec{w} &= \begin{bmatrix} 3 \\ -1 \\ 4 \\ -2 \end{bmatrix} \cdot \begin{bmatrix} 5 \\ 6 \\ -2 \\ 3 \end{bmatrix} \\ &= 3(5) + (-1)(6) + 4(-2) + (-2)(3) \\ &= 15 + (-6) + (-8) + (-6) \\ &= -5. \end{align*}

The result is the single real number $-5$ , not a vector.

Practice questions

5 questions

Let $\vec{v} = \begin{bmatrix} 4 \\ 3 \end{bmatrix}$ and $\vec{w} = \begin{bmatrix} 6 \\ 2 \end{bmatrix}$ .

Calculate $\vec{v} \cdot \vec{w}$ .

Select the correct answer:

$\begin{bmatrix} 24 \\ 6 \end{bmatrix}$

$24$

$26$

$30$

+ 4 more questions

What the dot product says about the angle

Explanation

Computing a dot product from components says nothing about what the result represents. The same number can also be computed from the lengths of the two vectors and the angle between them. Because the two always agree, the lengths and the angle explain what the components compute.

The dot product and the angle between two vectors

Theorem

For non-zero $\vec{v}, \vec{w} \in \mathbb{R}^n$ ,

\vec{v} \cdot \vec{w} = \|\vec{v}\| \|\vec{w}\| \cos(\theta),

where $\theta$ is the angle between the two vectors placed tail to tail at the origin, taken between $0^\circ$ and $180^\circ$ .

Key Point

The size of a dot product depends on both the lengths and the angle. Two short vectors pointing the same way can give a smaller number than two long ones at a wide angle.

Lengths are always positive, so the dot product and $\cos(\theta)$ always have the same sign. The diagram below is two-dimensional, but the result holds in any number of dimensions.

0,0

\vec{v}

\vec{w}

θ

Drag either arrowhead. Pulling it further out changes that length alone; swinging it round changes the angle alone.

‖ \vec{v} ‖ = 4.12

\times

‖ \vec{w} ‖ = 3.16

\times

\cos (58^{\circ}) = 0.537

=

\vec{v} \cdot \vec{w} = 7.00

oppositeright anglesame direction

An acute angle, so the dot product is positive

The components give the same number:

(4.0) (1.0) + (1.0) (3.0) = 7.00

.

The angle $\theta$ is taken to be the smaller of the two angles the vectors form, so $0^\circ \leq \theta \leq 180^\circ$ . The sign of the dot product tells us what kind of angle we have, and the two ends of that range tell us how large the number can get.

$\theta$	$\cos(\theta)$	$\vec{v} \cdot \vec{w}$	The two vectors
$0^\circ$	$1$	$\lVert\vec{v}\rVert \lVert\vec{w}\rVert$	point the same way
acute, below $90^\circ$	positive	positive	point broadly the same way
$90^\circ$	$0$	$0$	are at a right angle
obtuse, above $90^\circ$	negative	negative	point broadly opposite ways
$180^\circ$	$-1$	$-\lVert\vec{v}\rVert \lVert\vec{w}\rVert$	point opposite ways

In particular, taking $\vec{w}$ to be $\vec{v}$ itself puts the angle at $0^\circ$ , so a vector dotted with itself gives its length squared.

Practice questions

4 questions

The non-zero vectors $\vec{v}$ and $\vec{w}$ have lengths $\|\vec{v}\| = 8$ and $\|\vec{w}\| = 3$ , and the angle between them is $60^\circ$ , where $\cos(60^\circ) = \dfrac{1}{2}$ .

What is $\vec{v} \cdot \vec{w}$ ?

Select the correct answer:

$24$

$11$

$12$

$12\sqrt{3}$

+ 3 more questions

The cosine of the angle

Explanation

We've seen that the dot product of two non-zero vectors is the product of their two lengths and the cosine of the angle between them. Both lengths are non-zero, so we can divide by them, which expresses the cosine of the angle in terms of the dot product and the two lengths.

The cosine of the angle between two vectors

Theorem

For non-zero $\vec{v}, \vec{w} \in \mathbb{R}^n$ ,

\cos(\theta) = \dfrac{\vec{v} \cdot \vec{w}}{\|\vec{v}\| \|\vec{w}\|},

where $\theta$ is the angle between $\vec{v}$ and $\vec{w}$ .

Every quantity on the right comes from the components, so the cosine does too.

Making either vector longer multiplies both the dot product and that vector's length by the same factor, so the cosine does not move. It measures direction and nothing else, which is what lets two vectors of very different sizes be compared.

ML Context

In deep learning, a piece of text, an image or a sound is often represented by a vector of numbers called an embedding, learned so that similar inputs are given vectors pointing in similar directions.

Two embeddings are usually compared with the cosine of the angle between them, called cosine similarity: it is how a search engine ranks documents against a query, and how a recommender system finds things close to what someone already chose.

Dividing by the two lengths is deliberate, because the size of an embedding can reflect incidental things such as how long the input was, and that should not decide whether two inputs count as alike.

Example

Two drones are tracked, each velocity recorded in metres per second to the east, to the north and upwards.

The first has velocity $\vec{v} = \begin{bmatrix} 4 \\ 1 \\ -1 \end{bmatrix}$ and the second has velocity $\vec{w} = \begin{bmatrix} -3 \\ 4 \\ 5 \end{bmatrix}$ .

Find $\cos(\theta)$ , where $\theta$ is the angle between the two velocities.

Solution

The cosine needs three numbers: the dot product and each of the two lengths.

We multiply matching components and add the results:

\begin{align*} \vec{v} \cdot \vec{w} &= 4(-3) + 1(4) + (-1)(5) \\ &= -12 + 4 - 5 \\ &= -13. \end{align*}

Each length comes from the squares of its components, and the first square root simplifies because $18 = 9 \times 2$ :

\begin{align*} \|\vec{v}\| &= \sqrt{4^2 + 1^2 + (-1)^2} \\ &= \sqrt{18} \\ &= 3\sqrt{2}. \end{align*}

The same calculation for $\vec{w}$ gives a square root that simplifies in the same way, since $50 = 25 \times 2$ :

\begin{align*} \|\vec{w}\| &= \sqrt{(-3)^2 + 4^2 + 5^2} \\ &= \sqrt{50} \\ &= 5\sqrt{2}. \end{align*}

Substituting all three into the cosine, both lengths carry a factor of $\sqrt{2}$ , so their product is rational: $3\sqrt{2} \times 5\sqrt{2} = 15 \times 2 = 30$ .

\begin{align*} \cos(\theta) &= \dfrac{-13}{3\sqrt{2} \times 5\sqrt{2}} \\ &= -\dfrac{13}{30}, \end{align*}

which is about $-0.43$ .

The cosine is negative, so the angle between the two velocities is obtuse: the drones are heading in broadly opposite directions.

Practice questions

4 questions

Two short articles are each summarised by how many times four particular words appear in them.

The counts for the first article are $\vec{v} = \begin{bmatrix} 1 \\ 2 \\ 2 \\ 4 \end{bmatrix}$ and the counts for the second are $\vec{w} = \begin{bmatrix} 2 \\ 0 \\ 1 \\ 2 \end{bmatrix}$ . Let $\theta$ be the angle between these two vectors.

What is $\cos(\theta)$ ?

Select the correct answer:

$12$

$\dfrac{4}{5}$

$\dfrac{12}{5}$

$4$

+ 3 more questions

Master the linear algebra that powers modern ML

Explanation

Example

Explanation

Explanation

Example

Vectors in Euclidean space