Dot product and vector norm

So far, we've seen how to multiply a vector by a scalar to stretch or shrink it.

But can we multiply two vectors together?

It turns out there is a meaningful way to do this - though it's not the kind of multiplication we're used to.

The dot product (also called the scalar product) of two vectors $\vec{v}$ and $\vec{w}$ is defined as:

$$\vec{v} \cdot \vec{w} = |\vec{v}| |\vec{w}| \cos(\theta)$$

where $|\vec{v}|$ and $|\vec{w}|$ is the length or magnitude of the vectors $|\vec{v}|$ and $|\vec{w}|$ and $\theta$ is the angle (in degrees or radians) made between the two vectors when both of their "tails" are placed at the origin.

You might be wondering: why define multiplication this way? Why not just multiply vectors component by component, like we did for addition or scalar multiplication?

The answer is that the dot product captures something geometric - it measures how much two vectors "point in the same direction." The sign of the dot product indicates whether the vectors point in the same direction (positive), are perpendicular (zero), or point in opposite directions (negative).

Have a play around with the interactive visualisation below. Notice how the dot product changes based on the norms of the vectors and the angle between them.

---

Note: When using the formula $\vec{v} \cdot \vec{w} = |\vec{v}||\vec{w}|\cos(\theta)$, it doesn’t matter whether we measure the smaller angle between the vectors or the larger reflex angle - the result will be the same. This is because the cosine function is symmetric around $180^\circ$, i.e. $\cos(\theta) = \cos(360^\circ - \theta)$. The dot product depends only on the value of $\cos(\theta)$, not on the specific way we measure the angle between the vectors.

Let $\vec{v}$ and $\vec{w}$ be vectors with $|\vec{v}| = 7$ and $|\vec{w}| = \sqrt{3}$, respectively.

Suppose:

• $\vec{v}$ makes an angle of $20^\circ$ with the positive $x$-axis.
• $\vec{w}$ sits at an angle of $+30^\circ$ below the negative $x$-axis.

Calculate $\vec{v} \cdot \vec{w}$. Round your answer to three decimal places.

Solution

To compute the dot product, $\vec{v} \cdot \vec{w}$, we use the formula:

$$\vec{v} \cdot \vec{w} = |\vec{v}| |\vec{w}| \cos(\theta)$$

where $\theta$ is the angle between the two vectors, measured when both are placed tail-to-tail at the origin.

Visualising the two vectors will help us calculate the angle $\theta$ between them:

Let’s think through the geometry.

• $\vec{v}$ is $20^\circ$ above the positive $x$-axis.
• $\vec{w}$ is $30^\circ$ below the negative $x$-axis, which means it's at an angle of $180^\circ + 30^\circ = 210^\circ$ from the positive $x$-axis.

So the angle between the two vectors is:

$$\theta = 210^\circ - 20^\circ = 190^\circ.$$

Now, substitute into the formula:

$$\vec{v} \cdot \vec{w} = 7\sqrt{3} \cos(190^\circ)$$

Making sure our calculator is set in degrees, we have:

$$\vec{v} \cdot \vec{w} = 7\sqrt{3} \cdot \cos(190^\circ) = -11.940$$

There’s another, more practical way to compute the dot product between two vectors - by working directly with their components.

Suppose we have two vectors in $\mathbb{R}^2$:

$$\vec{v} = \begin{bmatrix} v_1 \\ v_2 \end{bmatrix}, \quad \vec{w} = \begin{bmatrix} w_1 \\ w_2 \end{bmatrix}.$$

Then their dot product is:

$$\vec{v} \cdot \vec{w} = v_1w_1 + v_2w_2.$$

That is, we simply multiply the components element-wise and add them up.

This formula generalises naturally to higher dimensions. For two vectors in $\mathbb{R}^n$:

$$\vec{v} = \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix}, \quad \vec{w} = \begin{bmatrix} w_1 \\ w_2 \\ \vdots \\ w_n \end{bmatrix},$$

the dot product is:

$$\vec{v} \cdot \vec{w} = v_1w_1 + v_2w_2 + \dots + v_nw_n.$$

This is sometimes referred to as the inner product, especially in more abstract or general mathematical settings. In regular (Euclidean) space, the dot product and inner product are the same thing.

Let $\vec{v} = \begin{bmatrix} 1 \\ 3 \\ -2 \end{bmatrix}$ and $\vec{w} = \begin{bmatrix} -2 \\ 2 \\ -2 \end{bmatrix}$.

Calculate the dot product $\vec{v} \cdot \vec{w}$.

Solution

We use the formula for the dot product in $\mathbb{R}^3$:

$$\vec{v} \cdot \vec{w} = v_1w_1 + v_2w_2 + v_3w_3$$

Substituting the values:

$$\begin{align*} \vec{v} \cdot \vec{w} &= (1)(-2) + (3)(2) + (-2)(-2) \\ &= -2 + 6 + 4 \\ &= 8. \end{align*}$$

Earlier, we used the Pythagorean theorem to define the length (also called the magnitude or norm) of a vector in terms of its components.

Now that we’ve introduced the dot product, we can express the same idea in a more compact and general way.

Let’s start in two dimensions. For a vector:

$$\vec{v} = \begin{bmatrix} v_1 \\ v_2 \end{bmatrix},$$

we previously defined its length as:

$$|\vec{v}| = \sqrt{v_1^2 + v_2^2}.$$

But from the definition of the dot product, we also have:

$$\vec{v} \cdot \vec{v} = v_1^2 + v_2^2.$$

So we can rewrite the length as:

$$|\vec{v}| = \sqrt{\vec{v} \cdot \vec{v}}.$$

This formulation is useful because it expresses the length compactly using the dot product. For a vector in $\mathbb{R}^n$:

$$\vec{v} = \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix},$$

the length (or norm) is:

$$||\vec{v}|| = \sqrt{\vec{v} \cdot \vec{v}} = \sqrt{v_1^2 + v_2^2 + \dots + v_n^2}.$$

We often refer to this as the vector norm.

Recall that normalising a vector means converting it into a unit vector - a vector that points in the same direction as the original vector but has length $1.$

We do this by dividing the vector by its own norm:

$$\hat{\vec{v}} = \frac{\vec{v}}{||\vec{v}||}.$$

The little hat on top of $\vec{v}$ is standard notation for a unit vector - it tells us that the vector has been normalised.

To check that this really gives a unit vector, we take the norm of the normalised vector and confirm that it equals $1$:

$$||\hat{\vec{v}}|| = \left\|\frac{\vec{v}}{||\vec{v}||} \right\| = \frac{1}{||\vec{v}||} \cdot ||\vec{v}|| = 1.$$

So $\hat{\vec{v}}$ has length $1$ and keeps the direction of $\vec{v}$ - exactly what we want from a normalised vector.

Suppose we have a feature vector representing a data point in a machine learning problem:

$$\vec{v} = \begin{bmatrix} -3 \\ -1 \\ 2 \end{bmatrix}$$

Calculate the norm of the feature vector, and normalise it (convert it to a unit vector).

Solution

We know that in $3$-dimensions,

$$\begin{align*} \|\vec{v}\| &= \sqrt{\vec{v} \cdot \vec{v}} \\ &= \sqrt{v_1^2 + v_2^2 + v_3^2} \\ &= \sqrt{(-3)^2 + (-1)^2 + (2)^2} \\ &= \sqrt{9 + 1 + 4} \\ &= \sqrt{14}. \end{align*}$$

Normalising the vector gives:

$$\begin{align*} \hat{\vec{v}} &= \frac{\vec{v}}{\|\vec{v}\|} \\ &= \frac{1}{\sqrt{14}} \begin{bmatrix} -3 \\ -1 \\ 2 \end{bmatrix}. \end{align*}$$

Converting vectors to unit length ensures that only the direction (the pattern of features) matters, not the overall magnitude. This means our comparisons are not affected by the scale or units of measurement used for each feature.