Free preview

83 lessons

Mathematical Foundations for ML

Free preview

Mathematical Foundations for ML · 83 lessons

Build the maths foundations that make ML click

No surprise gaps

Actually remember it

Skip what you know

One subscription. All learning paths included.

Our content is best on a larger screen

Sigma notation

Anatomy of sigma notation

Explanation

When we need to add up a long sequence of terms, writing them all out gets tedious fast. Mathematics has a shorthand for this: sigma notation, named after the capital Greek letter $\Sigma$ (sigma), which acts as a summation operator - it tells us to add up a series of terms.

Sigma notation

Definition

A summation written in sigma notation has five components:

\sum_{i=1}^{4} i^2

$\Sigma$ - the summation operator, meaning "add up".
$i$ - the index variable, a placeholder that takes each integer value in turn.
$i=1$ - the lower limit, the starting value of the index.
$4$ - the upper limit, the ending value of the index.
$i^2$ - the general term, the expression evaluated for each value of the index.

The expression above tells us to evaluate $i^2$ for each integer $i$ from $1$ to $4$ and add the results together.

The index letter is arbitrary

Gotcha

The choice of letter for the index variable does not affect the sum. The letters $i$ , $j$ , $k$ , and $n$ are all common choices. For example, $\displaystyle\sum_{i=1}^{4} i^2$ and $\displaystyle\sum_{k=1}^{4} k^2$ represent exactly the same sum - only the placeholder name has changed.

If the lower limit exceeds the upper limit, there are no terms to add. By convention, such an empty sum equals $0$ - the additive identity.

Example

Identify each component of the following expression:

\sum_{j=1}^{6} (3j - 1)

Solution

Summation operator: $\Sigma$ tells us to add up the terms.
Index variable: $j$ .
Lower limit: $j = 1$ , so $j$ starts at $1$ .
Upper limit: $6$ , so $j$ ends at $6$ .
General term: $3j - 1$ , the expression evaluated for each value of $j$ .

The expression tells us to evaluate $3j - 1$ for each integer $j$ from $1$ to $6$ and add the results together.

Practice questions

4 questions

In the expression $\displaystyle\sum_{j=2}^{7} (3j + 1)$ , what is the general term?

Select the correct answer:

$j$

$3j + 1$

$7$

$j = 2$

+ 3 more questions

Expanding sigma notation

Explanation

Now that we can identify the parts of sigma notation, the next step is to expand a summation - writing out each term individually.

Expanding a sigma expression

Procedure

To expand a sigma expression with lower limit $a$ and upper limit $b$ :

Substitute the lower limit $i = a$ into the general term.
Substitute $i = a + 1$ , then $i = a + 2$ , and so on.
Continue until reaching the upper limit $i = b$ .
Write the results separated by $+$ signs.

Example

Expand each of the following:

$\displaystyle\sum_{i=1}^{5} (2i - 1)$
$\displaystyle\sum_{i=1}^{n} a_i$

Solution

1. We substitute $i = 1, 2, 3, 4, 5$ into the general term $2i - 1$ :

\sum_{i=1}^{5} (2i - 1) = 1 + 3 + 5 + 7 + 9

Full substitution steps

Details

\begin{align*} i = 1: &\quad 2(1) - 1 = 1 \\ i = 2: &\quad 2(2) - 1 = 3 \\ i = 3: &\quad 2(3) - 1 = 5 \\ i = 4: &\quad 2(4) - 1 = 7 \\ i = 5: &\quad 2(5) - 1 = 9 \end{align*}

2. Here the upper limit is the letter $n$ rather than a specific number, so we cannot write every term. We write the first few, use " $\ldots$ " to show the pattern continues, and end with the last term:

\sum_{i=1}^{n} a_i = a_1 + a_2 + \ldots + a_n

Practice questions

4 questions

What is the expansion of $\displaystyle\sum_{i=1}^{4} (i + 2)$ ?

Select the correct answer:

$3 + 4 + 5 + 6$

$1 + 2 + 3 + 4$

$2 + 4 + 6 + 8$

$3 + 5 + 7 + 9$

+ 3 more questions

Writing sums in sigma notation

Explanation

We have seen how to expand a sigma expression into its terms. Now we go in the other direction: given an expanded sum, we write it in sigma notation.

Writing a sum in sigma notation

Procedure

Spot the pattern - how does each term relate to its position? The first term is position $1$ , the second is position $2$ , and so on.
Write the general term - express the pattern as a formula in terms of an index variable $i$ .
Set the limits - the lower limit is the first value of $i$ (usually $1$ ), and the upper limit is the last.

Example

Write $3 + 6 + 9 + 12 + 15$ in sigma notation.

Solution

Step 1: Spot the pattern. We label each term by its position:

Position ( $i$ )	$1$	$2$	$3$	$4$	$5$
Term	$3$	$6$	$9$	$12$	$15$

Each term is $3$ times its position: $3 \times 1 = 3$ , $3 \times 2 = 6$ , $3 \times 3 = 9$ , and so on.

Step 2: Write the general term. The $i$ -th term is $3i$ .

Step 3: Set the limits. The index starts at $i = 1$ (for the first term, $3$ ) and ends at $i = 5$ (for the last term, $15$ ).

Putting it together:

3 + 6 + 9 + 12 + 15 = \sum_{i=1}^{5} 3i

Practice questions

4 questions

Which sigma expression represents $4 + 8 + 12 + 16 + 20$ ?

Select the correct answer:

$\displaystyle\sum_{i=1}^{5} 4i$

$\displaystyle\sum_{i=1}^{5} (i + 4)$

$\displaystyle\sum_{i=1}^{4} 4i$

$\displaystyle\sum_{i=4}^{20} i$

+ 3 more questions

Part of

Summation, product, and indexed notation

See all 9 lessons, what comes first, and what this unlocks.