Free preview

59 lessons

Essential Linear Algebra for ML

Free preview

Essential Linear Algebra for ML · 59 lessons

Master the linear algebra that powers modern ML

No surprise gaps

Actually remember it

Skip what you know

One subscription. All learning paths included.

Our content is best on a larger screen

Eigenvalues and eigenvectors of 2×22 \times 2 matrices

Geometric interpretation

Explanation

Recall that a matrix AA represents a linear transformation: it takes every vector in the plane and moves it somewhere else. We’ll use this idea to build geometric intuition for eigenvalues and eigenvectors by visualising them in 2D.

1234−1−2−3−41234−1−2−3−4
v
Av
λ1=2.21
λ2=0.79

Matrix Controls

Adjust the entries of A with the steppers, or generate a random matrix. Drag the vector v around and view its transformation, Av.

Purple and green lines show the real eigenvector directions of A.

The grid shows the coordinate system. The blue grid is the original, and the red grid is what we get after applying the matrix A to every point in the grid. In other words, it shows how A stretches, shears, or rotates the whole space. (Hidden by default - use Toggle Grid to show or hide.)

The box shows the matrix AA, which you can edit. The arrow v\vec{v} is a test vector you can drag anywhere. The arrow labelled AvA\vec{v} shows where that vector ends up after the transformation. In most directions, the transformation changes both the length and the direction of the vector.

The eigenvectors are different. The purple and green lines mark the special directions where AA does not rotate a vector. If v\vec{v} lies on one of these lines, then AvA\vec{v} stays on the same line. The transformation only stretches, squashes, or flips it. The factor by which it is scaled is the corresponding eigenvalue.

Finally, the grid (toggle it on with the button). The blue grid is the original coordinate system. The red grid shows what happens when we apply AA to every point of the blue grid. Straight lines remain straight, parallel lines remain parallel, but the whole space is stretched, sheared, or compressed. This is the geometric meaning of a linear transformation: it reshapes the entire coordinate system in a uniform way.

Together, the picture ties three ideas: the grid shows how AA reshapes space, the test vector v\vec{v} shows how individual vectors are moved, and the eigenvectors reveal the privileged directions that survive the reshaping without rotation.

Grid explainer

Details

Example

We know that eigenvectors and eigenvalues are defined by the equation:

Av=λv.A\vec{v} = \lambda \vec{v}.

This means that if v\vec{v} is an eigenvector, then applying the matrix AA to it produces a new vector that is parallel to v\vec{v}. The scalar λ\lambda is the eigenvalue, and it tells us how much the vector is stretched or squashed.

How can we spot this in practice? If v\vec{v} and AvA\vec{v} are parallel, then the ratio of their lengths (norms) is exactly the eigenvalue:

λ=Avv.\lambda = \frac{|A\vec{v}|}{|\vec{v}|}.

The game

  • Drag the input vector v\vec{v} anywhere in the plane (drag from the outer edge of the arrow head).
  • The diagram shows both v\vec{v} (blue) and AvA\vec{v} (red).
  • The top‑left box displays v|\vec{v}|, Av|A\vec{v}|, and their ratio.
  • The aim is to adjust v\vec{v} until the two arrows are parallel. When this happens, we’ve found an eigenvector.
  • At that point, the ratio Avv\dfrac{|A\vec{v}|}{|\vec{v}|} gives us the corresponding eigenvalue.

The challenge

The matrix AA:

A=[3102],A = \begin{bmatrix} 3 & 1 \\ 0 & 2 \end{bmatrix},

has integer eigenvalues.

Find both eigenvectors by dragging v\vec{v} until it lines up with AvA\vec{v}. Then read off the two eigenvalues from the ratio.

1234−1−2−3−41234−1−2−3−4
v
Av
|v|=1.41
|Av|=4.47
|Av||v|=3.16
A=[3.001.000.002.00]

For the above matrix AA, we can see that the eigenvalues are λ1=2\lambda_1 = 2 and λ2=3\lambda_2 = 3. This is when both v\vec{v} and AvA \vec{v} are pointing in the same direction (along the eigenvectors) and AvA \vec{v} is simply a scaled version of v\vec{v} (scaled by exactly the corresponding eigenvalue).

More precisely, for:

  • Eigenvalue λ1=2\lambda_1 = 2, the eigenvectors are multiples of [11]\begin{bmatrix} -1 \\ 1 \end{bmatrix}.
  • Eigenvalue λ2=3\lambda_2 = 3, the eigenvectors are multiples of [10]\begin{bmatrix} 1 \\ 0 \end{bmatrix}.

Practice questions

3 questions

The matrix AA:

A=[2036],A = \begin{bmatrix} 2 & 0 \\ 3 & 6 \end{bmatrix},

has integer eigenvalues.

Find both eigenvectors by dragging v\vec{v} until it lines up with AvA\vec{v}. Then read off the two eigenvalues from the ratio.

Which option corresponds to the eigenvalues?

1234−1−2−3−41234−1−2−3−4
v
Av
|v|=1.41
|Av|=3.61
|Av||v|=2.55
A=[2.000.003.006.00]

Select the correct answer:

+ 2 more questions

Computing the eigenvalues and eigenvectors for a 2×22 \times 2 matrix

Explanation

So far, we have found the eigenvalues and eigenvectors separately. In practice, the two always come together: to fully understand how a matrix acts as a transformation, we need both the eigenvalues (the scaling factors) and the eigenvectors (the invariant directions).

The procedure for a 2×22 \times 2 matrix AA is as follows:

  1. Find the eigenvalues

    • Set up the characteristic equation:
    det(AλI)=0\det(A - \lambda I) = 0
    • Expand this determinant to obtain a quadratic equation in λ\lambda.
    • Solve the quadratic to find the eigenvalues.
  2. Find the eigenvectors

    • For each eigenvalue λi\lambda_i, substitute it into the eigenvector equation:
    (AλiI)vi=0.(A - \lambda_i I)\vec{v}_i = \vec{0}.
    • Solve the resulting system of linear equations to find the general form of vi\vec{v}_i.
    • Express the eigenvector in a simple representative form (e.g. by setting a free variable to 1).

This two-step process gives the complete set of eigenvalues and eigenvectors for the matrix.

Recall

Key Point

Eigenvalues tell us how much the matrix stretches space in certain directions. Eigenvectors tell us which directions those are. Together, they give the full picture.

How is this done for larger matrices?

Details

Example

Compute the eigenvalues and eigenvectors of the matrix:

A=[4123].A = \begin{bmatrix} 4 & 1 \\ 2 & 3 \end{bmatrix}.

Solution

Step 1: Find the eigenvalues

Set up the characteristic equation det(AλI)=0\det(A - \lambda I) = 0, by first calculating AλIA - \lambda I:

AλI=[4λ123λ].A - \lambda I = \begin{bmatrix} 4 - \lambda & 1 \\ 2 & 3 - \lambda \end{bmatrix}.

Take the determinant and set to 00:

(4λ)(3λ)(2)(1)=0.(4 - \lambda)(3 - \lambda) - (2)(1) = 0.

Expanding gives the quadratic equation:

λ27λ+10=0.\lambda^2 - 7\lambda + 10 = 0.

Solve the quadratic:

λ=7±49402=7±32.\lambda = \frac{7 \pm \sqrt{49 - 40}}{2} = \frac{7 \pm 3}{2}.

So the eigenvalues are:

λ1=5,λ2=2.\lambda_1 = 5, \quad \lambda_2 = 2.

Step 2: Find the eigenvectors

For λ1=5\lambda_1 = 5:

Solve (A5I)v=0(A - 5I)\vec{v} = \vec{0}:

[1122][v1v2]=[00].\begin{bmatrix} -1 & 1 \\ 2 & -2 \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}.

Which gives the equations:

v1+v2=0,2v12v2=0.-v_1 + v_2 = 0, \quad 2v_1 - 2v_2 = 0.

Both reduce to v1=v2v_1 = v_2.

So the general eigenvector is:

v1=[tt]=t[11],t0.\vec{v}_1 = \begin{bmatrix} t \\ t \end{bmatrix} = t \begin{bmatrix} 1 \\ 1 \end{bmatrix}, \quad t \neq 0.

A representative choice is to set t=1t=1:

v1=[11].\vec{v}_1 = \begin{bmatrix} 1 \\ 1 \end{bmatrix}.

For λ2=2\lambda_2 = 2:

Solve (A2I)v=0(A - 2I)\vec{v} = \vec{0}:

[2121][v1v2]=[00].\begin{bmatrix} 2 & 1 \\ 2 & 1 \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}.

Which gives the equation (both are the same):

2v1+v2=0.2v_1 + v_2 = 0.

And so v2=2v1v_2 = -2v_1.

Let v1=tv_1 = t and so the general eigenvector is:

v2=[t2t]=t[12],t0.\vec{v}_2 = \begin{bmatrix} t \\ -2t \end{bmatrix} = t \begin{bmatrix} 1 \\ -2 \end{bmatrix}, \quad t \neq 0.

A representative choice is to set t=1t=1:

v2=[12].\vec{v}_2 = \begin{bmatrix} 1 \\ -2 \end{bmatrix}.

Putting it together we have:

  • Eigenvalues: λ1=5\lambda_1 = 5, λ2=2\lambda_2 = 2
  • Corresponding eigenvectors:
    • For λ1=5\lambda_1 = 5: v1=[11]\vec{v}_1 = \begin{bmatrix} 1 \\ 1 \end{bmatrix}.
    • For λ2=2\lambda_2 = 2: v2=[12]\vec{v}_2 = \begin{bmatrix} 1 \\ -2 \end{bmatrix}.

Note that any non-zero scalar multiple of each of the above eigenvectors is also an eigenvector for the eigenvalue.

Practice questions

3 questions

Compute the eigenvalues and eigenvectors of the matrix:

A=[0816].A = \begin{bmatrix} 0 & -8 \\ 1 & 6 \end{bmatrix}.

Select the correct answer:

+ 2 more questions

Verifying eigenvalue-eigenvector pairs

Explanation

Once we’ve calculated the eigenvalues and eigenvectors of a matrix, it’s straightforward to verify whether a number λ\lambda and a non-zero vector v\vec{v} actually form an eigenvalue-eigenvector pair for a matrix AA. Simply check if the equation

Av=λvA\vec{v} = \lambda\vec{v}

is satisfied.

In practice, this means:

  1. Multiply the matrix AA by the vector v\vec{v}.
  2. Multiply the vector v\vec{v} by the scalar λ\lambda.
  3. Compare the results. If they are equal, then λ\lambda and v\vec{v} are indeed an eigenvalue and eigenvector for AA.

Since an n×nn \times n matrix has nn eigenvalues (counting multiplicities), we need to verify this relationship for each eigenvalue-eigenvector pair we find.

Example

Let

A=[4213]A = \begin{bmatrix} 4 & 2 \\ 1 & 3 \end{bmatrix}

Below are five candidate eigenvalue-eigenvector pairs for AA:

  1. λ=5\lambda = 5, v=[21]\vec{v} = \begin{bmatrix} 2 \\ 1 \end{bmatrix}
  2. λ=2\lambda = 2, v=[11]\vec{v} = \begin{bmatrix} -1 \\ 1 \end{bmatrix}
  3. λ=4\lambda = 4, v=[10]\vec{v} = \begin{bmatrix} 1 \\ 0 \end{bmatrix}
  4. λ=5\lambda = 5, v=[11]\vec{v} = \begin{bmatrix} 1 \\ 1 \end{bmatrix}
  5. λ=2\lambda = 2, v=[11]\vec{v} = \begin{bmatrix} 1 \\ -1 \end{bmatrix}

Which pairs are valid eigenvalue-eigenvector pairs for AA?

Solution

To verify each pair, we check whether Av=λvA\vec{v} = \lambda\vec{v}.

Pair 1: λ=5\lambda = 5, v=[21]\vec{v} = \begin{bmatrix} 2 \\ 1 \end{bmatrix}

Compute AvA\vec{v}:

[4213][21]=[105]\begin{bmatrix} 4 & 2 \\ 1 & 3 \end{bmatrix} \begin{bmatrix} 2 \\ 1 \end{bmatrix} = \begin{bmatrix} 10 \\ 5 \end{bmatrix}

Compute 5v5\vec{v}:

5[21]=[105]5 \begin{bmatrix} 2 \\ 1 \end{bmatrix} = \begin{bmatrix} 10 \\ 5 \end{bmatrix}

Valid: Yes, Av=5vA\vec{v} = 5\vec{v}.


Pair 2: λ=2\lambda = 2, v=[11]\vec{v} = \begin{bmatrix} -1 \\ 1 \end{bmatrix}

Compute AvA\vec{v}:

[4213][11]=[22]\begin{bmatrix} 4 & 2 \\ 1 & 3 \end{bmatrix} \begin{bmatrix} -1 \\ 1 \end{bmatrix} = \begin{bmatrix} -2 \\ 2 \end{bmatrix}

Compute 2v2\vec{v}:

2[11]=[22]2 \begin{bmatrix} -1 \\ 1 \end{bmatrix} = \begin{bmatrix} -2 \\ 2 \end{bmatrix}

Valid: Yes, Av=2vA\vec{v} = 2\vec{v}.


Pair 3: λ=4\lambda = 4, v=[10]\vec{v} = \begin{bmatrix} 1 \\ 0 \end{bmatrix}

Compute AvA\vec{v}:

[4213][10]=[41]\begin{bmatrix} 4 & 2 \\ 1 & 3 \end{bmatrix} \begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 4 \\ 1 \end{bmatrix}

Compute 4v4\vec{v}:

4[10]=[40]4 \begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 4 \\ 0 \end{bmatrix}

Valid: No, Av4vA\vec{v} \neq 4\vec{v}.


Pair 4: λ=5\lambda = 5, v=[11]\vec{v} = \begin{bmatrix} 1 \\ 1 \end{bmatrix}

Compute AvA\vec{v}:

[4213][11]=[64]\begin{bmatrix} 4 & 2 \\ 1 & 3 \end{bmatrix} \begin{bmatrix} 1 \\ 1 \end{bmatrix} = \begin{bmatrix} 6 \\ 4 \end{bmatrix}

Compute 5v5\vec{v}:

5[11]=[55]5 \begin{bmatrix} 1 \\ 1 \end{bmatrix} = \begin{bmatrix} 5 \\ 5 \end{bmatrix}

Valid: No, Av5vA\vec{v} \neq 5\vec{v}.


Pair 5: λ=2\lambda = 2, v=[11]\vec{v} = \begin{bmatrix} 1 \\ -1 \end{bmatrix}

Compute AvA\vec{v}:

[4213][11]=[22]\begin{bmatrix} 4 & 2 \\ 1 & 3 \end{bmatrix} \begin{bmatrix} 1 \\ -1 \end{bmatrix} = \begin{bmatrix} 2 \\ -2 \end{bmatrix}

Compute 2v2\vec{v}:

2[11]=[22]2 \begin{bmatrix} 1 \\ -1 \end{bmatrix} = \begin{bmatrix} 2 \\ -2 \end{bmatrix}

Valid: Yes, Av=2vA\vec{v} = 2\vec{v}.


So, the valid eigenvalue-eigenvector pairs for AA are:

  • Pair 1: λ=5\lambda = 5, v=[21]\vec{v} = \begin{bmatrix} 2 \\ 1 \end{bmatrix}
  • Pair 2: λ=2\lambda = 2, v=[11]\vec{v} = \begin{bmatrix} -1 \\ 1 \end{bmatrix}
  • Pair 5: λ=2\lambda = 2, v=[11]\vec{v} = \begin{bmatrix} 1 \\ -1 \end{bmatrix}
Why did we find 33 valid pairs?

Details

Practice questions

3 questions

Let

B=[6231]B = \begin{bmatrix} 6 & -2 \\ 3 & 1 \end{bmatrix}

Below are three candidate eigenvalue-eigenvector pairs for BB:

  1. λ=4\lambda = 4, v=[11]\vec{v} = \begin{bmatrix} 1 \\ 1 \end{bmatrix}
  2. λ=3\lambda = 3, v=[23]\vec{v} = \begin{bmatrix} 2 \\ 3 \end{bmatrix}
  3. λ=4\lambda = 4, v=[23]\vec{v} = \begin{bmatrix} 2 \\ 3 \end{bmatrix}

Which of the following lists all the valid eigenvalue-eigenvector pairs for BB?

Select the correct answer:

+ 2 more questions