Free preview
59 lessons
Essential Linear Algebra for ML
Free preview
Essential Linear Algebra for ML · 59 lessons
No surprise gaps
Actually remember it
Skip what you know
One subscription. All learning paths included.
Our content is best on a larger screen
Geometric interpretation
Recall that a matrix represents a linear transformation: it takes every vector in the plane and moves it somewhere else. We’ll use this idea to build geometric intuition for eigenvalues and eigenvectors by visualising them in 2D.
Adjust the entries of
Purple and green lines show the real eigenvector directions of
The grid shows the coordinate system. The blue grid is the original,
and the red grid is what we get after applying the matrix
The box shows the matrix , which you can edit. The arrow is a test vector you can drag anywhere. The arrow labelled shows where that vector ends up after the transformation. In most directions, the transformation changes both the length and the direction of the vector.
The eigenvectors are different. The purple and green lines mark the special directions where does not rotate a vector. If lies on one of these lines, then stays on the same line. The transformation only stretches, squashes, or flips it. The factor by which it is scaled is the corresponding eigenvalue.
Finally, the grid (toggle it on with the button). The blue grid is the original coordinate system. The red grid shows what happens when we apply to every point of the blue grid. Straight lines remain straight, parallel lines remain parallel, but the whole space is stretched, sheared, or compressed. This is the geometric meaning of a linear transformation: it reshapes the entire coordinate system in a uniform way.
Together, the picture ties three ideas: the grid shows how reshapes space, the test vector shows how individual vectors are moved, and the eigenvectors reveal the privileged directions that survive the reshaping without rotation.
Details
We know that eigenvectors and eigenvalues are defined by the equation:
This means that if is an eigenvector, then applying the matrix to it produces a new vector that is parallel to . The scalar is the eigenvalue, and it tells us how much the vector is stretched or squashed.
How can we spot this in practice? If and are parallel, then the ratio of their lengths (norms) is exactly the eigenvalue:
The game
The challenge
The matrix :
has integer eigenvalues.
Find both eigenvectors by dragging until it lines up with . Then read off the two eigenvalues from the ratio.
For the above matrix , we can see that the eigenvalues are and . This is when both and are pointing in the same direction (along the eigenvectors) and is simply a scaled version of (scaled by exactly the corresponding eigenvalue).
More precisely, for:
Practice questions
3 questions
The matrix :
has integer eigenvalues.
Find both eigenvectors by dragging until it lines up with . Then read off the two eigenvalues from the ratio.
Which option corresponds to the eigenvalues?
Select the correct answer:
+ 2 more questions
Computing the eigenvalues and eigenvectors for a matrix
So far, we have found the eigenvalues and eigenvectors separately. In practice, the two always come together: to fully understand how a matrix acts as a transformation, we need both the eigenvalues (the scaling factors) and the eigenvectors (the invariant directions).
The procedure for a matrix is as follows:
Find the eigenvalues
Find the eigenvectors
This two-step process gives the complete set of eigenvalues and eigenvectors for the matrix.
Key Point
Eigenvalues tell us how much the matrix stretches space in certain directions. Eigenvectors tell us which directions those are. Together, they give the full picture.
Details
Compute the eigenvalues and eigenvectors of the matrix:
Solution
Step 1: Find the eigenvalues
Set up the characteristic equation , by first calculating :
Take the determinant and set to :
Expanding gives the quadratic equation:
Solve the quadratic:
So the eigenvalues are:
Step 2: Find the eigenvectors
For :
Solve :
Which gives the equations:
Both reduce to .
So the general eigenvector is:
A representative choice is to set :
For :
Solve :
Which gives the equation (both are the same):
And so .
Let and so the general eigenvector is:
A representative choice is to set :
Putting it together we have:
Note that any non-zero scalar multiple of each of the above eigenvectors is also an eigenvector for the eigenvalue.
Practice questions
3 questions
Compute the eigenvalues and eigenvectors of the matrix:
Select the correct answer:
+ 2 more questions
Verifying eigenvalue-eigenvector pairs
Once we’ve calculated the eigenvalues and eigenvectors of a matrix, it’s straightforward to verify whether a number and a non-zero vector actually form an eigenvalue-eigenvector pair for a matrix . Simply check if the equation
is satisfied.
In practice, this means:
Since an matrix has eigenvalues (counting multiplicities), we need to verify this relationship for each eigenvalue-eigenvector pair we find.
Let
Below are five candidate eigenvalue-eigenvector pairs for :
Which pairs are valid eigenvalue-eigenvector pairs for ?
Solution
To verify each pair, we check whether .
Pair 1: ,
Compute :
Compute :
Valid: Yes, .
Pair 2: ,
Compute :
Compute :
Valid: Yes, .
Pair 3: ,
Compute :
Compute :
Valid: No, .
Pair 4: ,
Compute :
Compute :
Valid: No, .
Pair 5: ,
Compute :
Compute :
Valid: Yes, .
So, the valid eigenvalue-eigenvector pairs for are:
Details
Practice questions
3 questions
Let
Below are three candidate eigenvalue-eigenvector pairs for :
Which of the following lists all the valid eigenvalue-eigenvector pairs for ?
Select the correct answer:
+ 2 more questions