Eigenvalues, eigenvectors and singular value decomposition

Learn how to set up and solve the characteristic equation for a $2\times2$ matrix in order to calculate its eigenvalues, understanding their significance in matrix transformations and machine learning applications.

Learn how to set up and solve the eigenvector equation $(A - \lambda I)\vec{v} = \vec{0}$ for a given $2 \times 2$ matrix and its eigenvalues, and understand why eigenvectors are only determined up to a non-zero scalar multiple.

Learn how to interpret, calculate and verify the eigenvalues and eigenvectors of a $2 \times 2$ matrix, understanding both their geometric meaning as invariant directions and their computation using the characteristic equation.

Eigendecomposition expresses a diagonalisable matrix as $A = P D P^{-1}$, revealing that in the eigenvector basis $A$ acts as simple scaling, so applying $A$ to any vector can be understood as changing basis to the eigenbasis, scaling by the eigenvalues, then changing back.

Understand why the classical eigendecomposition fails for many matrices, see how singular values are defined as the square roots of the eigenvalues of $A^T A$, and recognise that $A^T A$ is always square, symmetric, and has non-negative eigenvalues, ensuring that singular values are always real and non-negative for any matrix.

Learn how any real matrix can be factorised into orthogonal matrices and a diagonal matrix using the singular value decomposition, and how to compute its right and left singular vectors and singular values via the eigenvectors and eigenvalues of $A^T A$ and $A A^T$.

Learn how to calculate the singular value decomposition of small matrices by finding singular values and singular vectors, assembling the $U$, $\Sigma$, and $V$ matrices, and using orthonormal completion when matrices are rectangular.

See how the singular value decomposition breaks any matrix transformation into an initial rotation, a stretch along perpendicular directions, and a final rotation, and how the singular values and vectors correspond to the axes and shape of the transformed unit circle, with zero singular values indicating collapsed directions.