Matrix Decompositions

By Victor Arce

A single matrix can be hard to reason about. Decompositions factor it into a product of simpler matrices — pure rotations, scalings, or triangular steps — that are each easy to understand and to compute with. Four show up constantly.

Below we rebuild M = [[2, 1], [1, 2]] from simple pieces and look at its eigenvectors.

Eigenvectors: directions that only scale

Most vectors change direction when a matrix hits them. A few special ones don't — they only stretch. Those are the eigenvectors, and the stretch factor is the eigenvalue λ. For our M, the vector (1, 1) maps to (3, 3) — same direction, ×3, so λ = 3; (1, −1) stays put in length, λ = 1. The eigendecomposition M = Q Λ Q⁻¹ uses the eigenvectors as a basis in which M is just a diagonal scaling Λ.

SVD: rotate · scale · rotate

The Singular Value Decomposition factors any matrix (even non-square, even non-invertible) as

M = U Σ Vᵀ

a rotation Vᵀ, then a scaling Σ along the axes (its diagonal entries are the singular values), then another rotation U. The animation runs exactly those three stages and lands back on M. SVD is the workhorse behind low-rank approximation, PCA, and image compression — keep the largest singular values and you keep most of the matrix.

LU and QR: the workhorses of computation

• LU M = L·U splits a matrix into a lower- and an upper-triangular factor. It's exactly what Gaussian elimination produces, and it makes solving Mx = b cheap: solve two triangular systems instead of inverting M.
• QR M = Q·R splits it into an orthonormal matrix Q (a rotation/reflection) times an upper-triangular R. It's the stable way to solve least-squares problems and the engine inside many eigenvalue algorithms.

Each one trades a hard matrix for a couple of easy ones — that's the whole point.

Back to Matrix Math