In linear algebra, one is often interested in the canonical forms of a linear transformation. Given a particularly nice basis for the vector spaces in which one is working, the matrix of a linear transformation may also be particularly nice, revealing some information about how the transformation operates on the vector space.
The spectral theorem provides a sufficient criterion for the existence of a particular canonical form. Specifically, the spectral theorem states that if equals the transpose of , then is diagonalizable: there exists an invertible matrix such that is a diagonal matrix. Recall that a diagonal matrix is any matrix for which all entries off the main diagonal (the diagonal from top left to bottom right) are zero.
A matrix with entries in is called symmetric if . The spectral theorem states that any symmetric matrix is diagonalizable.
Consider the matrix
The eigenvalues of this matrix are , with corresponding eigenvectors and . These eigenvectors form a basis for , with change of basis matrix
sending the standard basis vector to and to . In the basis , the matrix for the linear transformation described by is precisely
This diagonal matrix is easier to parse than ; looking at it immediately tells one that acts on by scaling two axes, one by a factor of and one by a factor of (i.e. reflection).
A matrix is called diagonalizable if there is a basis in which the linear transformation described by has a diagonal matrix, i.e. a matrix whose entries off the main diagonal (the diagonal from top left to bottom right) are all zero. Equivalently, is diagonalizable if and only if there exists an invertible matrix such that is a diagonal matrix. Diagonalizable matrices are easier to work with than non-diagonalizable matrices since they can be placed in this canonical diagonal form with a change of basis.
Let's start off by proving the following useful lemma:
Lemma: All the eigenvalues of a real symmetric matrix are real.
Let be an eigenvalue of with corresponding eigenvector . Now I will show that by evaluating in two ways:
On comparing equations (1) and (2), it's clear that so
Proof by Induction
For and are scalars, so where Thus, we can pick any non-zero real scalar to form a basis for
Every symmetric matrix is diagonalisable for So, let be symmetric. We break the induction part into 3 steps:
We can choose eigenvalue by the above lemma (see comment for details), and let's normalize a corresponding eigenvector such that . Now we can extend to a basis for and apply the Gram-Schmidt procedure to obtain an orthonormal basis for .
Define . Now the columns of are orthonormal, so is an orthogonal matrix. That is, . Also, define then so is symmetric. Now we use this to show how can be written in block matrix form. So, pre-multiplying the standard basis vector by gives So, has the matrix block form where is symmetric.
Now, by the induction hypothesis, there exists a diagonal matrix and orthogonal matrix such that . Now define We aim to show that is orthogonal and is diagonal. For the first part, Finally, So there exists an orthogonal matrix such that is diagonal, which is equivalent to saying that there exists an orthonormal basis for consisting of eigenvalues of
(Note: The proof is yet to be peer-reviewed, so it may contain errors.)