Fundamental Subspaces

The fundamental subspaces are four vector spaces defined by a given \(m \times n\) matrix \(A\) (and its transpose): the column space and nullspace (or kernel) of \(A\), the column space of \(A^T\) \((\)also called the row space of \(A),\) and the nullspace of \(A^T\) \((\)also called the left nullspace of \(A).\)

The fundamental subspaces are useful for a number of linear algebra applications, including analyzing the rank of a matrix. The subspaces are also closely related by the fundamental theorem of linear algebra.

The column space of a matrix \(A\) is the vector space formed by the columns of \(A\), essentially meaning all linear combinations of the columns of \(A\). Equivalently, the column space consists of all matrices \(Ax\) for some vector \(x\).

For this reason, the column space is also known as the image of \(A\) \(\big(\)denoted \(\text{im}(A)\big),\) as it is the result when \(A\) is viewed as a linear transformation of the vector space \(\mathbb{R}^m\) \((\text{where } m\) is the number of rows of \(A).\) In particular, the image of \(A\) is necessarily a subspace of \(\mathbb{R}^m\), hence the term "fundamental subspace."

For example, consider the matrix

\[A = \begin{pmatrix} 1 & 2 & 3 & 3 \\ 2 & 0 & 6 & 2 \\ 3 & 4 & 9 & 7\end{pmatrix}.\]

Then the column vectors are

\[\begin{pmatrix}1\\2\\3\end{pmatrix}, \begin{pmatrix}2\\0\\4\end{pmatrix}, \begin{pmatrix}3\\6\\9\end{pmatrix},\begin{pmatrix}3\\2\\7\end{pmatrix},\]

which form a vector space that has a basis of \(\left\{\begin{pmatrix}1\\2\\3\end{pmatrix}, \begin{pmatrix}2\\0\\4\end{pmatrix}\right\}\). As there are 2 vectors in this basis, the dimension of the column space is 2; hence, the rank of \(A\) is 2.

The nullspace or kernel of a matrix \(A\) \(\big(\)denoted \(\text{ker}(A)\big)\) is the set of all vectors \(x\) for which

\[Ax = 0.\]

If \(x\) and \(y\) are in the nullspace, then \(c_1x+c_2y\) is also in the nullspace as

\[A(c_1x + c_2y) = c_1(Ax) + c_2(Ay) = 0 + 0 = 0,\]

so the nullspace is in fact a vector space. When \(A\) is viewed as a linear transformation, the nullspace is the subspace of \(\mathbb{R}^n\) that is sent to 0 under the map \(A\), hence the term "fundamental subspace."

For example, consider the matrix

\[A = \begin{pmatrix} 1 & 2 & 3 & 3 \\ 2 & 0 & 6 & 2 \\ 3 & 4 & 9 & 7\end{pmatrix}.\]

The nullspace consists of all vectors \(x\) such that \(Ax = 0\). This defines a system of linear equations that can be solved to give the family of solutions

\[\begin{pmatrix} -x-3y\\-x\\y\\x\end{pmatrix}, x,y \in \mathbb{R},\]

which defines a vector space with basis \(\left\{\begin{pmatrix}-1\\-1\\0\\1\end{pmatrix}, \begin{pmatrix}-3\\0\\1\\0\end{pmatrix}\right\}\). As there are two vectors in this basis, the dimension of the nullspace is 2.

The row space and left nullspace are defined as the column space and nullspace of \(A^T\), the transpose of \(A\), respectively. In this way, they are subspaces of \(\mathbb{R}^n\).

The fundamental theorem of linear algebra relates all four of the fundamental subspaces in a number of different ways. There are main parts to the theorem:

Part 1:
The first part of the fundamental theorem of linear algebra relates the dimensions of the four fundamental subspaces:

The column and row spaces of an \(m \times n\) matrix \(A\) both have dimension \(r\), the rank of the matrix. The nullspace has dimension \(n-r\), and the left nullspace has dimension \(m-r\).

This is illustrated by the example in previous sections: the dimension of the column space of

\[A = \begin{pmatrix} 1 & 2 & 3 & 3 \\ 2 & 0 & 6 & 2 \\ 3 & 4 & 9 & 7\end{pmatrix}\]

is 2, and the dimension of the nullspace of \(A\) is \(n - r = 4 - 2 = 2\).

This first part of the fundamental theorem of linear algebra is sometimes referred to by name as the rank-nullity theorem.

Part 2:
The second part of the fundamental theorem of linear algebra relates the fundamental subspaces more directly:

The nullspace and row space are orthogonal. The left nullspace and the column space are also orthogonal.

In other words, if \(v\) is in the nullspace of \(A\) and \(w\) is in the row space of \(A\), the dot product \(v \cdot w\) is 0. This is true because any vector in the nullspace is orthogonal to each row vector by definition, so it is also orthogonal to any linear combination of them.

Part 3:
The final part of the fundamental theorem of linear algebra constructs an orthonormal basis, and demonstrates a singular value decomposition: any matrix \(M\) can be written in the form \(U\sum V^T\), where

• \(U\) is an \(m \times m\) unitary matrix;
• \(\sum\) is an \(m \times n\) matrix with nonnegative values on the diagonal;
• \(V\) is an \(n \times n\) unitary matrix.

This part of the fundamental theorem allows one to immediately find a basis of the subspace in question.

This can be summarized in the following table:

Or it can also be summarized in the following image, where the arrows represent the results of multiplication: The relationship between the four fundamental subspaces. Source: https://commons.wikimedia.org/wiki/File:Thefoursubspaces.svg

• Row and Column Spaces
• Kernel (Nullspace)
• Rank
• Basis and Dimension (Linear Algebra)