Fundamental Subspaces

The fundamental subspaces are four vector spaces defined by a given $m \times n$ matrix $A$ (and its transpose): the column space and nullspace (or kernel) of $A$, the column space of $A^T$ $($also called the row space of $A),$ and the nullspace of $A^T$ $($also called the left nullspace of $A).$

The fundamental subspaces are useful for a number of linear algebra applications, including analyzing the rank of a matrix. The subspaces are also closely related by the fundamental theorem of linear algebra.

The column space of a matrix $A$ is the vector space formed by the columns of $A$, essentially meaning all linear combinations of the columns of $A$. Equivalently, the column space consists of all matrices $Ax$ for some vector $x$.

For this reason, the column space is also known as the image of $A$ $\big($denoted $\text{im}(A)\big),$ as it is the result when $A$ is viewed as a linear transformation of the vector space $\mathbb{R}^m$ $(\text{where } m$ is the number of rows of $A).$ In particular, the image of $A$ is necessarily a subspace of $\mathbb{R}^m$, hence the term "fundamental subspace."

For example, consider the matrix

$$A = \begin{pmatrix} 1 & 2 & 3 & 3 \\ 2 & 0 & 6 & 2 \\ 3 & 4 & 9 & 7\end{pmatrix}.$$

Then the column vectors are

$$\begin{pmatrix}1\\2\\3\end{pmatrix}, \begin{pmatrix}2\\0\\4\end{pmatrix}, \begin{pmatrix}3\\6\\9\end{pmatrix},\begin{pmatrix}3\\2\\7\end{pmatrix},$$

which form a vector space that has a basis of $\left\{\begin{pmatrix}1\\2\\3\end{pmatrix}, \begin{pmatrix}2\\0\\4\end{pmatrix}\right\}$. As there are 2 vectors in this basis, the dimension of the column space is 2; hence, the rank of $A$ is 2.

The nullspace or kernel of a matrix $A$ $\big($denoted $\text{ker}(A)\big)$ is the set of all vectors $x$ for which

$$Ax = 0.$$

If $x$ and $y$ are in the nullspace, then $c_1x+c_2y$ is also in the nullspace as

$$A(c_1x + c_2y) = c_1(Ax) + c_2(Ay) = 0 + 0 = 0,$$

so the nullspace is in fact a vector space. When $A$ is viewed as a linear transformation, the nullspace is the subspace of $\mathbb{R}^n$ that is sent to 0 under the map $A$, hence the term "fundamental subspace."

For example, consider the matrix

$$A = \begin{pmatrix} 1 & 2 & 3 & 3 \\ 2 & 0 & 6 & 2 \\ 3 & 4 & 9 & 7\end{pmatrix}.$$

The nullspace consists of all vectors $x$ such that $Ax = 0$. This defines a system of linear equations that can be solved to give the family of solutions

$$\begin{pmatrix} -x-3y\\-x\\y\\x\end{pmatrix}, x,y \in \mathbb{R},$$

which defines a vector space with basis $\left\{\begin{pmatrix}-1\\-1\\0\\1\end{pmatrix}, \begin{pmatrix}-3\\0\\1\\0\end{pmatrix}\right\}$. As there are two vectors in this basis, the dimension of the nullspace is 2.

The row space and left nullspace are defined as the column space and nullspace of $A^T$, the transpose of $A$, respectively. In this way, they are subspaces of $\mathbb{R}^n$.

The fundamental theorem of linear algebra relates all four of the fundamental subspaces in a number of different ways. There are main parts to the theorem:

Part 1:
The first part of the fundamental theorem of linear algebra relates the dimensions of the four fundamental subspaces:

The column and row spaces of an $m \times n$ matrix $A$ both have dimension $r$, the rank of the matrix. The nullspace has dimension $n-r$, and the left nullspace has dimension $m-r$.

This is illustrated by the example in previous sections: the dimension of the column space of

$$A = \begin{pmatrix} 1 & 2 & 3 & 3 \\ 2 & 0 & 6 & 2 \\ 3 & 4 & 9 & 7\end{pmatrix}$$

is 2, and the dimension of the nullspace of $A$ is $n - r = 4 - 2 = 2$.

This first part of the fundamental theorem of linear algebra is sometimes referred to by name as the rank-nullity theorem.

Part 2:
The second part of the fundamental theorem of linear algebra relates the fundamental subspaces more directly:

The nullspace and row space are orthogonal. The left nullspace and the column space are also orthogonal.

In other words, if $v$ is in the nullspace of $A$ and $w$ is in the row space of $A$, the dot product $v \cdot w$ is 0. This is true because any vector in the nullspace is orthogonal to each row vector by definition, so it is also orthogonal to any linear combination of them.

Part 3:
The final part of the fundamental theorem of linear algebra constructs an orthonormal basis, and demonstrates a singular value decomposition: any matrix $M$ can be written in the form $U\sum V^T$, where

• $U$ is an $m \times m$ unitary matrix;
• $\sum$ is an $m \times n$ matrix with nonnegative values on the diagonal;
• $V$ is an $n \times n$ unitary matrix.

This part of the fundamental theorem allows one to immediately find a basis of the subspace in question.

This can be summarized in the following table:

Or it can also be summarized in the following image, where the arrows represent the results of multiplication: The relationship between the four fundamental subspaces. Source: https://commons.wikimedia.org/wiki/File:Thefoursubspaces.svg

• Row and Column Spaces
• Kernel (Nullspace)
• Rank
• Basis and Dimension (Linear Algebra)