Rank

In linear algebra, the rank of a matrix is the dimension of its row space or column space. It is an important fact that the row space and column space of a matrix have equal dimensions.

Let $A$ be a matrix. If $R(A)$ denotes the row space of $A$ and $C(A)$ denotes the column space of $A$, then $\dim\big(R(A)\big) = \dim\big(C(A)\big)$. This quantity is called the rank of $A$, and denoted $\text{rk}(A)$.

Intuitively, the rank measures how far the linear transformation represented by a matrix is from being injective or surjective. Suppose $A$ is an $m$-by-$n$ matrix representing a linear transformation $T: \mathbb{R}^n \to \mathbb{R}^m$. Since $C(A) = \text{Im}(T)$, the rank of $A$ is an integer between $0$ and $m$, and equals $m$ precisely when $T$ is surjective. A similar interpretation exists for $R(A)$ in terms of injectivity: the rank of $A$ is an integer between $0$ and $n$, and equals $n$ precisely when $T$ is injective.

Let $A$ be an $m$-by-$n$ matrix, representing a linear transformation $T: \mathbb{R}^n \to \mathbb{R}^m$. We define the row rank of $A$ to be $\dim\big(R(A)\big)$, and similarly the column rank $\dim\big(C(A)\big)$. A priori, these numbers are not necessarily equal, but we will prove they actually are.

Suppose the row rank of $A$ is $r$. Our approach will be to produce $r$ linearly independent vectors in $C(A)$. This will prove $\dim\big(C(A)\big) \ge \dim\big(R(A)\big)$. By applying the same argument to the transpose of $A$, we obtain the reverse inequality $\dim\big(R(A)\big) \ge \dim\big(C(A)\big)$, hence the result; this works because taking the transpose simply swaps row and column spaces.

How could one produce $r$ linearly independent vectors in $C(A) = \text{Im}(T) \subset \mathbb{R}^m?$ A possible approach is to choose $r$ linearly independent vectors in $\mathbb{R}^n$, and then apply $T$ to each of them, hoping the resultant $r$ vectors in $\mathbb{R}^m$ will themselves be linearly independent. To ensure this works, the set of $r$ linearly independent vectors in $\mathbb{R}^n$ should have some additional structure we can work with; they cannot just be $r$ random vectors. But there is a very natural set of $r$ linearly independent vectors in $\mathbb{R}^n$: a basis for the row space $R(A)!$

Accordingly, let $x_1, \ldots, x_r$ be a basis for $R(A)$. We must prove $Tx_1, \ldots, Tx_r$ are linearly independent. Suppose they aren't; then there are $c_i \in \mathbb{R}$ (not all zero) such that

$$0 = \sum_{i=1}^{r} c_i Tx_i = T(c_1 x_1 + \cdots + c_r x_r).$$

By construction, we know $v= c_1 x_1 + \cdots + c_r x_r$ is in the row space of $A$. Furthermore, since $Tv = 0$, a computation done in the article on row and column spaces implies $v$ is orthogonal to every row of $A$. But $v$ is a linear combination of these rows, so $v$ is orthogonal to itself; thus, $v = 0$. This implies the relation $c_1 x_1 + \cdots + c_r x_r = 0$, which is absurd on account of linear independence of the collection $\{x_i\}$. $_\square$

What is the rank of the matrix

$$A = \begin{pmatrix} 2 & 1 & 0 \\ 3& -1 & 2 \end{pmatrix}?$$

---

Note that the first two columns of $A$ are linearly independent, since they are not multiples of each other. Thus, $C(A) = \mathbb{R}^2 \implies \text{rank}(A) = 2$. $_\square$

Let $A$ be an $n$-by-$n$ square matrix. Prove $A$ has rank $n$ if and only if $A$ is invertible.

---

$A$ has rank $n$ if and only if the row space $R(A)$ equals $\mathbb{R}^n$. But the kernel of $A$ is precisely the orthogonal complement of the row space, and hence $R(A) = \mathbb{R}^n \iff \text{Ker}(A) = \{0\}$. We know $A$ is invertible if and only if its kernel is zero, so we are done. $_\square$