← Back to Math 010

4.9 Rank, Nullity, and the Fundamental Matrix Spaces: Class Notes

Theorem 4.9.1

The row space and column space of a matrix $A$ have the same dimension.

Definition

The common dimension of the row space and column space of a matrix $A$ is called the rank of $A$ and is denoted by $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)$; the dimension of the null space of $A$ is called the nullity of $A$ and is denoted by $\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}(A)$. That is, $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)=\dim [\mathrm{r}\mathrm{o}\mathrm{w}(A)]$ $=\dim [\mathrm{c}\mathrm{o}\mathrm{l}(A)]$ and $\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}(A)=\dim [\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(A)]$.

#1

Find the rank and nullity of the matrix $A$ by reducing it to row echelon form.

Solution

a. Every row is a multiple of row 1, so subtracting multiples of row 1 gives

$$\left[\begin{array}{cccc}1& 2& -1& 1\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right]$$

There is one leading 1, so $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)=1$.

The one equation gives $x_1=-2x_2+x_3-x_4$. Then, with $x_2=r$, $x_3=s$, and $x_4=t$,

$$\overrightarrow{x}=\left[\begin{array}{c}-2r+s-t\\ r\\ s\\ t\end{array}\right]=\left[\begin{array}{c}-2\\ 1\\ 0\\ 0\end{array}\right]r+\left[\begin{array}{c}1\\ 0\\ 1\\ 0\end{array}\right]s+\left[\begin{array}{c}-1\\ 0\\ 0\\ 1\end{array}\right]t.$$

There are three parameters, so $\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}(A)=3$.

b.

$$A⟶\left[\begin{array}{ccccc}1& -2& 0& -1& 0\\ 0& 0& 1& 2& 0\\ 0& 0& 0& 0& 1\end{array}\right]$$

$\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)=3$ (the number of leading 1’s, or leading variables).

$\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}(A)=2$ (the number of free variables, or parameters).

Theorem 4.9.2 Dimension Theorem for Matrices

If $A$ is a matrix with $n$ columns, then $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)+\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}(A)=n$.

In terms of the system $A\overrightarrow{x}=\overrightarrow{0}$: $n$ is the number of variables, $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)$ is the number of leading variables, and $\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}(A)$ is the number of parameters (free variables).

Theorem 4.9.3

If $A$ is an $m\times n$ matrix, then
a) $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)$ = the number of leading variables in the general solution of $A𝐱=\mathrm{𝟎}$.
b) $\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}(A)$ = the number of parameters in the general solution of $A𝐱=\mathrm{𝟎}$.

Theorem 4.9.4

If $A𝐱=𝐛$ is a consistent linear system of $m$ equations in $n$ unknowns, and if $A$ has rank $r$, then the general solution of the system contains $n-r$ parameters.

Fundamental Spaces of a Matrix

Considering a matrix $A$ and its transpose $A^T$, we have six spaces: $\mathrm{r}\mathrm{o}\mathrm{w}(A)$, $\mathrm{c}\mathrm{o}\mathrm{l}(A)$, $\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(A)$, $\mathrm{r}\mathrm{o}\mathrm{w}(A^T)$, $\mathrm{c}\mathrm{o}\mathrm{l}(A^T)$, and $\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(A^T)$. Since the rows of $A$ are the columns of $A^T$ and the columns of $A$ are the rows of $A^T$, two pairs are the same:

$$\mathrm{r}\mathrm{o}\mathrm{w}(A)=\mathrm{c}\mathrm{o}\mathrm{l}(A^T)\mathrm{c}\mathrm{o}\mathrm{l}(A)=\mathrm{r}\mathrm{o}\mathrm{w}(A^T)$$

The four fundamental spaces of a matrix $A$ are $\mathrm{r}\mathrm{o}\mathrm{w}(A)$, $\mathrm{c}\mathrm{o}\mathrm{l}(A)$, $\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(A)$, and $\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(A^T)$. The null space of $A^T$ is also called the left null space of A. The left nullity of A is $\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}(A^T)=\dim [\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(A^T)]$.

It is called the left null space because $A^T\overrightarrow{x}=\overrightarrow{0}$ becomes ${\overrightarrow{x}}^{T}A={\overrightarrow{0}}^T$ (transpose both sides), with ${\overrightarrow{x}}^T$ multiplying $A$ on the left.

Theorem 4.9.5

If $A$ is any matrix, then $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)=\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A^T)$.

Let $A$ be $m\times n$. Since $A^T$ has $m$ columns, the dimension theorem gives $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A^T)+\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}(A^T)=m$, and Theorem 4.9.5 lets us replace $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A^T)$ by $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)$:

$$\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)+\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}(A^T)=m.$$

So, with $r=\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)$,

$$\begin{array}{cc}\hfill {\displaystyle \dim [\mathrm{r}\mathrm{o}\mathrm{w}(A)]}& {\displaystyle =r,\dim [\mathrm{c}\mathrm{o}\mathrm{l}(A)]=r,}\hfill \\ \hfill {\displaystyle \dim [\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(A)]}& {\displaystyle =n-r,\dim [\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(A^T)]=m-r.}\hfill \end{array}$$

#11

Find the dimensions and bases for the four fundamental spaces of the matrix.

Solution

Here $m=3$, so augment with $I_3$ and row reduce. The pivot rows of R give the row space, the columns of A that correspond to the pivot columns of $R$ give the column space, and the bottom row of E gives the left null space.

$${\displaystyle \left[\begin{array}{ccccc}\hfill {\displaystyle 1}\hfill & \hfill {\displaystyle 4}\hfill & \hfill {\displaystyle 1}\hfill & \hfill {\displaystyle 0}\hfill & \hfill {\displaystyle 0}\hfill \\ \hfill {\displaystyle 0}\hfill & \hfill {\displaystyle 3}\hfill & \hfill {\displaystyle 0}\hfill & \hfill {\displaystyle 1}\hfill & \hfill {\displaystyle 0}\hfill \\ \hfill {\displaystyle -9}\hfill & \hfill {\displaystyle 0}\hfill & \hfill {\displaystyle 0}\hfill & \hfill {\displaystyle 0}\hfill & \hfill {\displaystyle 1}\hfill \end{array}\right]⟶\left[\begin{array}{ccccc}\hfill {\displaystyle 1}\hfill & \hfill {\displaystyle 0}\hfill & \hfill {\displaystyle 1}\hfill & \hfill {\displaystyle -\frac{4}{3}}\hfill & \hfill {\displaystyle 0}\hfill \\ \hfill {\displaystyle 0}\hfill & \hfill {\displaystyle 1}\hfill & \hfill {\displaystyle 0}\hfill & \hfill {\displaystyle \frac{1}{3}}\hfill & \hfill {\displaystyle 0}\hfill \\ \hfill {\displaystyle 0}\hfill & \hfill {\displaystyle 0}\hfill & \hfill {\displaystyle 9}\hfill & \hfill {\displaystyle -12}\hfill & \hfill {\displaystyle 1}\hfill \end{array}\right]}$$

$\mathrm{r}\mathrm{o}\mathrm{w}(A)\text{ basis: }\{(1,0),(0,1)\}$, dim 2

$\mathrm{c}\mathrm{o}\mathrm{l}(A)\text{ basis: }\{\left[\begin{array}{c}1\\ 0\\ -9\end{array}\right],\left[\begin{array}{c}4\\ 3\\ 0\end{array}\right]\}$, dim 2

$\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(A^T)\text{ basis: }\left\{\left[\begin{array}{ccc}9& -12& 1\end{array}\right]\right\}$, dim 1

$n=2$ with $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)=2$, so $\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}(A)=0$. Basis: $\varnothing$.

Definition

If $W$ is a subspace of $R^n$, then the set of all vectors in $R^n$ that are orthogonal to every vector in $W$ is called the orthogonal complement of $W$ and is denoted by the symbol $W^{⟂}$ (pronounced “$W$ perp”).

Theorem 4.9.6

If $W$ is a subspace of $R^n$, then:
a) $W^{⟂}$ is a subspace of $R^n$.
b) The only vector common to $W$ and $W^{⟂}$ is $\mathrm{𝟎}$, that is, $W\cap W^{⟂}=\{\mathrm{𝟎}\}$.
c) The orthogonal complement of $W^{⟂}$ is $W$, that is, ${\left(W^{⟂}\right)}^{⟂}=W$.

Proof of (a)

$\overrightarrow{0}⟂\overrightarrow{v}$ for all $\overrightarrow{v}\in R^n$, so $\overrightarrow{0}⟂\overrightarrow{w}$ for all $\overrightarrow{w}\in W$, and $\overrightarrow{0}\in W^{⟂}$. So $W^{⟂}$ is not empty.

Let ${\overrightarrow{w}}_1,{\overrightarrow{w}}_2\in W^{⟂}$. Then ${\overrightarrow{w}}_1⟂\overrightarrow{w}$ for all $\overrightarrow{w}\in W$ and ${\overrightarrow{w}}_2⟂\overrightarrow{w}$ for all $\overrightarrow{w}\in W$. Let $\overrightarrow{w}\in W$. Then

$$\begin{array}{cc}\hfill {\displaystyle ({\overrightarrow{w}}_1+{\overrightarrow{w}}_2)·\overrightarrow{w}}& {\displaystyle ={\overrightarrow{w}}_1·\overrightarrow{w}+{\overrightarrow{w}}_2·\overrightarrow{w}\text{(property of the dot product)}}\hfill \\ \hfill {\displaystyle }& {\displaystyle =0+0}\hfill \\ \hfill {\displaystyle }& {\displaystyle =0.}\hfill \end{array}$$

So ${\overrightarrow{w}}_1+{\overrightarrow{w}}_2\in W^{⟂}$.

Let $k\in R$. Then

$$\begin{array}{cc}\hfill {\displaystyle \left(k{\overrightarrow{w}}_1\right)·\overrightarrow{w}}& {\displaystyle =k({\overrightarrow{w}}_1·\overrightarrow{w})\text{(property of the dot product)}}\hfill \\ \hfill {\displaystyle }& {\displaystyle =k(0)=0.}\hfill \end{array}$$

So $k{\overrightarrow{w}}_1\in W^{⟂}$, and therefore $W^{⟂}$ is a subspace of $R^n$.

Theorem 4.9.7

If $A$ is an $m\times n$ matrix, then:
a) The null space of $A$ and the row space of $A$ are orthogonal complements in $R^n$.
b) The null space of $A^T$ and the column space of $A$ are orthogonal complements in $R^m$.

#15

Confirm the orthogonality statements in the two parts of Theorem 4.9.7 for the matrix

Solution

From #11:

a) $\mathrm{r}\mathrm{o}\mathrm{w}(A)\text{ basis: }\{(1,0),(0,1)\}$, so $\mathrm{r}\mathrm{o}\mathrm{w}(A)=R^2$, and $\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(A)=\{\overrightarrow{0}\}$. These are orthogonal complements in $R^2$. ✓

b) $\mathrm{c}\mathrm{o}\mathrm{l}(A)\text{ basis: }\{\left[\begin{array}{c}1\\ 0\\ -9\end{array}\right],\left[\begin{array}{c}4\\ 3\\ 0\end{array}\right]\}=\{{\overrightarrow{u}}_1,{\overrightarrow{u}}_2\}$

$\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(A^T)\text{ basis: }\left\{\left[\begin{array}{ccc}9& -12& 1\end{array}\right]\right\}=\{{\overrightarrow{v}}_1^T\}$

We see that

$${\overrightarrow{u}}_1·{\overrightarrow{v}}_1=9+0-9=0,{\overrightarrow{u}}_2·{\overrightarrow{v}}_1=36-36+0=0.$$

Thus these are orthogonal, so their spans are orthogonal (and since the dimensions add to $2+1=3$, their spans are orthogonal complements in $R^3$).

#27

Suppose that $A$ is a $3\times 3$ matrix whose null space is a line through the origin in 3-space. Can the row or column space of $A$ also be a line through the origin? Explain.

Solution

No. $\dim [\mathrm{r}\mathrm{o}\mathrm{w}(A)]=\dim [\mathrm{c}\mathrm{o}\mathrm{l}(A)]=\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)$ and $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)+\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}(A)=n$, the number of columns of $A$, which in this case is 3. A line has dimension 1, so $\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}(A)=1$ and $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(A)=2$. The row and column spaces have to be planes.

Theorem 4.9.8 Equivalent Statements (extends Theorem 2.3.8)

If $A$ is an $n\times n$ matrix, then the following statements are equivalent.
a) $A$ is invertible.
b) $A𝐱=\mathrm{𝟎}$ has only the trivial solution.
c) The reduced row echelon form of $A$ is $I_n$.
d) $A$ is expressible as a product of elementary matrices.
e) $A𝐱=𝐛$ is consistent for every $n\times 1$ matrix $𝐛$.
f) $A𝐱=𝐛$ has exactly one solution for every $n\times 1$ matrix $𝐛$.
g) $det(A)\ne 0$.
h) The column vectors of $A$ are distinct and linearly independent.
i) The row vectors of $A$ are distinct and linearly independent.
j) The column vectors of $A$ span $R^n$.
k) The row vectors of $A$ span $R^n$.
l) The column vectors of $A$ form a basis for $R^n$.
m) The row vectors of $A$ form a basis for $R^n$.
n) $A$ has rank $n$.
o) $A$ has nullity 0.
p) The orthogonal complement of the null space of $A$ is $R^n$.
q) The orthogonal complement of the row space of $A$ is $\{\mathrm{𝟎}\}$.