Download Understanding the Row Space and Column Space of a Matrix: Rank and Nullity and more Exercises Geometry in PDF only on Docsity!
Rank and Nullity
In section 4.7, we defined the row space and column space of a matrix A as the vector spaces spanned
by the rows and columns of A, respectively. For example, we saw that the row space of the matrix
A =
is the three dimensional vector space spanned by the vectors
r 1
′
r 2
′
r 3
′
the column space of A is the three dimensional vector space spanned by the vectors
c 1 =
, c 2 =
, and c 3 =
It is interesting to note that, while the column space and row space of matrix A are not the
same vector space (indeed the row space is “living” in R^5 , whereas the column space is in R^4 ),
they are vector spaces of the same dimension. We will see in this section that this is no fluke. We
will explore this idea and many more of the interconnections among row space, column space, null
space, and solutions to the system Ax = b in this section.
Rank and Nullity
Let’s think about why our matrix A above had the row space and column space of the same
dimension: we reduced A by Gaussian elimination to
R =
R and A have the same row space, and while they do not have the same column space, their column
spaces have the same dimension.
So we can ask our question in a different way: why do the dimensions of the column space and
row space of R match up? Let’s inspect R again:
R =
We saw a theorem in 4.7 that told us how to find the row space and column space for a matrix
in row echelon form:
Theorem. If a matrix R is in row echelon form, then the row vectors with leading 1s form a basis
for the row space of R (and for any matrix row equivalent to R), and the column vectors with
leading 1s form a basis for the column space of R.
In other words, the dimensions of the column spaces and row spaces are determined by the
number of leading 1s in columns and rows, respectively. The leading 1s for R are highlighted in
red below:
R =
notice the the leading 1s of the rows are also the leading 1s of the columns. That is, every leading
1 is a leading 1 for both a row and a column; in general, any matrix in row echelon form has the
same number of leading 1s in its rows as it does in its columns, thus its row space and column
space must have the same dimension. We make this idea precise in the next theorem:
Theorem 4.8.1. The row space and column space of a matrix A have the same dimension.
We name the shared dimensions of the row and column spaces of A, as well as the dimension
of the vector space null (A), in the following:
Definition 1. The dimension of the row space/column space of a matrix A is called the rank of
A; we use notation rank (A) to indicate that
dim(row (A)) = dim(column (A)) = rank (A).
The dimension of the vector space null (A) is called the nullity of A, and is denoted nullity (A).
Example
Given
A =
find:
- rank (A)
- nullity (A)
We investigated matrix A in Section 4.7; to find rank (A), we simply need to determine the
dimension of either row (A) or column (A) (they’re the same number!), and to find nullity (A), we
need to know the dimension of null (A).
Notice that the 1 in position 2, 3 is not a leading 1 of a column, as the column before it
(column 2) has a leading 1 in the same position. Since there are three leading 1s, we know
that the vector spaces row (A) and column (A) both have dimension 3, so that
rank (A) = 3.
- Let’s calculate the dimension of the null space of A, that is the dimension of the solution
space to
Ax = 0.
The augmented equation for the system is
1 3 1 2 5 | 0
0 1 1 0 − 1 | 0
0 0 0 1 6 | 0
0 0 0 0 0 | 0
0 0 0 0 0 | 0
0 0 0 0 0 | 0
the third row tells us that
x 4 = − 6 x 5 ,
and the second row says that
x 2 = −x 3 + x 5.
Combining these equalities with the data from the first row, we have
x 1 = − 3 x 2 − x 3 − 2 x 4 − 5 x 5
= −3(−x 3 + x 5 ) − x 3 − 2(− 6 x 5 ) − 5 x 5
= 3 x 3 − 3 x 5 − x 3 + 12x 5 − 5 x 5
= 2 x 3 + 4x 5.
Thus any vector x in the solution space to the equation Rx = 0 has form
x =
2 x 3 + 4x 5
−x 3 + x 5
x 3
− 6 x 5
x 5
so the solution space consists of all linear combinations of the form
x 3
The null space of A is thus two-dimensional, so that
nullity (A) = 2.
Dimension Theorems
Just as with the example we investigated in Section 4.7, we see that the row space of A above is
a three-dimensional subspace of R
5 ; since row (A) took up three dimensions of R
5 , there were only
two dimensions left for null (A). We make these ideas more precise in the following theorem.
Theorem 4.8.2. If A is an m × n matrix (in particular, A has n columns) then
rank (A) + nullity (A) = n.
If A is m × n, then the row space and null space of A are both subspaces of R
n
. As indicated
in the previous examples, the theorem states that the row space and null space “use up” all of Rn.
Key Point. Recall that the column space (a subspace of R
m ) and the row space (a subspace of
R
n ) must have the same dimension. In this case, the maximum value for dim(column (A)) is m,
and the maximum value for dim(row (A)) is n. So
dim(row (A)) = dim(column (A)) = min(m, n).
Example
A matrix A is 4 × 9. Find:
- The maximum possible value for rank (A) and the minimum possible value for nullity (A).
- rank (A) given that nullity (A) = 7.
- Since A has 4 rows and 9 columns, the maximum possible value for rank (A) is 4, and we
know that
rank (A) + nullity (A) = 9.
Thus nullity (A) must be at least 5, and will be more if rank (A) < 4.
- If nullity (A) = 7, then rank (A) = 2 since
rank (A) = 9 − nullity (A).
Definition. The four fundamental spaces of matrices A and A
⊤ are
row (A) = column (A
⊤ ) column (A) = row (A
⊤ )
null (A) null (A
⊤ ).
We can make a few more quick observations about these spaces:
Key Point. If A is m × n, so that A
⊤ is n × m, then we know that
rank (A) + nullity (A) = n
and
rank (A
⊤ ) + nullity (A
⊤ ) = m.
However, we have already seen that rank (A) = rank (A
⊤ ), so
m = rank (A
⊤ ) + nullity (A
⊤ )
= rank (A) + nullity (A
⊤ ).
So if A is an m × n matrix with rank (A) = r, we have the following relationships:
rank (A) = rank (A
⊤ ) = r
rank (A) + nullity (A) = n rank (A) + nullity (A
⊤ ) = m
nullity (A) = n − r nullity (A
⊤ ) = m − r.
Geometric Relationships Among the Fundamental Spaces
We have mentioned several times that, if A is an m × n matrix, then the vector spaces row (A)
and null (A) are both subspaces of R
n
. Given this information, it makes sense to try to understand
what relationships such as
rank (A) + nullity (A) = n
mean in terms of the geometry of Euclidean space.
Before we look at the details of the ideas, let’s build some intuition by considering a simple
example.
Example
Find row (A) and null (A), given
A =
and describe the vector spaces geometrically.
A is 2 × 3, so we know that the vector spaces row (A) and null (A) are both subspaces of R
3 ,
and we also know that
rank (A) + nullity (A) = 3.
Since A is already in row echelon form, it is easy to see that row (A) is the two dimensional vector
space spanned by vectors
v 1 =
(^) and v 2 =
These two vectors are graphed below, along with their span, which is a plane in R
3 :
Let’s calculate the null space null (A) of A: if Ax = 0 , the augmented matrix for the equation
is (^) (
1 0 1 | 0
0 1 1 | 0
we see that
x 1 + x 3 = 0
x 2 + x 3 = 0 ,
From the graph, it appears that the basis vector for null (A) is orthogonal (perpendicular) to
the plane, i.e. to the vectors in row (A). We can check that this is true quite easily, using the idea
of a normal vector from Calculus 3: the vector that results from calculating the “determinant” of
the matrix (^)
i j k
1 0 1
0 1 1
is said to be normal to the vectors
v 1 =
(^) and v 2 =
in the last two rows of the matrix; in particular, this vector is orthogonal to the plane formed by
the span of v 1 and v 2. Let’s make the calculation:
det
i j k
1 0 1
0 1 1
(^) = −i − j + k
So the vector that forms the basis for null (A) (and indeed every vector in null (A)) is orthogonal
to every vector in the vector space row (A)! We will see soon that this surprising result is actually
true in general. Accordingly, we record a few relevant definitions below.
Orthogonal Complements
Definition 2. If W is a subspace of R
n , the orthogonal complement of W , denoted W
⊥ , is the set
of all vectors in R
n that are orthogonal to every vector in W.
In terms of our example above, with
W = span
= row (A)
in R
3 , we see that the orthogonal complement of W in R
3 is given by
W
⊥ = span
= null (A).
The red line graphed below is a subspace of R
2 :
Its orthogonal complement is the plane graphed in orange below:
We record a few facts about orthogonal complements in the next theorem:
Theorem 4.8.6. If W is a subspace of R
n , then:
1. W
⊥ is a subspace of R
n
- The only vector in both subspaces W and W
⊥ is 0
- The orthogonal complement of the subspace W
⊥ is W.
Given our first example above, you may have already guessed the reason that orthogonal com-
plements come up now in our discussion:
Theorem 4.8.7. If A is an m × n matrix, then:
(a) The null space null (A) and the row space row (A) of A are orthogonal complements in R
n .
(b) The null space null (A
⊤ ) and the column space column (A) of A are orthogonal complements
in R
m .
This theorem is remarkable in the sense that it gives a relationship between the algebraic
structures of two vector spaces and the geometric structures of the same vector spaces: if vector
space W 1 spans the set of solutions to Ax = b and W 2 spans the set of solutions to Ax = 0 , then
every vector in W 1 is orthogonal to every vector in W 2.
We can now extend the list of equivalent conditions we have been maintaining:
Theorem 4.8.8. Let A be an n × n matrix. Then the following are equivalent:
(a) A is invertible.
(b) Ax = 0 has only the trivial solution.
(c) The reduced row echelon form of A is In.
(d) A is a product of elementary matrices.
(e) Ax = b is consistent for every n × 1 matrix b.
(f) Ax = b has exactly one solution for every n × 1 matrix b.
(g) det A ̸= 0.
(h) The column vectors of A are linearly independent.
(i) The row vectors of A are linearly independent.
(j) The column vectors of A span R
n (column (A) = R
n ).
(k) The row vectors of A span R
n (row (A) = R
n ).
(l) The column vectors of A form a basis for R
n .
(m) The row vectors of A form a basis for R
n .
(n) rank (A) = n.
(o) nullity (A) = 0.
(p) The orthogonal complement of the null space of A is R
n .
(q) The orthogonal complement of the row space of A is 0.
