Download LU Decomposition and Matrix Analysis: Column Space and Null Space and more Study notes Mathematics in PDF only on Docsity!
IB Paper 7: Linear Algebra Handout 4
Tom Hynes
3.7 Bases for the Column Space and Row Space of A
LU decomposition gives us an immediate answer to how to general convenient descriptions
for the Row Space and Column Space of A. For a general m × n matrix
Column Space = all vectors formed by taking a linear combination of the columns of A
1 1 2 2 3 3
n n
λ a + λ a + λ a + +λ a
as the λ’s vary.
Row Space = all vectors formed by taking a linear combination of the rows of A
1 1 2 2 3 3
m m
μ a + μ a + μ a + +μ a
as the μ’s vary.
We shall use the matrix A on which we performed LU decomposition in section 3.3 denoted AI
AI =
= LUI
and in order to show the range of behaviour,
AII =
= LUII
You will see that, since it is only the bottom right-hand corner of A that is different, it is only the
last row of U that changes and L is the same for both. You should check this by either LU or by
simply multiplying out.
Now in terms of the outer products of the columns and rows of L and UI and UII ,
T T T
1 1 2 2 3 3
= l u + l u + l u I
A ^ ^ ^
T T T
1 1 2 2 3 3
= l u + l u + l u II
A ^ ^
Basis for Column Space
Remembering that we can consider matrix multiplication as a relationship between columns (see
section 2.6)
[ ] [ ] [ ]
11 21 31
1 1 2 3
u u u
a l l l
, i.e. 1 11 1 21 2 313
a = u l + u l + u l , etc.
Changed
Since all of the columns of A can be written in terms of them, this means that l 1 , l 2 , … form a
basis for the column space of A (at least the set of them for which the corresponding u ^ is non-zero
do).
We see immediately that for matrix A I
1 1
a = l 2 1 2
a = 2 l + 2 l 3 1 2 3
a = l + l + l 4 1 2 3
a = 3 l + 6 l − l
while for matrix A II
1 1
a = l 2 1 2
a = 2 l + 2 l 3 1 2
a = l + l 4 1 2
a = 3 l + 6 l
Since the columns of L are independent (see next section),
a basis of the column space of AI is
2 , 1 and 0
while one for AII is
2 and 1
Basis for Row Space
Remembering that we can also consider matrix multiplication as a relationship between rows (see
section 2.7)
[ ] [ ] [ ]
1 11 1 12 2 13 3
← a → = b ← c → + b ← c → + b ← c →
, i.e. 1 11 1 12 2 13 3
a ^ = l u ^ + l u ^ + l u ^ , etc.
we see immediately that for matrix AI
1 1
a ^ = u ^ 2 1 2
a ^ = 2 u ^ + u ^ 3 1 2 3
a ^ = − u ^ + 2 u ^ + u
while for matrix A II
1 1
a = u 2 1 2
a = 2 u + u 3 1 2 3
a = − u + 2 u + u
A basis of the row space of A I
is
, and
while A II
is
and
4. The Solution of Ax = b
4.1 What are we expecting?
We know that for a 3 × 3 matrix A
Equations
independent?
No
Yes
No Solution
Equations
compatible?
Unique solution
No
x = x 0
Yes
Tell this by
det A ≠ 0
This involves a
condition on b
A x = b
For a general m × n case, as for example the matrix associated with the structure analysed in section
1 of Handout 1, A t = f
I
E II
E III
IV (^) F
V F
VI
x
y
x
y
t
f t
f t
t f
t f W
t
− ^
−^ −^ ^
^
we will amend this to
Yes
No Solution
Equations
compatible?
i.e.
b in column
space of A?
No A x = b
Yes
x = x 0
+ λ n
1
+ μ n
2
and the issues are now (i) how to tell if b is in the column space of A and (ii) how do we find x 0
n 1
, n 2
, ... (and how many of them should there be). We don’t really need to consider the case of a
unique solution as special; this will come out in the wash as there being zero n ’s.
λ arbitrary
arbitrary
4.2 How do we check whether b is in Column Space?
The answer to (i) is that we when we express b in terms of the columns of L , it should only need
those columns that are in the column space of A.
AI =
= L UI Col Space
2 , 1 and 0
AII =
= L UII Col Space
2 and 1
When we solve A x = b using LU decomposition, LU x = b , then, we recast the problem as L c = b
and U x = c. The first step is to find c this is expressing b in terms of the columns of L.
1
1 2 3 2
3
c
c c c c
c
1
c = 4 2 2 2 1
c = − c = 2 2 1 3 1 2
c = + c − c =− c =
3
2
1
c
c
c
There will be a solution for A I
but not for A II
4.3 Completing the Solution U x = c (finding the Null-Space of A)
As part of the following procedure, we will be generating n 1 , n 2 , ... which will be the general
solution of A n = 0. We are, in effect, generating a basis for the null space of A
CASE I U x = c
t
z
y
x
Starting with the last of these and working upwards
z − t = − 1
This is the first evidence of the problem being under-determined. We can not solve this equation.
We will, instead use it to find z in terms of t.
Pivots
e.g. A x = b =
We can also approach this problem using a “a particular solution” plus “general solution of A x = 0”
method. This would be (taken from the Maths Databook)
- Set the free variable to zero and find a particular solution x 0
- Set the RHS to zero (i.e. U x = 0), put the free variable equal to the value 1 and solve
to find n.
There may actually be more than one free variable, when this becomes
- Set all free variables to zero and find a particular solution x 0
- Set the RHS to zero, give each free variable in turn the value 1 while the others are zero,
and solve to find a set of vectors which span the nullspace of A.
CASE II Ux = c
x
y
z
t
^
As noted earlier, we can not solve this, unless c 3
If b , on the other hand, had been
, then L c = b gives
1
2
3
c
c
c
c =
1
2
3
c
c
c
and the equations are compatible.
U x = c , is now
x
y
z
t
Only x and y have pivots and this time both z and t are free variables.
- To find x 0
, set the free variables to zero. This gives (using back substitution)
2 y = 2 y = 1 and
x = 1 − 2 y x = − 1 i.e.
- Set the RHS to zero, give each free variable in turn the value 1 while the others are zero,
and solve to find a set of vectors which span the nullspace of A.
0
x
^ −
Put t = 1, z = 0
x
y
By back substitution, 2 y = − 6 y = − 3 and x = − 3 − 2 y = 3 i.e.
Put t = 0, z = 1
x
y
^
By back substitution,
2 y = − 1 y = −
and x = − 1 − 2 y = x = 0
The general solution is, therefore,
x t z
^ −
− ^ −
You can now do Examples Paper 7/7 Question 8.
1
n
2
i.e. n
