Download Linear Algebra, Lecture Notes - Mathematics - 1 and more Study notes Mathematics in PDF only on Docsity!
IB Paper 7: Linear Algebra Handout 1
Tom Hynes
Aims
To introduce the ideas and techniques of Linear Algebra, and illustrate some applications to Engineering.
Syllabus
- Solution of the matrix equation Ax = b : Gaussian elimination, LU factorization.
- Four fundamental subspaces of a matrix.
- Least squares solution of Ax = b for an m × n matrix with n independent columns: Solving AT^ A x = AT^ b , Gram-Schmidt orthogonalization, QR decomposition.
- Solution of Ax = λ x , eigenvectors and eigenvalues.
- Singular Value Decomposition
Text book
Gilbert Strang, Linear Algebra and its Applications, Harcourt Brace Jovanich 3rd edition,1988. EC 62
Examples Papers
7/7 was issued 25th^ February. 7/8 will be issued 11th^ March.
Notes
These handouts contain some gaps to be filled in as the lectures progress. Completed versions (pdf’s with the missing stuff in red) will eventually appear on my website which can be accessed directly
http://www.eng.cam.ac.uk/~tph
or through a link from the teaching page which contains the syllabus, etc.
1 Example Application
Many problems in engineering involve linear equations. The most recent example you have met is probably in structures, involving a statically indeterminate truss. (a) The equilibrium matrix relates the forces in the members to the applied external forces.
A t = f
Cases like this were analysed in the Structures Course, where A was shown to be:
I^ E
II E III
IV F V
VI (^) F
x
y
x
y
t^ f
t f t t f t
t (^) f W
^
^ ^
^ ^ ^
− − ^
^ ^
^ ^ ^
=^ =
^ ^
− ^ ^ ^
^
^ ^
^ ^ ^
^ ^ ^ ^ −^
The static indeterminacy shows up in the fact that we have 4 equations for 6 unknowns , indicating probably 2 redundant members.
(b) The extensions in all of the members must be compatible with the displacements. Written in matrix form, this introduces the compatibility matrix C , C d = e
A B C D
E F
V I^
II III
VI
IV
W
x
y
l l l
l
( ) E IV V
E.g. joint E R^ →^ f^ x +^ t^ −^ t = tIV
tV
tI
f E
E.g. bar VI
dFx
d F
dFy VI Fx Fy
e = − d − d
N.B. Error on handout
Q4 What set of nodal displacements produce zero extensions in the bars?
i.e. How do we find/describe the set of d , for which there is a solution to C d = 0?
Again for consistency with Q2, we should call this the Null Space of C , but since C = A T , we
prefer to call it the Left Null Space of A.
C d = 0 A T^ d = 0 d TA = 0 (taking the transpose).
These correspond, for this application, to mechanisms.
A similar set of questions tends to crop up in applications of matrix methods to other branches of engineering. The first part of this course, then, is devoted to the general solution of A x = b , and methods to find the four sets of vectors which answer these four questions. i.e. to find the column space, the null space, the row space and the left-null space of the matrix A.
2 The Geometry of n dimensions
Geometrical interpretation is a great help when considering how to solve systems of equations, which in 3 dimensions are planes, lines, etc. In this section, we will try and extend the ideas of lines and planes to dimensions higher than 3.
Considered as a mapping, the 4 × 6 matrix A above maps a 6-dimensional vector into a 4- dimensional one
I II E III E IV F V F VI
x y x y
t t f t f t f t f t
^
I E (^) II E (^) III F IV F V VI
x y x y
e d (^) e d (^) e d e d e e
^
^
^
→^ ^
^
^
In general, an m × n matrix A , transforms an n -dimensional vector x into a corresponding m - dimensional vector b. A x = b
As we move to dimensions higher than 3, most of the familiar vector properties generalise, and only a few do not.
A
b
n
m x n m
m × n
columns rows
n components (^) m components
A AT
4 dimensions
(i) We still have 4 independent unit vectors along the “axis” directions
e 1 =
, e (^) 2 =
, e 3 =
and e 4 =
(ii) Any vector has 4 components x = x e 1 1 (^) + x e 2 2 (^) + x e 3 3 (^) + x e (^44)
(iii) Length is
2 2 2 2 x = x 1 (^) + x 2 (^) + x 3 (^) + x 4
(iv) Dot product survives
x y. = x y 1 1 (^) + x y 2 2 (^) + x y 3 3 (^) + x y 4 4 = x T y =
[ 1 2 3 4 ] (^1) 2 3 4
x x x x y y y y
We can think of this as defining an angle between two vectors cos
x y x y
, and if we do so
x ⊥ y ⇔ x y. = 0
As in 3-d T 2 x = x x. = x x
(v) All of this is compatible with the corresponding definitions in 3-d, and we still have
x e. (^) 1 = x 1 e 1. e 1 = 1 e 1. e 2 = 0 , etc.
with the angle between e 1 and e 2 being 90°, etc.
(vi) An example of something which doesn’t generalise is cross product x × y. We could try using
x y sin θ n^ ˆ, but we come unstuck with n ˆ since, in four dimensions, there are two unit vectors
perpendicular to x and y.
(vii) We can replace 4 by m or n in the above with obvious generalisations.
x = x e 1 1 (^) + x e 2 2 (^) + ... + x en (^) n
The n -dimensional “world” is referred to as n. We live in ^3. An m × n matrix A maps n^ to
m^. n real co-ordinates
2.2 The Column Picture for Simultaneous Equations.
Let us stay in ^3 for the present and consider the problem of solving 3 equations in 3 unknowns.
x + 3 y − z = 11 3 x − 2 y − z = 7 − x + y + 4 z = − 9
The solution of which can be shown to be x = 3 y = 2 z = − 2
This problem can be written in vector form as
x y z
and the vectors on the left hand side are, as expected, the columns of the matrix A.
The problem then is to find which linear combination of the columns on the LHS will give the vector on the RHS. We will refer to this as column visualization or as the column picture.
In this case
1 3 1 11 3 3 2 2 2 1 7 1 1 4 9
If the vectors which are the columns of A are independent (or rather linearly independent ), i.e. you can not express one as a linear combination of the other two, then any vector can be written as a linear combination of them (in a unique way). So the equations are guaranteed to have a (unique) solution for any RHS b.
When might the equations not have a solution?
If the matrix A is singular , then the columns of A are not independent as is the case for the following set
x y z
= ^ − −
A x = b^ A
a 1
b
a 2
a 3
α a 1
β a 2
γ a 3
b
= ^
1 2 3
Using can get anywhere on a plane Using can always get onto that plane
a a a
i.e. xa 1 + y a 2 (^) + z a 3 = b
b − γ a 3 = α a 1 +β a 2
This has the third column lying in the plane spanned by the first two (see below).
i.e. a 3 = α a 1 + β a 2. It follows that A anything must also lie in the plane of a 1 and a 2 :
A x = xa 1 (^) + ya (^) 2 + za 3 (^) = xa 1 (^) + ya 2 (^) + z (^) ( α a 1 (^) + β a (^) 2 ) = (^) ( x + z α (^) ) a 1 (^) + (^) ( y + z β) a 2
If b does not also lie in this plane, then there is no solution.
If b does lie in this plane, then there is an infinite number of solutions.
When we are dealing with 3 × 3 matrices, we know how to determine whether the columns of a matrix are independent and, if so, whether a given vector can be written in terms of them, although the methods we know are laborious. For the matrix referred to above:
(i) If we write the equations in the form A x = b , then
Determinant A = ( ) ( )
This means that A is singular (has no inverse).
(ii) The columns of A can not, therefore, be independent. To prove the columns are not independent, we write the third one as a linear combination of the first two.
Put
The first two equations give
α= − β= − (and this also satisfies the third).
(iii) Since A x = x a 1 1 + x a 2 2 + x a 3 3 = ( x 1 + α x 3 ) a 1 + ( x 2 + β x 3 ) a 2 , for a solution, b must
also be a combination of the first two columns of A :-
a 1
b No solution
a 2
a 3
b (^) ∞ solutions a 1
a 3
a 2
1 2 2 3 or any linear combination
b a a a a
λ μ λ μ
= ′^ + ′
m^ is itself a vector space and “smaller” ones within it are said to be sub-spaces of m.
For example: The non-trivial sub-spaces of ^3 are
(a) x
= u lines through the origin (b) x
= u + μ v planes through the origin.
(c) x
= u + μ v + ν w = whole of ^3
2.4 How many dimensions?
We would naturally describe a line as a one-dimensional sub-space of ^3 and a plane as a two- dimensional subspace since within these spaces we have one and two “degrees of freedom”. The term “dimension” is thus used in two senses – number of degrees of freedom and number of co- ordinates. Every point on a line has 3 co-ordinates and is thus a three-dimensional vector but the line is a one-dimensional object. This ambiguity does not, in general, cause confusion.
In drawing the diagrams above, we have assumed that the vectors u , v and w are independent. For case (d), if u , v , w are not independent (i.e. one of them can be written as a sum of the other two), we can drop it from the list of vectors necessary to span the space. If
w = α u + β v x = (^) ( + να (^) ) u + (^) ( μ + νβ) v x = λ ′ u^ +μ′ v
and we only have a plane. Similarly considerations apply for (b) and (c).
A set of vectors u , v , ... which span the space and are also independent are said to form a basis and the number of these is the dimension of the sub-space. The basis of a vector sub-space is not unique
- any full set of independent vectors will do, but it will always contain the same number of vectors.
u (^) v
(^0) u
(^0) v u
w
i.e. less than m spanning vectors
u ≠ 0
e.g. The sub-spaces of ^3 which are given by
1 0 1 1 0 1
x λ μ
= ^ +
x λ μ
= ′^ ^ + ′^
x λ μ ν
= ′′^ ^ + ′′^ ^ ^ + ′′
turn out to be the same and, moreover, these are not the only way of representing the plane. Whatever vectors are used, however, it needs two and only two independent vectors to describe it.
Sets of Independent Vectors (typical properties)
- Complete the following, for vectors in ^3 :
Two vectors are linearly dependent if they lie on the same line (through the origin)
Three vectors are linearly dependent if they lie in the same plane (through the origin)
Four vectors are certain to be linearly dependent.
What is the maximum number of vector in an independent set in ^6? 6
The mathematical test for linear independence is:
The vectors xi , i = 1, ... , n are linearly independent if, whenever
λ 1 x 1 + λ 2 x 2 + ... + λ n xn = 0 for any scalars λ i , i = 1, ... , n ,
then we must have λ 1 = λ 2 = ... = λ n = 0
(If one of the λ's is non-zero, then we can solve this equation for the vector it multiplies in terms of
the others).
- Show that if xi , i = 1, ... , n are a basis for the vector space S , then every vector in S has a
unique representation in terms of them.
x ∈ S s = λ 1 x 1 + λ 2 x 2 + ... + λ n xn.
If also s = μ 1 x 1 + μ 2 x 2 + ... + μ n xn then subtracting gives
0 = ( λ 1 − μ 1 ) x 1 + ( λ 2 − μ 2 ) x 2 + ... + ( λ n − μ n ) xn 0 = λ 1 − μ 1 = λ 2 − μ 2 etc
2.5 Column Space
The vector space spanned by the columns of a general m × n matrix A is called the column space of A. The dimension (in the degrees of freedom sense) of column space is called the rank of A. So, for example, if A is a 3 × 3 matrix
If rank( A ) = 3 column space = whole of ^3
If rank( A ) = 2 column space = a plane. (We lose 1 dimension in the mapping)
If rank( A ) = 1 column space = a line (We lose 2 dimensions in the mapping)
If rank( A ) = 0 column space = 0 (We lose 3 dimensions in the mapping).
If we lose dimensions, then we can not reverse a mapping. If A is n × n , then if it loses dimensions, rank( A ) < n , it is singular. If it doesn’t, rank( A ) = n , then its inverse will exist.
