Download Linear Algebra: Vector Spaces, Subspaces, and Transformations - Prof. E. Cliff and more Study notes Aerospace Engineering in PDF only on Docsity!
n
0 v 0 v
R
e.g.
sum product
Eugene M. Cliff
August 15, 1996
- Background Linear Algebra -
1 Introduction
2 Vector Spaces
These notes are intended to supplement the treatment in the Gill-Murray & Wright text [1]. While the discussion in the text is mostly adequate for our needs, it does present some unfortunate limitations in its outlook. We shall present some material to broaden the approach. Our presentation is heavily influenced by Luenberger [2]. There are a number of excellent texts for most of this material ( [3]).
A vector space is a set of objects (the vectors), the real (or complex) field of scalars and two operations connecting them. The operations are: vector addition and multiplication by scalars. Associated with any two vectors is a (unique) third vector - their. Also, given a vector and a scalar there is a unique vector - the. The operations must enjoy certain properties (vector addition is commutative and associative). These are generally stated in the form of certain axioms that the vector operations must satisfy ( see [2], pp 11-12). Among these axioms is the requirement that the set contain a unique vector , the additive identity element (meaning that + = , for all elements). The usual ‘vectors’ in are the most common example. Addition means add component-wise. Somewhat more abstractly, we also mention the space of continuous functions on the interval [0 1]. Vector-addition and scalar multiplication are defined in the obvious way. To be rigorous one should
1 2 p
( ) ( ) ( )
p p
p i i^ i p i ı^ ı^ ı
v v v v
v
S v v v S
v 0 S v 0
R
S
new 1 1 2 2
new
=
=
2
1 2 3
α α α
α
α α
α , α , α
e.g.
i.e.
proper subspace
2.1 Linear Combinations
2.2 Linear Dependence
2.3 Subspaces
prove the ‘closure’ property, that the sum of two continuous functions is a continuous function.
Given a finite set of vectors, and a corresponding set of scalars we can form a ‘new’ vector by the operation
= + + +
In this case we say that is a linear combination of the underlying vectors.
Strictly said, linear dependence is best thought of as a property of certain finite-sets. That is, given a finite set of vectors, say we say that is a linearly dependent set iff there is a set of scalars (at least one such set), not all zero, such that the corresponding linear combination is the zero vector ( Σ = ). A set of vectors that is not linearly dependent is said to be linearly independent. Note that if is a linearly independent set and if Σ = , then necessarily all of the scalars ( ) must be zero. With this definition, any set that includes the zero-vector is linearly de- pendent. For another example of a linearly dependent set, consider the vector space and the set of vectors given by
1 0
In this case the scalars ( = 1 = 1 = 1) demonstrate the depen- dence.
It happens that certain sets of vectors have the ‘closure’ property, meaning that addition or scalar multiplication of anything in this set produces another element of the set. Such sets are called subspaces - they are vector spaces in their own right. It’s clear that any subspace must contain the zero vector. We agree that the set consisting of only the zero vector is a subspace and the whole space is also a subspace. We reserve the term to
n n
n
n
U V U V
v v v v
v
R
S
S
basis
dimension
α α α ,
α , α ,... , α
n
sample 1 1 2 2
1 2 sample
2.7 A Vector and Its Representation
any more vectors the span of the thus-reduced set will not be the same as the span of the original. A spanning set that is linearly independent is said to be a for the subspace. It turns out that the number of elements in a basis is a property of the subspace - this is the of the subspace. Again, it requires proof to show that if we start with two sets with the same span, this process of throwing out results in linearly independent sets with the same number of elements. Our description characterizes a basis as a minimal spanning set. If we throw out any more elements then the span of the resulting set is decreased. It’s sometimes useful to think of starting with a small set and adding vectors
- to produce the desired spanning set. In this view if we add a vector to a basis and keep the same spanning set, it’s clear that the new set is not linearly independent. Hence one can also think of a basis as the largest linearly independent set (with a given span). Any basis has a naturally associated subspace; namely the space spanned by the basis. While the basis is not unique (a given subspace has many bases) the number of elements in any such basis is always the same. As noted above, this number is the dimension of the subspace. When adding subspaces we have dim[( + )] dim[ ] + dim[ ], with equality iff the sum is direct.
Once we have a basis then any vector can be uniquely described by providing the scalar components in its representation. That is, if
= + + +
then the n-tuple of scalars ( ) represent the vector in terms of the given basis. Perhaps the canonical example of a -dimensional space is , wherein the vectors are these n-tuples of scalars. In other examples the basis is fixed so that we can blur this distinction and think of the collection of scalars as the vector. In most cases this causes no harm, but we should be aware of it. To make this more concrete, consider the space of continous functions on the unit interval, and the subspace spanned by the set of vectors = cos( ) sin( ). Since the two vectors are linearly independent, the set is a basis for this
=1 =
p i i^ i^
p i i^ i
X Y
Y X
D X
D X Y
Y
u u
Y
X Y X Y
U V X Y
x X x u
x u u
3 Transformations and Functionals
α, β f t α t β t
f α, β
T
T α α T.
T n m
, α
T T α α T.
blur
transformation map
functional
linear func- tional
3.1 Linear Transformations
3.2 Representation of a Linear Transformation
two-dimensional subspace. Now however, there should be a clear distinction between the pair ( ) and the function ( ) = cos( )+ sin( ). In fact, we derive alot of benefit from the fact that one can the distinction between ( ) and the pair ( ).
One of the things we do with spaces (or their subspaces) is to define certain related maps or functions. Suppose and are two vector spaces, then a rule that associates a unique element of to every element of is a or a. In some settings this idea must be generalized so that the rule is defined for only some proper subset. We write this as :
A special case of particular interest occurs when the image space is the scalar field; such a transformation is called a.
Linearity of maps is a natural idea in the vector space setting; it means that applying the transformation to a linear combination of vectors, produces the same result as applying the transformation to each component vector and then forming the linear combination. In symbols, we have
(Σ ) = Σ ( )
In the case when is the (real or complex) field we speak of a .
Suppose we are in the situation : , that dim[ ] = , dim[ ] = and that we have bases, say , and , for and , respectively. Then for any we have = Σ , so that
( ) = (Σ ) = Σ ( )
≤ ≤ ∫
∫
1 2 1 2
t
n n i ,n i i T
0 1
1 0
1 2 1 2 =
1 0
4.1 Inner Products
u v u v
v v
v
v
u v
u v v u
u u v u v u v
u v u v
v v v v v 0
v v v X
u v
normed space
e.g.
inner-product pair
inner-product space pre-Hilbert
α α
C ,
v t C , v t dt C ,
< α , > α < , >
< , > < , >
< , >
< α , α ,... , α , β , β ,... , β > α β α β.
< , > u t v t dt
A vector space and an associated norm are called a. A commonly cited non-Euclidean example is [0 1] - the space of real-valued continuous functions on the interval [0 1]. The vector space structure has been noted above and the norm in this case is given by max ( ). The notation [0 1] means the vector space with this norm. Note that we can define a different norm on the same vector space ( ( ). The normed space with this integral norm is different from [0 1].
It turns out that there is a way to generalize the Euclidean case that will produce a special class of norms and normed spaces. An is a complex (possibly real) valued function that assigns to any of vectors a scalar value - their inner (or dot) product. We shall write this as:. Again, we require certain axioms to end up with a useful concept
- = (overbar means complex conjugate)
- ( + ) = +
- =
- 0 and = 0 iff =.
With these properties it turns out that is a norm. The vec- tor space , together with the inner-product define an (sometimes called a space). Here again, the most common exam- ple is the Euclidean case:
( ) ( ) = Σ =
Note we have mixed in some matrix notation here. A common non-Euclidean example is the space of continuous functions along with the inner-product ( ) ( ).
⊕
T T T
2 2 2
Y
X Y
Y X
⊥ ⊥
⊥
⊥ ⊥⊥ ⊥ ⊥
∗
∗
∗ ∗
∗
T
< T , >
T
T
T
< , > < T , >.
T T
T T
M
< T , > < , >
M x y x M y.
4.2 Orthogonality
4.3 Adjoints and Transposes
u v u u
S
S
S S S
S S
S X
S S X
X
S
S
S S
S S
S S
X Y
X Y x X y Y x y Y y x X x Y x Y y X X x X
x x x y x X
Y X
x y x x
orthogonal
orthogonal complement -perp
uniquely
inner-product
adjoint
Just as in the Euclidean case we say that two vectors are if the inner-product is zero. In this case one can easily show the Pythagorean Theorem holds: ( + ) = +. The idea of orthogonality of two vectors can be extended by saying that a vector is orthogonal to a set iff it is orthogonal to every element of the set. For given the collection of all vectors orthogonal to the given set is called the of ( we say and write ). In fact, one can prove that is a subspace (even if isn’t). If we start with a set that is a subspace , then we have a neat decomposition:
=
This means that any vector in can be decomposed into a part that’s in and a second part in its orthogonal complement. In the plane we picture a line through the origin ( ) and a second line at right-angles to it ( ). In three-dimensions we might have a plane for ( ) and a perpendicular line for the orthogonal complement. Finally, if is a subspace then
(meaning [ ] ) gets us back to.
Suppose we have two spaces ( and ) and a linear map between them :. For a given and we can compute the real-number , the inner-product in the -space. Now we get a little wierd. For the given and fixed suppose we want to evaluate the inner-product for a variety of vectors. We complain that this is alot of work: first map the new to by computing ( ), and then compute the -space inner-product. For this fixed and is there an element in that will work with the -space inner-product? We are asking for an element so that
=
In fact, this requirement defines a new linear operator :. called the of the operator. In the real-Euclidean case, where is represented by a matrix the calculation looks like:
= ( ) = ( )
References
[1] Practical Optimization, Gill, P.E., Murray, W. and Wright, M.H., Aca- demic Press, 1982
[2] Optimization by vector space methods, Luenberger, David G., Wiley, 1969, QA402.5 L
[3] Finite-dimensional vector spaces, Halmos, Paul R., Van Nostrand, 1958, QA261 H33 1958
