Download Vector Spaces: Subspaces, Linear Independence, and Basis and more Study notes Algebra in PDF only on Docsity!
Chapter 1
Vectors and Vector Spaces
1.1 Vector Spaces
Underlying every vector space (to be defined shortly) is a scalar field F. Examples of scalar fields are the real and the complex numbers
R := real numbers C := complex numbers.
These are the only fields we use here.
Definition 1.1.1. A vector space V is a collection of objects with a (vector) addition and scalar multiplication defined that closed under both operations and which in addition satisfies the following axioms:
(i) (α + β)x = αx + βx for all x ∈ V and α, β ∈ F
(ii) α(βx) = (αβ)x
(iii) x + y = y + x for all x, y ∈ V
(iv) x + (y + z) = (x + y) + z for all x, y, z ∈ V
(v) α(x + y) = αx + αy
(vi) ∃O ∈ V z 0 + x = x; 0 is usually called the origin
(vii) 0x = 0
(viii) ex = x where e is the multiplicative unit in F.
8 CHAPTER 1. VECTORS AND VECTOR SPACES
The “closed” property mentioned above means that for all α, β ∈ F and x, y ∈ V
αx + βy ∈ V
(i.e. you can’t leave V using vector addition and scalar multiplication). Also, when we write for α, β ∈ F and x ∈ V
(α + β)x
the ‘+’ is in the field, whereas when we write x + y for x, y ∈ V , the ‘+’ is in the vector space. There is a multiple usage of this symbol.
Examples.
(1) R 2 = {(a 1 , a 2 ) | a 1 , a 2 ∈ R} two dimensional space. (2) Rn = {(a 1 , a 2 ,... , an) | a 1 , a 2 ,... , an ∈ R}, n dimensional space. (a 1 , a 2 ,... , an) is called an n-tuple. (3) C 2 and Cn respectively to R 2 and Rn where the underlying field is C, the complex numbers.
(4) Pn =
l n j=
aj xj^ | a 0 , a 1 ,... , an ∈ R
M
is called the polynomial space of
all polynomials of degree n. Note this includes not just the polynomials of exactly degree n but also those of lesser degree. (5) fp = {(ai,... ) | ai ∈ R, Σ|ai|p^ < ∞}. This space is comprised of vectors in the form of infinite-tuples of numbers. Properly we would write fp(R) or fp(C) to designate the field.
(6) TN =
F N
n=
an sin nπx | a 1 ,... , an ∈ R
k , trigonometric polynomials.
Standard vectors in Rn
e 1 = (1, 0 ,... , 0) e 2 = (0, 1 , 0 ,... , 0) e 3 = (0, 0 , 1 , 0 ,... , 0) .. . en = (0,^0 ,... ,^0 ,^ 1)
These are the unit∗^ vec- tors which point in the n orthogonal∗ directions.
10 CHAPTER 1. VECTORS AND VECTOR SPACES
e 1 = (1, 0) e 2 = (0, 1) (^) (1,0)
1
2
e
Graphical representa- tion of e 1 and e 2 in the usual two dimensional plane.
Recall the usual vector addition in the plane uses the parallelogram rule
y
x+y
For R^3 , the standard vectors are
e 1 = (1, 0 , 0) e 2 = (0, 1 , 0) e 3 = (0, 0 , 1)
e
e
e (0,1,0)
2
3
1
Graphical representa- tion of e 1 , e 2 , and e 3 in the usual
Linear algebra is the mathematics of vector spaces and their subspaces. We will see that many questions about vector spaces can be reformulated as questions about arrays of numbers.
1.1.1 Subspaces
Let V be a vector space and U ⊂ V. We will call U a subspace of V if U is closed under vector addition, scalar multiplication and satisfies all of the vector space axioms. We also use the term linear subspace synonymously.
1.1. VECTOR SPACES 11
Examples. Proofs will be given later
let V = R^3 = {(a, b, c) | a, b, c ∈ R} (1.1) U = {(a, b, 0) | a, b ∈ R}.
Clearly U ⊂ V and also U is a subspace of V.
let v 1 , v 2 ∈ R^3 (1.2) W = {av 1 + bv 2 | a, b ∈ R} W is a subspace of R^3.
In this case we say W is “spanned” by {v 1 , v 2 }. In general, let S ⊂ V , a vector space, have the form
S = {v 1 , v 2 ,... , vk}.
The span of S is the set
U =
3 k
j=
aj vj | a 1 ,... , ak ∈ R
We will use the notion
S(v 1 , v 2 ,... , vk)
for the span of a set of vectors.
Definition 1.1.2. We say that
u = a 1 v 1 + · · · + akvk
is a linear combination of the vectors v 1 , v 2 ,... , vk.
Theorem 1.1.1. Let V be a vector space and U ⊂ V. If U is closed under vector addition and scalar multiplication, then U is a subspace of V.
Proof. We remark that this result provides a “short cut” to proving that a particular subset of a vector space is in fact a subspace. The actual proof of this result is simple. To show (i), note that if x ∈ U then x ∈ V and so
(ab)x = ax + bx.
Now ax, bx, ax + bx and (a + b)x are all in U by the closure hypothesis. The equality is due to vector space properties of V. Thus (i) holds for U. Each of the other axioms is proved similarly.
1.2. LINEAR INDEPENDENCE AND LINEAR DEPENDENCE 13
Example 1.1.3. More subspaces of R 3. There are two other important methods to construct subspaces of R 3. Besides the set builder notation used above, we have just considered the method of spanning sets. For example, let S = {v 1 , v 2 } ⊂ R 3. Then S (S) is a subspace of R 3. Simi- larly, if T = {v 1 } ⊂ R 3. Then S (T ) is a subspace of R 3. A third way to construct subspaces is by using inner products. Let x, w ∈ R 3. Ex- pressed in coordinates x = (x 1 , x 2 , x 3 ) and w = (w 1 , w 2 , w 3 ). Define the inrner product of x and w by x · w = x 1 w 1 + x 2 w 2 + x 3 w 3. Then Uw = {x ∈ R 3 | x · w = 0} is a subpace of R 3. To prove this it is neces- sary to prove closure under vector addition and scalar multiplication. The latter is easy to see because the inner product is homogeneous in α, that is, (αx) · w = αx 1 w 1 + αx 2 w 2 + αx 3 w 3 = α (x · w). Therefore if x · w = 0 so also is (αx) · w. The additivity is also straightforward. Let x, y ∈ U. Then the sum
(x + y) · w = (x 1 + y 1 ) w 1 + (x 2 + y 2 ) w 2 + (x 3 + y 3 ) w 3 = (x 1 w 1 + x 2 w 2 + x 3 w 3 ) + (y 1 w 1 + y 2 w 2 + y 3 w 3 ) = 0 + 0 = 0
However, by choosing two vectors v, w, ∈ R 3 we can define Uv,w = {x ∈ R 3 | x · y = 0 and x · w = 0}. Establishing Uv,w is a subspace of R 3 is proved similarly. In fact, what is that both these sets of subspaces, those formed by spanning sets and those formed from the inner products are the same set of subspaces. For example, referring to the previous example, it follows that V 13 = S (e 1 , e 3 ) = Ue 2. Can you see how to correspond the others?
1.2 Linear independence and linear dependence
One of the most important problems in vector spaces is to determine if a given subspace is the span of a collection of vectors and if so, to deter- mine a spanning set. Given the importance of spanning sets, we intend to examine the notion in more detail. In particular, we consider the concept of uniqueness of representation. Let S = {v 1 ,... , vk} ⊂ V , a vector space, and let U = S(v 1 ,... , vk) (or S(S) for simpler notation). Certainly we know that any vector v ∈ U has the representation
v = a 1 v 1 + · · · + akvk
for some set of scalars a 1 ,... , ak. Is this representation unique? Or, can we
14 CHAPTER 1. VECTORS AND VECTOR SPACES
find another set of scalars b 1 ,... , bk not all the same as a 1 ,... , ak respec- tively for which
v = b 1 v 1 + · · · + bkvk.
We need more information about S to answer this question either way.
Definition 1.2.1. Let S = {v 1 ,... , vk} ⊂ V , a vector space. We say that S is linearly dependent (l.d.) if there are scalars a 1 ,... , ak not all zero for which
a 1 v 1 + a 2 v 2 + · · · + akvk = 0. (T)
Otherwise we say S is linearly independent (l.i.).
Note. If we allow all the scalars to be zero we can always arrange for (T) to hold, making the concept vacuous.
Proposition 1.2.1. If S = {v 1 ,... , vk} ⊂ V , a vector space, is linearly dependent, then one member of this set can be expressed as a linear combi- nation of the others.
Proof. We know that there are scalars a 1 ,... , ak such that
a 1 v 1 + a 2 v 2 + · · · + akvk = 0
Since not all of the coefficients are zero, we can solve for one of the vectors as a linear combination of the other vectors.
Remark 1.2.1. Actually we have shown that there is no vector with a unique representation in S(S).
Corollary 1.2.1. If 0 ∈ S = {v 1 ,... , vk}, then S is linearly dependent.
Proof. Trivial.
Corollary 1.2.2. If S = {v 1 ,... , vk} is linearly independent then every subset of S is linearly independent.
16 CHAPTER 1. VECTORS AND VECTOR SPACES
Remark 1.3.1. Note how we resolved the linearly dependent/linearly in- dependent issue by converting a vector problem to a numbers problem. This is at the heart of linear algebra.
Exercise. Let S = {v 1 , v 2 } = {(1, 0 , 1), (1, − 1 , 0)} ⊂ R^3. Show that S is linearly independent and therefore a basis of S(S).
1.4 Extension to a basis
In this section, we show that given a linearly independent set of vectors from a vector space with a finite spanning set, it is possible add to this set more vectors until it becomes a basis. Thus any set of linearly independent vectors can be a part (subset) of a basis.
Theorem 1.4.1 (Extension to a basis). Assume that the given vector space V has a finite spanning set S 1 , i.e. V = S(S 1 ). Let S 0 = {x 1 ,... , xf} be a linearly independent subset of V so that S(S 0 ) V. Then, there is a subset S 1 I of S 1 , such that S 0 ∪ SI^ is a basis for V.
Proof. Our intention is to add vectors to S 0 keeping it linearly independent and eventually becoming a basis. There are a couple of steps. Steps.
- Since S(S 1 ) S S(S 0 ), there is a vector y 1 ∈ S 1 such that S 0 , 1 = {S 0 , y 1 } is linearly independent and thus S(S 0 , 1 ) S S(S 0 ).
- Continue this process generating sets
S 0 , 1 = {S 0 , y 1 } S 0 , 2 = {S 0 , 1 , y 2 } .. . S 0 ,j = {S 0 ,j− 1 , yj− 1 } .. .
At each step S 0 , 1 , S 0 , 2 ,... are linearly independent sets. Since S 1 is finite we must eventually have that
S(S 0 ,m) = S(S 1 ) = V.
- Since S 0 ,m is linearly independent and spans V , it must be a basis.
1.5. DIMENSION 17
Remark 1.4.1. In the proof it was important to begin with any spanning set for V and to extract vectors from it as we did. Assuming merely that there exists a finite spanning set and extracting vectors directly from V leads to a problem of terminus. That is, when can we say that the new linearly independent set being generated in Step 2 above is a spanning set for V? What we would need is a theorem that says something to the effect that if V has a finite basis, then every linearly independent set having the same number of vectors is also a basis. This result is the content of the next section. However, to prove it we need the Extension theorem.
Corollary 1.4.1. If S = {v 1 ,... , vk} is linearly dependent then the repre- sentation of vectors in S(S) is not unique.
Proof. We know there are scalars a 1 ,... , ak not all zero, for which
a 1 v 1 + · · · + akvk = 0
let v ∈ S(S) have the representation
v = b 1 v 1 + b 2 v 2 + · · · + bkvk.
Then we also have the representation
v = (a 1 + b 1 )v 1 + (a 2 + b 2 )v 2 + · · · + (ak + bk)vk
establishing the result.
Remark 1.4.2. The upshot of this construction is that we can always con- struct a basis from a spanning set. In actual practice this process may be quite difficult to carry out. In fact, we will spend some time achieving this goal. The main tool will be matrix theory.
1.5 Dimension
One of the most remarkable features of vector spaces is the notion of dimension. We need one simple result that makes this happen, the basis theorem.
Theorem 1.5.1 (Basis Theorem). Let S = {v 1 ,... , vk} ⊂ V be a basis for V. Then every basis of V has k elements.
1.5. DIMENSION 19
(b) If at least one vector in S is nonzero (that is V W= { 0 }, the smallest vector space), then there is a subset S 0 ⊂ S that is linearly independent and spans V.
Proof. (Left to reader.)
Definition 1.5.1. The dimension of a vector space V is the (unique) num- ber of vectors in a basis of V. We write dim(V ) for the dimension.
Remark 1.5.1. This definition make sense possible only because of our basis theorem from which we are assured all every linearly independent spanning sets of V , that is all bases, have the same number of elements.
Examples.
(1) dim(Rn) = n,
(2) dim(Pn) = n + 1.
Exercise. Let M = all rectangular arrays of two rows and three columns with real entries. Find a basis for M, and find the dimension of M. Note
M =
F}
a b c d e f
] ee ee a, b, c, d, e, f ∈ R
k
Example 1.5.1. Pn = {anxn^ + an− 1 xn−^1 + · · · + a 1 x + a 0 = 0} is the vector space of polynomials of degree n. We claim that the powers, x^0 = 1, x, x^2 ,... , xn^ are linearly independent, and since
Pn = S(1, x,... , xn)
they form a basis of Pn.
Proof. There are several ways we can prove this fact. Here is the most direct and it requires essentially no machinery. Suppose they are linearly dependent, which means that there are coefficients a 0 , a 1 ,... , an so that
anxn^ + an− 1 xn−^1 + · · · + a 1 x + a 0 = 0, (T)
the function. (This functional view is critically important because every polynomial has roots.) There must be a coefficient which is nonzero and which corresponds to the highest power. Let us assume that an W= 0, for convenience, and with no loss in generality.
20 CHAPTER 1. VECTORS AND VECTOR SPACES
Solve for xn^ to get
xn^ = − an− 1 an
xn−^1 + · · · + − a 1 an
x − a 0 an
(TT)
Now compute the ratio of this expression divided by xn^ on both sides, and let x → ∞. The left side of course will be 1. Again for convenience we take n = 2. So, condensing terms we will have
b 1 x + b 0 x^2 = b 1
w 1 x
W
w 1 x^2
W
where bj = −aj /a 2. But as x → ∞ the expression b 1
D 1
x
i
D 1
x^2
i → 0. This is a contradiction. It cannot be that the functions 1, x, and x^2 are linearly dependent. In the general case for n we have
bn− 1
w 1 x
W
w 1 x^2
W
w 1 xn
W
where bj = −aj /an. Apply the same limiting argument to obtain the con- tradiction. Thus
T = { 1 , x,... , xn}
is a basis of Pn. A calculus proof is available. It is also based on the fact that if the powers are linearly independent and (T) holds, then we can assume that the same relation (TT) is true. Now take the nth^ derivative of both sides. We obtain n! = 0
a contraction, and the result if proved Finally, one more technique used to prove this result is by using the Fundamental Theorem of Algebra.
Theorem 1.5.2. Every polynomial (T) of exactly nth^ degree (i.e. with an W= 0) has exactly n roots counted with multiplicity (i.e. if q(x) = qnxn^ + qn− 1 xn−^1 + · · · + q 1 x + q 0 ∈ Pn(C), qn W= 0 then the number of solutions of q(x) = 0 is exactly n).
From (T) above we have an nth^ degree polynomial that is zero for every x. Thus the polynomial is zero, and this means all the coefficients are zero. This is a contradiction to the hypothesis, and therefore the theorem is proved.
22 CHAPTER 1. VECTORS AND VECTOR SPACES
Theorem 1.5.3 (Uniqueness). Let S = {v 1 ,... , vk} be a basis of V. Then each vector v ∈ V has a unique representation with respect to S.
Proof. Since S(S) = V we have that
v = a 1 v 1 + a 2 v 2 + · · · + akvk
for some coefficients a 1 , a 2 ,... , ak in the given field. (This is the represen- tation of v with respect to S.) If it is not unique there is another
v = b 1 v 1 + b 2 v 2 + · · · + bkvk.
So, subtracting we have
(a 1 − b 1 )v 1 + (a 2 − b 2 )v 2 + · · · + (ak − bk)vk = 0
where the differences aj −bj are not all zero. This implies that S is a linearly dependent set.
Theorem 1.5.4. Suppose that S = {v 1 ,... , vk} is a basis of the vector space V. Suppose that T = {w 1 ,... , wm} is a linearly independent subset of V. Then m ≤ k.
Proof. We know that S is a linearly independent spanning set. This means that every linearly independent set of k vectors is also a spanning set. There- fore, m > k renders a contradiction as T 0 = {w 1 ,... , wk} is a spanning set and wk+1 ∈ S(T 0 ).
Definition 1.5.2. If A is any set we define
|A| := cardinality of A,
that is to say |A| is the number of elements of A.
Example 1.5.3. Let T = { 1 , x, x^2 , x^3 }. Then |T | = 4.
Theorem 1.5.5. Both Rk and Ck are k-dimensional and Sk = {e 1 , e 2 ,... , ek} is a basis of both.
Proof. It is easy to see that e 1 ,... , ek are linearly independent, and any vector x in Rk has the form
x = a 1 e 1 + a 2 e 2 + · · · + akek
for a 1 ,... , ak ∈ R. Thus Sk is a linearly independent spanning set and hence a basis of Rk.
1.6. NORMS 23
Question: What single change to the proof above gives the theorem for Ck?
The following results follow easily from previous results.
Theorem 1.5.6. Let V be a k-dimensional vector space.
(i) Every set T with |T | > k is linearly dependent.
(ii) If D = {v 1 ,... , vj } is linearly independent and j < k, then there are vectors vf 1 ,... , vfk−j ∈ V such that
D ∪ {vf 1 ,... , vfk−j }
is a basis of V.
(iii) If D ⊂ V , |D| = k, and D is either a spanning set for V or linearly independent, then D is a basis for V.
1.6 Norms
Norms are a way of putting a measure of distance on vector spaces. The purpose is for the refined analysis of vector spaces from the viewpoint of many applications. It is also to all the comparison of various vectors on the basis of their length. Ultimately, we wish to discuss vector spaces as representatives of points. Naturally, we are all accustomed to the “shortest distance” distance from the Pythagorean theorem. This is an example of a norm, but we shall consider them as real valued functions with very special properties.
Definition 1.6.1. Norms on vector spaces over C, or R. Let V be a vec- tor space and suppose that , · , : V → R+^ is a function from V to the nonnegative reals for which
(i) ,x, ≥ 0 for all x ∈ V and ,x, = 0 if and only if x = 0 (ii) ,αx, = |α|,x, for all α ∈ C, R and x ∈ V (iii) ,x + y, ≤ ,x, + ,y, for all x, y ∈ V “The Triangle inequality”.
Then , · , is called a norm on V. The second condition is often termed the (positive) homogeneity property.
Remark 1.6.1. The notation is a substitute function notation. The ex- pression , · ,, without the vector, is just the way a norm is expressed.
1.6. NORMS 25
Then
Σ(ai + tbi)^2 = Σa^2 i − 2
Σa^2 i Σaibi Σaibi +
D
Σa^2 i
i 2
(Σaibi)^2
Σb^2 i
= −Σa^2 i +
D
Σa^2 i
i 2 Σb^2 i (Σaibi)^2
=
D
Σa^2 i
i w −1 + Σa^2 i Σb^2 i (Σaibi)^2
W
Since the left side is ≥ 0 and since Σa^2 i ≥ 0, we must have that
w −1 + Σa^2 i Σb^2 i (Σaibi)^2
W
Solving this inequality we have
(Σaibi)^2 ≤ Σa^2 i Σb^2 i.
Now that square roots to get the result.
To prove that (i) is a norm, we note that conditions (i) and (ii) are straightforward. The truth of condition (iii) is a consequence of another famous result.
Theorem 1.6.1 (Minkowski). ,x + y, 2 ≤ ,x, 2 + ,y, 2.
Proof.
Σ(ai + bi)^2 = Σa^2 i + 2Σaibi + Σb^2 i ≤ Σa^2 i + 2
D
Σa^2 i
i 1 / 2 D Σb^2 i
i 1 / 2
=
pD Σa^2 i
i 1 / 2
D
Σb^2 i
i 1 / 2 Q 2 .
Taking square roots gives the result.
Continuity and Equivalence of Norms
Lemma 1.6.2. Every vector norm on Cn is continuous in the vector com- ponents.
26 CHAPTER 1. VECTORS AND VECTOR SPACES
Proof. Let x ∈ Cn and ,·, some norm on Cn. We need to show that if the vector δ → 0, in components, then ,x + δ, → ,x,. First, by the triangle inequality
,x + δ, ≤ ,x, + ,δ, or ,x + δ, − ,x, ≤ ,δ,
Similarly
,x, ≤ ,x + δ − δ, ≤ ,x + δ, + ,δ, or − ,δ, ≤ ,x + δ, − ,x,
Therefore
|,x + δ, − ,x,| ≤ ,δ,
Now expressing δ in components and standard bases vectors, we write δ = δ 1 e 1 + · · · + δnen and
,δ, ≤ |δ 1 | ,e 1 , + · · · + |δn| ,en, ≤ max 1 ≤i≤n |δi| (,e 1 , + · · · + ,en,) ≤ M max 1 ≤i≤n |δi|
where M = ,e 1 , + · · · + ,en,. We know that if δ → 0 in components, then max 1 ≤i≤n |δi| → 0. Therefore |,x + δ, − ,x,| → 0, as well.
Definition 1.6.2. Let ,·,a and ,·,b be two vector norms on Cn. We say that these norms are equivalent if there are postive constants m, M such that for all x ∈ Cn
m ,x,a ≤ ,x,b ≤ M ,x,a
The remarkable fact about vector norms on Cn is that they are all equiv- alent. The only tool we need to prove this is the following result: Every continuous function on a compact set of Cn assumes its maximum (and minimum) on that set. The term “compact” refers to a particular kind of set K, one which is both bounded and closed. Bounded means that for maxx∈K ,x, ≤ B < ∞ and closed means that if limn→∞ xn = x, then x ∈ K.
