Download Linear Algebra Lecture Notes: Vector Spaces and Linear Independence and more Summaries Linear Algebra in PDF only on Docsity!
A Do It Yourself Guide to
Linear Algebra
Lecture Notes based on REUs, 2001-
Instructor: L´aszl´o Babai
Notes compiled by Howard Liu
1 Vector Spaces
1.1 Basics
Definition 1.1.1. A vector space (over the reals) is a set V with
(a) addition V × V → V , (x, y) 7 → x + y, and
(b) scalar multiplication R × V → V , (α, x) 7 → αx,
satisfying following axioms
(a) (V, +) is an abelian group, i. e.,
(a1) (∀x, y ∈ V )(∃! x + y ∈ V ), (a2) (∀x, y ∈ V )(x + y = y + x) (commutative law), (a3) (∀x, y, z ∈ V )((x + y) + z = x + (y + z)) (associative law), (a4) (∃ 0 ∈ V )(∀x)(x + 0 = 0 + x = x) (existence of zero), (a5) (∀x ∈ V )(∃(−x) ∈ V )(x + (−x) = 0),
b) (b1) (∀x ∈ V )(∀α ∈ R)(∃! αx ∈ V ) (b2) (∀α, β ∈ R)(∀x ∈ V )(α(βx)) = (αβ)x (“associativity” linking two operations), (b3) (∀α, β ∈ R)(∀x ∈ V )((α + β)x = αx + βx) (distributivity over scalar addition), (b4) (∀α ∈ R)(∀x, y ∈ V )(α(x + y) = αx + αy) (distributivity over vector addition),
(c) (∀x ∈ V )(1 · x = x) (normalization).
Exercise 1.1.2. Show (∀x ∈ V )(0x = 0). (The first 0 is a number, the second a vector.)
Exercise 1.1.3. Show (∀α ∈ R)(α0 = 0).
Exercise 1.1.4. Show (∀α ∈ R)(∀x ∈ V )(αx = 0 ⇔ (α = 0 or x = 0))
Example 1.1.5. Examples of vector spaces
- Rn- the set of n-tuples of real numbers
- geometric vectors in 2 or 3 dimensions
- k × ` matrices
- C[0, 1], the continuous functions from [0,1] to R
- the space of infinite sequences of real numbers
- R[x] = {polynomials with real coefficients}.
- R(x) = {rational functions with real coefficients} (see Definition 1.1.6 below).
- RΩ, the functions from Ω to R
- Ω = { 1 ,... , n} is example 1 above
- Ω = { 1 ,... , k} × { 1 ,... , `} is example 3
- Ω = [0, 1] contains example 4
- Ω = N is example 5
Definition 1.1.6. Rational functions are equivalence classes of formal fractions pq with p,
q ∈ R[x], q 6 = 0, under the equivalence relation p q 11 = p q^22 iff p 1 q 2 = p 2 q 1.
Definition 1.1.7. A linear combination of vectors v 1 ,... , vk ∈ V is a vector α 1 v 1 +· · ·+αkvk where α 1 ,... αk ∈ R.
Definition 1.1.8. The span of v 1 ,... , vk ∈ V is the set of all linear combinations of v 1 ,... , vk, i. e., Span(v 1 ,... , vk) = {α 1 v 1 + · · · + αkvk | α 1 ,... , αk ∈ R}.
Remark 1.1.9. We let Span(∅) = { 0 }, by virtue of the fact that an empty sum is always zero.
Definition 1.1.10. A linear combination of an infinite list of vectors is a linear combination of a finite sublist. The span of an infinite list S of vectors is, as defined before, as the set of all linear combinations of S.
Exercise 1.2.4. Prove: A one-element lists {v} of vectors is linearly independent ⇐⇒ v 6 = 0.
Exercise 1.2.5. A list containing 0 is never linearly independent.
Exercise 1.2.6. A list of vectors with repetitions is never linearly independent.
Exercise 1.2.7. Show that if T ⊆ S ⊆ V and S is linearly independent then T is linearly independent.
Exercise 1.2.8. Find a curve in Rn, i.e., a continuous function f : R → Rn, such that any n points on the curve are linearly independent, i.e., for any n distinct real numbers α 1 ,... , αn, the vectors f (α 1 ),... , f (αn) are linearly independent. (Give a simple explicit formula.)
Definition 1.2.9. We say that vectors u, v ∈ V are parallel if u, v 6 = 0 and (∃α ∈ R)(u = αv).
Exercise 1.2.10. Show that vectors u, v ∈ V are linearly dependent if and only if u = 0 or v = 0 or u, v are parallel.
Remark 1.2.11. We say that a property P is a finitary property if a set S has the property P if and only if all finite subsets of S have property P. Note that linear independence is, by Definition 1.2.2, a finitary property.
Exercise 1.2.12. ∗^ (Erd˝os – deBruijn) Show that 3-colorability of a graph is a finitary property. (The same holds for 4-colorability, etc.) (Hints for three different proofs. (1) Use Zorn’s Lemma. (2) Use the Compactness Theorem of first-order logic. (3) Use Tikhonoff’s Theorem (compactness of the product of compact topological spaces).
Exercise 1.2.13. Let α 1 ,... , αn be distinct real numbers. Prove: (^) x−^1 α 1 ,... , (^) x−^1 αn are linearly independent rational functions.
Exercise 1.2.14. Prove: for all α, β ∈ R, sin(x), sin(x + α), sin(x + β) are linearly dependent functions R → R.
Exercise 1.2.15. Prove: 1 , cos(x), cos(2x), cos(3x),... , sin(x), sin(2x),... are linearly inde- pendent functions R → R.
1.3 Rank, Dimension
Definition 1.3.1. A maximal linearly independent subset of a set S ⊆ V is a subset T ⊆ S such that
(a) T is linearly independent, and
(b) if T ( T ′^ ⊆ S then T ′^ is linearly dependent.
Definition 1.3.2. A maximum linearly independent subset of a set S ⊆ V is a subset T ⊆ S such that
(a) T is linearly independent, and
(b) if T ′^ ⊆ S is linearly independent then |T | ≥ |T ′|.
Exercise 1.3.3. (Independence of vertices in a graph.) Find a graph which has a maximal independent set of vertices which is not maximum.
We shall see that this cannot happen with linear independence: every maximal linearly independent set is maximum.
Exercise 1.3.4. Let S ⊆ V. Then there exists T ⊆ S such that T is a maximal independent subset of S.
Exercise 1.3.5. Let L ⊆ S ⊆ V. Assume L is linearly independent. Then there exists a maximal linearly independent subset T ⊆ S such that L ⊆ T. (Every linearly independent subset of S set can be extended to a maximal linearly independent subset of S.)
Remark 1.3.6. It is easy to prove Exx. 1.3.4, 1.3.5 by successively adding vectors until our set becomes maximal as long as all linearly independent subsets of S are finite. For the infinite case, we need “transfinite induction” or a result from set theory called Zorn’s Lemma (a version of the Axiom of Choice).
Definition 1.3.7. A vector v ∈ V depends on S ⊆ V if v ∈ Span(S), i. e. v is a linear combination of S.
Definition 1.3.8. A set of vectors T ⊆ V depends on S ⊆ V if T ⊆ Span(S).
Exercise 1.3.9. Show that dependence is transitive: if R ⊆ Span(T ) and T ⊆ Span(S) then R ⊆ Span(S). (“A linear combination of linear combinations is a linear combination.”)
Definition 1.3.10. A vector v depends on a list S of vectors if v ∈ Span(S). A list T of vectors depends on a list S of vectors if every member of T depends on S.
Exercise 1.3.11. Prove: dependence is transitive, i. e., if R, S, T are lists of vectors, R depends on S and S depends on T then R depends on T (“linear combinations of linear combinations are linear combinations).
Exercise 1.3.12. Suppose that
αivi is a nontrivial linear combination. Then (∃i) such that vi depends on the rest (i. e. on {vj | j 6 = i}). Indeed, this will be the case whenever αi 6 = 0.
Exercise 1.3.13. A list S of vectors is linearly dependent iff some member of the list depends on the rest.
Exercise 1.3.14. If v 1 ,... , vk are linearly independent and v 1 ,... , vk, vk+1 are linearly depen- dent then vk+1 depends on v 1 ,... , vk.
The next fundamental result asserts the impossibility of boosting linear independence.
1.4 Basis
Definition 1.4.1. A basis of V is a linearly independent set of generators of V , i.e., it is a linearly independent set that spans V.
Definition 1.4.2. A basis of S ⊆ V is a linearly independent subset of S on which S depends. In other words, a basis B of S is a linearly independent set satisfying B ⊆ S ⊆ Span(B).
Exercise 1.4.3. A list B of vectors is a basis of V iff every vector can be written uniquely as a linear combination of B.
Definition 1.4.4. Let B = (b 1 ,... , bn) be a basis of V. Write x ∈ V as x =
∑n i=1 βibi. The coefficients in this unique linear combination are called the coordinates of x with respect to B.
Definition 1.4.5. For a basis B of V , regarded as a list of vectors, we associate with each x ∈ V the column vector
[x]B :=
β 1 .. . βk
where the βi are the coordinates of x with respect to B.
Exercise 1.4.6. B is a basis of S if and only if B is a maximal independent subset of S.
In view of Ex. 1.3.5 we have the following corollary.
Corollary 1.4.7. Every vector space has a basis. In fact, every subset of a vector space has a basis. (Note: in the infinite dimensional case the proof requires Zorn’s Lemma.)
Exercise 1.4.8. Show that if the vectors of B ⊆ S are linearly independent, then B can be extended to a basis of S.
Exercise 1.4.9. Show that every set of generators of V contains a basis. In fact this is true for any subset S ⊆ V : if T ⊆ S such that T generates S, i.e., S ⊆ Span(T ), then there exists a basis B of S such that B ⊆ T.
Exercise 1.4.10. Prove: “All bases for S have equal size.” Show that the this statement is equivalent to the “First Miracle”, Theorem 1.3.15.
Exercise 1.4.11. Prove: if B is a basis of V then dim(V ) = |B|.
Exercise 1.4.12. A “Fibonacci-type sequence” is a sequence (a 0 , a 1 , a 2 ,... ) such that (∀n)(an+2 = an+1 + an).
(a) Prove that the Fibonacci-type sequences form a 2-dimensional vector space.
(b) Find a basis in this space consisting of two geometric progressions.
(c) Express the Fibonacci sequence (0, 1 , 1 , 2 , 3 , 5 , 8 , 13 ,... ) as a linear combination of the basis found in item (b).
1.5 Linear maps, isomorphism of vector spaces
Definition 1.5.1. Let V and W be vector spaces. We say that a map f : V → W is a homomorphism or a linear map if
(a) (∀x, y ∈ V )(f (x + y) = f (x) + f (y))
(b) (∀x ∈ V )(∀α ∈ R)(f (αx) = αf (x))
Exercise 1.5.2. Show that if f is a linear map then f (0) = 0.
Exercise 1.5.3. Show that f (
∑k i=1 αivi) =^
∑k i=1 αif^ (vi).
Definition 1.5.4. We say that f is an isomorphism if f is a bijective homomorphism.
Definition 1.5.5. Two spaces V and W are isomorphic if there exists an isomorphism between them.
Exercise 1.5.6. Show the relation of being isomorphic is an equivalence relation.
Exercise 1.5.7. Show that an isomorphism maps bases to bases. Therefore, isomorphic vector spaces have the same dimension.
Theorem 1.5.8. If dim(V ) = n then V ∼= Rn.
Proof: Choose a basis, B of V , now map each vector to its coordinate vector, i.e., v 7 → [v]B.
Exercise 1.5.9. Prove: two vector spaces are isomorphic if and only if they have the same dimension.
Exercise 1.5.10. (Degree of freedom in choosing a linear map) Let V, W be vector spaces. Let B = (b 1 ,... , bk) be a basis of V. Let w 1 ,... , wk be arbitrary vectors in W. Prove: there is a unique linear map f : V → W such that (∀i)(f (bi) = wi).
Exercise 1.5.11. Show that the linear maps V → W form a vector space. This space is called Hom(V, W ). Prove that its dimension is dim(V ) dim(W ).
Definition 1.5.12. The image of f is the set
Im(f ) = {f (x) : x ∈ V }
Definition 1.5.13. The kernel of f is the set
Ker(f ) = {x ∈ V : f (x) = 0}
Exercise 1.5.14. For a linear map f : V → W show that Im(f ) ≤ W and Ker(f ) ≤ V.
Theorem 1.5.15. For a linear map f : V → W we have
dim Ker(f ) + dim Im(f ) = dim V.
Lemma 1.5.16. If U ≤ V and A is a basis of U then A can be extended to a basis of V.
Exercise 1.5.17. Prove Theorem 1.5.15. Hint: apply Lemma 1.5.16 setting U = Ker(f ).
(c) Switch vi and vj.
Exercise 1.7.5. Show that the rank of a list of vectors doesn’t change under elementary operations.
Exercise 1.7.6. Let {v 1 ,... , vk} have rank r. Show that by a sequence of elementary op- erations we can get from {v 1 ,... , vk} to a set {w 1 ,... , wk} such that w 1 ,... , wr are linearly independent and wr+1 = · · · = wk = 0.
Consider a matrix. An elementary row-operation is an elementary operation applied to the rows of the matrix. Elementary column operations are defined analogously. Exercise 1.7. shows that elementary row-operations do not change the row-rank of A.
Exercise 1.7.7. Show that elementary row-operations do not change the column-rank of a matrix.
Exercise 1.7.8. Use Exercises 1.7.5 and 1.7.7 to prove the Second Miracle.
