Linear Independence, Basis, and Dimension in Linear Algebra, Lecture notes of Machine Learning

Download Linear Independence, Basis, and Dimension in Linear Algebra and more Lecture notes Machine Learning in PDF only on Docsity!

Linear Independence, Basis, and Dimension

Vectors x 1 , x 2 , · · · , x (^) n are linearly independent if no combination gives c 1 x 1 + c 2 x 2 + · · · + c (^) n xn = 0 except for c 1 = c 2 = · · · = c (^) n = 0. For real vectors, linear independence is equivalent to non-collinearity (algebraic vs. geometrical meaning) 0 vector cannot be independent with any non-zero vector v , because we always have c 0 + 0v = 0 for c 6 = 0. Column vectors v 1 , v 2 , · · · , v (^) n of matrix A (^) m⇥n are Linearly independent if N(A) only includes 0 ) r = n  m, no free variables Linearly dependent if 9 vector x 6 = 0 s.t. Ax = 0 ) r < n, there are n r free variables Vectors v 1 , v 2 , · · · , v (^) n span a space if that space consists of ALL linear combinations of those vectors. e.g., two vectors cannot span R 3 e.g., four non-coplanar vectors span R 3 , but they must be linearly dependent!

Linear Independence, Basis, and Dimension

Check if a set of vectors forms a basis

List them as the column vectors of A and perform Gaussian elimination, then count the number of pivots in the Echelon form.

In general, n vectors form a valid basis of Rn^ if the n ⇥ n matrix they are involved in is invertible (non-singular). There can be infinite number of di↵erent bases Common: number of vectors (rank) ) Dimension of the space

Example

A =

(^5) r = dim C (A) = 2

The first 2 columns are pivot columns, so {v 1 , v 2 } is a basis Since v 3 = v 1 + v 2 , {v 1 , v 3 } is also a basis Since v 4 = 2v 1 , {v 1 , v 4 } is NOT a basis

Four fundamental subspaces

We have already defined Column space C (A) ⇢ R m^ : dim C (A) = r Null space N(A) ⇢ R n^ : Kernel of A, dim N(A) = n r ”Nullity” Similarly, by switching the roles of columns and rows, Row space C (A T^ ) ⇢ R n Left null space N(A T^ ) = {x|A T^ x = 0 or x T^ A = 0} ⇢ R m

Properties of fundamental subspaces

Subspace C (A) C (AT^ ) N(A) N(AT^ ) Dimension r r n r m r Basis Pivot col. Pivot col. Setting each Do the same of A of AT^ free variables 1 for AT^ as and solve Ux = 0 solving N(A)