Linear Algebra Review Lecture Notes, Lecture notes of Linear Algebra

Download Linear Algebra Review Lecture Notes and more Lecture notes Linear Algebra in PDF only on Docsity!

Lecture 5a

Linear Algebra Review

W. Gambill

Department of Computer Science University of Illinois at Urbana-Champaign

February 2, 2010

Vector Space Example

The set of n-tuples in Rn^ form a vector space.

v =

v 1 v 2 .. . vn

By convention we will write vectors in column form. The transpose operator T converts column vectors to row vectors and vice versa.     

v 1 v 2 .. . vn

T

[

v 1 v 2... vn

]

Vector Operations

Addition and Subtraction Multiplication by a scalar Transpose Linear Combinations of Vectors Inner Product Outer Product Vector Norms

Vector Addition and Subtraction

Addition and subtraction are element-by-element operations

c = a + b ⇐⇒ ci = ai + bi i = 1 ,... , n d = a − b ⇐⇒ di = ai − bi i = 1 ,... , n

a =

 (^) b =

a + b =

 (^) a − b =

Vector Transpose

The transpose of a row vector is a column vector:

u =

[

]

then uT^ =

Likewise if v is the column vector

v =

 (^) then vT^ =

[

]

Linear Combinations

Combine scalar multiplication with addition

α

u 1 u 2 .. . um

• β

v 1 v 2 .. . vm

αu 1 + βv 1 αu 2 + βv 2 .. . αum + βvm

w 1 w 2 .. . wm

r =

 (^) s =

t = 2 r + 3 s =

Linear Combinations

Graphical interpretation: Vector tails can be moved to convenient locations Magnitude and direction of vectors is preserved [1,0]

[0,1]

[2,4] [1,-1]

[4,2] (^) [-1,1]

[1,1]

0 1 2 3 4 5 6

0

1

2

3

4

Vector Inner Product

In physics, analytical geometry, and engineering, the dot product has a geometric interpretation

σ = x · y ⇐⇒ σ =

∑^ n

i= 1

xiyi

x · y = ‖x‖ 2 ‖y‖ 2 cos θ

Vector Inner Product

For two n-element column vectors, u and v, the inner product is

σ = uTv ⇐⇒ σ =

∑^ n

i= 1

uivi

The inner product is commutative so that (for two column vectors) uTv = vTu

Computing the Inner Product in Matlab

The * operator performs the inner product if two vectors are compatible. 1 >> u = (0:3) ’; % u and v are 2 >> v = (3: -1:0) ’; % column vectors 3 >> s = uv 4 ??? Error using ==> * 5 Inner matrix dimensions must agree. 6 7 >> s = u’v 8 s = 9 4 10 11 >> t = v’*u 12 t = 13 4 14 15 >> dot(u,v) 16 ans = 17 4

Computing the Outer Product in Matlab

The * operator performs the outer product if two vectors are compatible.

1 u = (0:4) ’; 2 v = (4: -1:0) ’; 3 A = u*v’ 4 A = 5 0 0 0 0 0 6 4 3 2 1 0 7 8 6 4 2 0 8 12 9 6 3 0 9 16 12 8 4 0

Vector Norms

Compare magnitude of scalars with the absolute value ∣ ∣α

∣β

Compare magnitude of vectors with norms

‖x‖ > ‖y‖

There are several ways to compute ||x||. In other words the size of two vectors can be compared with different norms.

The L 2 Norm

The notion of a geometric length for 2D or 3D vectors can be extended vectors with arbitrary numbers of elements. The result is called the Euclidian or L 2 norm:

‖x‖ 2 =

x^21 + x^22 +... + x^2 n

( (^) n ∑

i= 1

x^2 i

The L 2 norm can also be expressed in terms of the inner product

‖x‖ 2 =

x · x =

xTx

p-Norms

For any positive integer p

‖x‖p =

|x 1 |p^ + |x 2 |p^ +... + |xn|p

) 1 /p

The L 1 norm is sum of absolute values

‖x‖ 1 = |x 1 | + |x 2 | +... + |xn| =

∑^ n

i= 1

|xi|

The L∞ norm or max norm is

‖x‖∞ = max (|x 1 |, |x 2 |,... , |xn|) = max i (|xi|)

Although p can be any positive number, p = 1 , 2 , ∞ are most commonly used.