Linear Algebra for Computer
Vision - part 2
CMSC 828 D
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Linear Algebra for Computer Vision: Part 2 - Eigenvalues, Orthogonal Basis, SVD - Prof. Ra, Study notes of Computer Science
A part of the linear algebra for computer vision course (cmsc 828 d) and covers topics such as eigenvalues, eigenvectors, orthogonal basis, gram schmidt orthogonalization, fredholm alternative theorem, least squares formulation, singular value decomposition, and applications. It discusses the concepts of linear spaces, dot product, linear dependence, basis, orthogonality, projection theorem, and gram schmidt orthogonalization. It also explains the use of summation convention and the concept of operators and matrices. Examples and formulas for various vector and matrix operations.
Typology: Study notes
Pre 2010
1 / 23
This page cannot be seen from the preview
Don't miss anything!
Partial preview of the text
Download Linear Algebra for Computer Vision: Part 2 - Eigenvalues, Orthogonal Basis, SVD - Prof. Ra and more Study notes Computer Science in PDF only on Docsity!
Linear Algebra for Computer
Vision - part 2
CMSC 828 D
Outline
• Background and potpourri• Summation Convention• Eigenvalues and Eigenvectors• Rank and Degeneracy• Gram Schmidt Orthogonalization• Fredholm Alternative Theorem• Least Squares Formulation• Singular Value Decomposition• Applications
•^
Linear Manifolds (M): linear spaces that are subsets of the spacethat are closed under vector addition and scalar multiplication–
If vectors
u
and
v
belong to the manifold then
so do
α^1
u
α^2
v
-^
Manifold must contain zero vector
-^
Essentially a full linear space ofsmaller dimension.
•^
Span of a set of vectors: set of allvectors that can be created byscalar multiplication and addition.
-^
Vectors in the space that are in the restof the space are orthogonal to vectorsin M. (M
⊥^
•^
Projection Theorem: any vector in the space X can be writtenonly one way in terms of a vector in M and a vector in M
O
u
2u
-u
v
Manifold spannedby
u
Gram Schmidt Orthogonalization•
Given a set of basis vectors (
b
b
b
) constructn
an orthonormal basis (
e^1
, e
e
) from it.n
–^
Set
e
b
b
–^
g^2
b
b
e
e
e^2
g
g^2
- For k=3, …,n
g k
b
−Σk
<j
b
,k e
j
e
,^ j
e k
g
/||k
g k
Summation Convention
-^
Boldface, transpose symbol and summation signs aretiresome.– Especially if you have to do things such as differentiation•^
Vectors can be written in terms of unit basis vectors
a
=a
i 1
+a
j 2
+a
k 3
= a
e 1
+a 1
e 2
+a 2
e 3
3
-^
However, even this is clumsy. E.g., in 10 dimensions
a
= a
e 1
+a 1
e 2
+…+a 2
10
e^10
i=
10
a
e i
i
-^
Notice that index
i
occurs twice in the expression.
- Einstein noticed this always occurred, so whenever index was
repeated twice he avoided writing
i
- instead of writing
ai bi^
i,^
write a
bi i^
with the
impliedi
Permutation Symbol
-^
Permutation symbol
ijk
-^
If i, j and k are in cyclic order
ε
ijk^
=
-^
Cyclic => (1,2,3) or (2,3,1) or (3,1,2)
-^
If in anticyclic order
ε
ijk^
=-
-^
Anticyclic => (3,2,1) or (2,1,3) or (1,3,2)
-^
Else,
ε
ijk^
=
-^
(1,1,2), (2,3,3), …
-^
c^
a
×
b
ci
ijk
aj
b
k
-^
identity
εijk
ε irs
jr^
ks
-^
js^
kr
- Very useful in proving vector identities -^
Indicial notation is also essential for working with tensors– Tensors are essentially linear operators (matrices or their
generalizations to higher dimensions)
Operators / Matrices
-^
Linear Operator
A(
α
1
u^
α
v) 2
α
1
Au
α
2
Av
-^
maps one vector to another
Ax
b
–^
m
× n
dimensional matrix
A
multiplying a
n
dimensional vector
x^
to produce a
m
dimensional vector
b
in the dual space
-^
Square matrix of dimension
n
by
n
takes vector to
another vector in the same space.
-^
Matrix entries are representations of the matrix usingbasis vectors
A
ij^
Ab
, b j
>i
-^
Eigenvectors are characteristic directions of the matrix.
-^
Matrix decomposition is a factorization of a matrix intomatrices with specific properties.
Rank and Null Space
•^
Range of a
m
× n
dimensional matrix
A
Range (
A
{y
∈
m : y
Ax
for some
x
∈
n }
•^
Null space of
A
is the set of vectors which it takes to zero.Null(
A
{x
∈
n^ : Ax
^0
•^
Rank of a matrix is the dimension of its range.Rank (
A
) = Rank (
A
t^ )
-^
Maximal number of independent rows
or
columns
•^
Dimension of
Null(
A
)+Rank(
A
n
Norm of a matrix
•^
|| A
||^
≥^0
|| Ax
|| A
x ||
•^
|| A
|| F
=[
a^ ij
a
] ij 1/
Froebenius norm. If
A
is diagonal
|| A
|| F
[ a
11
a
22
a
nn
2 ]
1/
•^
|| A
||^2
max
x^
|| Ax
||^2
x ||
Can show 2 norm = square root of largest eigenvalue of
A
t A
Rotation in 2D and 3D
• Rotation through an angle
• Rotation + translation• Rotation in 3D
φ^
about
z
axis,
θ^
about new
x
axis,
ψ
about new y axis.
θ y
x ’x
’y
' '
cos
sin
sin
cos
x
x
y
y
θ^
θ
θ^
θ
^
^
^
= ^
^
^
−
^
'^
'
1
1
'^
'
2
2
cos
sin
cos
sin
sin
cos
sin
cos
t^
x^
p
x
x^
x
t^
y^
p
y
y^
y
θ^
θ
θ^
θ
θ^
θ
θ^
θ
^
^
^
^
^
^
^
^
^
=^
+^
=
^
^
^
^
^
^
^
^
^
^
−^
−
^
^
^
^
^
^
^
^
^
'^
cos
cos
cos
sin
sin
cos
sin
cos
cos
sin
sin
sin
'^
sin
cos
cos
sin
cos
sin
sin
cos
cos
cos
cos
sin
'^
sin
sin
sin
cos
cos
x^
x
y^
y
z^
z
ψ
φ^
θ^
φ^
ψ
ψ
φ^
θ^
φ^
ψ
ψ
θ
ψ
φ^
θ^
φ^
ψ
ψ
φ^
θ^
φ^
ψ
ψ
θ
θ^
φ
θ^
φ
θ
−^
^
^
^
^
^
^
= −
−^
−^
^
^
^
^
^
^
−
^
^
^
Rotation matrix
• Rotates a vector represented in one
orthogonal coordinate system into a vectorin another coordinate system.– Since length of vector should not change||
Qx
x
|| for all
x
- Since Q will not change a vector along
coordinate directions
t^ =
I
- Columns of Q are its eigenvectors.– Eigenvalues are all 1.
Eigenvalue problem
Remarks: Eigenvalues and Eigenvectors
-^
Eigenvalues and Eigenvectors of a real symmetric matrix are real.
-^
In general since eigenvalues are determined by solving apolynomial equation, they can be complex.
-^
Further roots can be repeated
multiple eigenvectors correspond
to a single eigenvalue.
-^
Transforming matrix into eigenbasis yields a diagonal matrix.
Q
t AQ
is a matrix of eigenvalues
- Knowing the eigenvectors we can solve an equation
Ax
= b.
Rewrite
it as
Q
t AQQ
t x
= Q
t b
ΛΛΛΛ
y =
f
- Where
y
= Q
t x
and
f =
Q
t b
- Can get
x
from
y
x =
(Q
t^ -1)^
y^
=^
Qy
•^
Determinant is unchanged by an orthogonal transformation.
-^
Determinant: Det(
A
λ^1
λ^2
λ n
-^
If 2
nd
row of A is a sum of the 1
st^
and 3
rd^
rows, then
b
= 2 b^1
- b
3
-^
If there are
k
independent solutions to equation (1) then
A
has a
k
dimensional
nullspace
.
-^
A
*****^ also has a
k
dimensional nullspace (but with different solutions).
- Let these solutions be
n
(^1) ***** ,^ n
*^2
, …,
n
- k
-^
For
Ax
= b
can have solutions iff
< b,n
=0* j
j^ =
1,…,k
-^
b^
must be orthogonal to the nullspace of
A
*^.
-^
Any solution with y
=-y 2
1
and y
=y 3
1
satisfies the adjoint equation
or the nullspace of
A
*^ is
α
[1,-1,1]
t
-^
Here <
b,n
*^1
= -1(
≠0). So equation has no solution.
-^
However if
b
=[1, 2, 1]
t^ we would have a solution
-^
General solution is
x
=
x
~^ +c
n k *k
where
x
~^ is a particular solution.
1
1
2
2
3
3
1
1
1
1
1
2
1
0
2
1
1
3
or
1
1
2
0
1
2
0
1
1
1
0
0
x^
y
x^
y
x^
y
^
^
^
^
^
^
^
^
−^
=^
=^
−^
−^
=
^
^
^
^
− ^
^
^
^
^
^
^
^
Ax
b
Least Squares
-^
Number of equations and unknowns may not match
-^
Look for solution by maximizing ||
Ax - b
-^
(A
xij^
-bj
).(Ai^
xik
-bk
)i^
with respect to
x
l
-^
Recall
-^
Same as the solution of
A
t Ax
A
t b
-^
Shows the power of the index notation– See again the appearance of
A
t A
(^
)
(^
-^
) (
-^
) =
(^
) (
-^
)^
(^
-^
) (
)^
0
(^
-^
)^
(^
-^
)^
2
0
ij^
j^
i^
ik^
k^
i
l ij^
jl^
ik^
k^
i^
ij^
j^
i^
ik^
kl
il^
ik^
k^
i^
ij^
j^
i^
il^
il^
ik^
k^
il^
i
il^
ik^
k^
il^
i
A x
b^
A x
b
x A^
A x
b^
A x
b^
A
A^
A x
b^
A x
b^
A
A A x
A b
A A x
A b
δ
δ
∂ ∂
+^
=
+^
=^
−^
=
=
%
%^
%
%^
%
i
il
x xl
δ
∂^
= ∂