Greg Grudic Intro AI 1
Support Vector Machine (SVM)
Classification
Greg Grudic
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Support Vector Machine Classification - Slides | CSCI 3202, Study notes of Computer Science
Material Type: Notes; Professor: Grudic; Class: Introduction to Artificial Intelligence; Subject: Computer Science; University: University of Colorado - Boulder; Term: Unknown 1989;
Typology: Study notes
Pre 2010
1 / 31
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Support Vector Machine Classification - Slides | CSCI 3202 and more Study notes Computer Science in PDF only on Docsity!
Greg Grudic
Intro AI^
Support Vector Machine (SVM)
Classification^ Greg Grudic
Last Class
-^ Linear separating hyperplanes for binaryclassification •^ Rosenblatt’s Perceptron Algorithm^ –^ Based on Gradient Descent^ –^ Convergence theoretically guaranteed if data is linearlyseparable -^ Infinite number of solutions -^ For nonlinear data:^ –^ Mapping data into a nonlinear space where it is linearlyseparable (or almost)^ –^ However, convergence still not guaranteed… Greg Grudic
Intro AI^
Why Classification?
Greg Grudic
Introduction to AI world Sensing
Actions Computation^ State Decisions/Planning
Agent
Uncertainty
Signals Symbols^ (The Grounding^ Problem)
Not typically addressed in CS
Image 1: Poly Mahalanobis
9/24/^
Intro AI^ Image 1
The Problem Domain for Project Test 1:Identifying (and Navigating) Paths Non-path^
Path
Data^
Data
Data^
Construct a^ Classifier
Path labeled Image Classifier
Greg Grudic
Intro AI^
Support Vector Machine (SVM)
Classification
-^ Classification as a problem of findingoptimal (canonical) linear hyperplanes. •^ Optimal Linear Separating Hyperplanes:^ –^ In Input Space^ –^ In Kernel Space -^ Can be non-linear
Greg Grudic
Intro AI^
Linear Separating Hyper-Planes How many lines can separate these points?
NO!
Which line should we use?
Greg Grudic
Intro AI^
10
Linear Separating Hyper-Planes
x^1
x^2
(^0) b w x ⋅^ +
<
(^0) b w x ⋅^ +
(^0) b w x ⋅^ +
= 1 y^ = −
y^ = +
Greg Grudic
Intro AI^
Linear Separating Hyper-Planes • Given data: • Finding a separating hyperplane can be posed as aconstraint satisfaction problem (CSP): • Or, equivalently: • If data is linearly separable, there are an infinitenumber of hyperplanes that satisfy this CSP
(^ 1,...,^ ,^ find)
and
such that 1 if^
1 if^
i^
i i^
i
i^
N^
b b^
y b^
w y ∀ ∈ w x ⋅^ + w x
≥ +^
⋅^ +^
≤ −^
(^
)^ (^
)
,^ ,..., 1 1
, N^
N
y^
y
x^
x ( )
i^
i
y^
b^
i
w x ⋅^ +
Greg Grudic
Intro AI^
Calculating the Margin of a Classifier^ P2 P0 P
- P0: Any separating hyperplane • P1: Parallel to P0, passing throughclosest point in one class • P2: Parallel to P0, passing throughpoint closest to the opposite class^ Margin (M)
: distance measured along a line perpendicular to P1 and P
x^1
x^2
Model parameters
must be chosen such that,
for^ on P1 and for
on P2:
SVM Constraints on the Model Parameters Greg Grudic^
Intro AI^ , b w ( ) 1
P1:^
1 b w x ⋅^
+^ = − 2 P2:^
1 b w x ⋅^
+^ = +
For any P0, these constraints are always^ attainable.
Given the above, then the linear separating boundary lies half way between P1 and P2 and is given by:
(^0) b w x ⋅^ +
=^ (
)
ˆ^ sgn y^
b w x =^
⋅^ +
x^^1 Resulting Classifier:
x^2
Calculating the Margin (1) Greg Grudic^
Intro AI^ (^
) (^
) (^
) (^
)
2 2
1 1
1 1
2 1
b
M^ d
P
b
M^ d
P
b
d P
b
d P
w x
x^
w w x
x^
w
⋅^ w xx w w xx w
+^ +
=^
= ⋅^
+^ −
=^
= ⋅^
+^ +
=^
= ⋅^
+^ −
=^
Calculating the Margin (2) Greg Grudic^
Intro AI^
Take absolute value to get the unsigned margin:
M^^ = w
Signed Distance
(^ )^
(^ ) (^
)
2
1
2
1 2
1
2 1
1
(^1 )
1
1,^
2, Therefore:
1
1
2
Therefore:
2 1
2 (
1)^2
(0)^2 b^
b
M^ d P
d P b
b b
b
M
w x^
w x
x^
x^
w^
w
w x^
w x^
w x^ w x
w^
w w x^
w x w^
w^
w^
w
⋅^ +^ +
⋅^
+^ −
=^
=^
=^
=
⋅^ +^ +
⋅^
+^ −=
⇒^ ⋅^
=^ ⋅^ −
⋅^ −^ +
+^ −
+^ ⋅^
+^ +^
−^ +^
−
=^
=^
=^
=
Greg Grudic
Intro AI^
Different P0’s have Different Margins
P
P
P
- P0: Any separating hyperplane • P1: Parallel to P0, passing throughclosest point in one class • P2: Parallel to P0, passing throughpoint closest to the opposite class^ Margin (M)
: distance measured along a line perpendicular to P1 and P
Greg Grudic
Intro AI^
Different P0’s have Different Margins
P2 P0 P
- P0: Any separating hyperplane • P1: Parallel to P0, passing throughclosest point in one class • P2: Parallel to P0, passing throughpoint closest to the opposite class^ Margin (M)
: distance measured along a line perpendicular to P1 and P