Support Vector Machine - Pattern Recognition - Lecture Slides, Slides of Mechanical Engineering

Download Support Vector Machine - Pattern Recognition - Lecture Slides and more Slides Mechanical Engineering in PDF only on Docsity!

The SVM approach

• We have briefly discussed Support Vector Machine(SVM) idea. PR NPTEL course – p.1/

The SVM approach

• We have briefly discussed Support Vector Machine(SVM) idea. - The idea is to map the feature vectors nonlinearly intoanother space and learn a linear classifier there. PR NPTEL course – p.2/

• Recall the simple example we saw earlier. PR NPTEL course – p.4/

• Recall the simple example we saw earlier. - Let

X

= [

x 1 x 2

]

PR NPTEL course – p.5/

• Recall the simple example we saw earlier. - Let

X

= [

x 1 x 2

]

and let φ

2

5 given by Z

φ

X

) = [

x 1 x 2 x 2 1 x 2 2 x 1 x 2

]

• Now, g

X

a 0

a 1 x 1

a 2 x 2

a 3 x 2 1

a 4 x 2 2

a 5 x 1 x 2 is a quadratic discriminant function in

2 ; PR NPTEL course – p.7/

• Recall the simple example we saw earlier. - Let

X

= [

x 1 x 2

]

and let φ

2

5 given by Z

φ

X

) = [

x 1 x 2 x 2 1 x 2 2 x 1 x 2

]

• Now, g

X

a 0

a 1 x 1

a 2 x 2

a 3 x 2 1

a 4 x 2 2

a 5 x 1 x 2 is a quadratic discriminant function in

2 ; but g

Z

a 0

a 1 z 1

a 2 z 2

a 3 z 3

a 4 z 4

a 5 z 5 is a linear dscriminant function in the ‘ φ

X

’ space. PR NPTEL course – p.8/

• There are two major issues in naively using this idea. - If we want, e.g., p th degree polynomial discriminant function in the original feature space (

m ), then the transformed feature vector, Z, has dimension

O

m p

. PR NPTEL course – p.10/

• There are two major issues in naively using this idea. - If we want, e.g., p th degree polynomial discriminant function in the original feature space (

m ), then the transformed feature vector, Z, has dimension

O

m p

.

• Results in huge computational cost both for learningand and final operation of the classifier. PR NPTEL course – p.11/

• There are two major issues in naively using this idea. - If we want, e.g., p th degree polynomial discriminant function in the original feature space (

m ), then the transformed feature vector, Z, has dimension

O

m p

.

• Results in huge computational cost both for learningand and final operation of the classifier. - We need to learn

O

m p

parameters rather than O

m

parameters. Hence may need much larger number of examples for achieving propergeneralization.

• SVM offers an elegant solution to both. PR NPTEL course – p.13/

Support Vector Machines

• Learning of optimal hyperplane. PR NPTEL course – p.14/

Support Vector Machines

• Learning of optimal hyperplane. - Separating hyperplane that maximizes separationbetween Classes. - Effectively maps original feature vectors into a high dimensional space. Hence learns nonlineardiscriminant functions. PR NPTEL course – p.16/

Support Vector Machines

• Learning of optimal hyperplane. - Separating hyperplane that maximizes separationbetween Classes. - Effectively maps original feature vectors into a high dimensional space. Hence learns nonlineardiscriminant functions. - By using Kernel function we never need to explicitly calculate the mapping. PR NPTEL course – p.17/

Support Vector Machines

• Learning of optimal hyperplane. - Separating hyperplane that maximizes separationbetween Classes. - Effectively maps original feature vectors into a high dimensional space. Hence learns nonlineardiscriminant functions. - By using Kernel function we never need to explicitly calculate the mapping. - We need solve only a quadratic optimization problem. - Now we formulate the SVM method, first for linearlyseparable case. PR NPTEL course – p.19/

• Training set: {

X

i , y i

i

,... , n

,

X

i

m , y i

. PR NPTEL course – p.20/