Perceptron Algorithm - Introduction to Pattern Recognition - Lecture Slides, Slides of Advanced Algorithms

Download Perceptron Algorithm - Introduction to Pattern Recognition - Lecture Slides and more Slides Advanced Algorithms in PDF only on Docsity!

Recap

We have been considering learning of linear modelsfor classification and regression.

PR NPTEL course – p.1/

Recap

We have been considering learning of linear modelsfor classification and regression.

We have looked at different methods such asPerceptron algorithm, linear least squares method,logistic regression, Fisher linear discriminant etc.

PR NPTEL course – p.2/

Recap

We have been considering learning of linear modelsfor classification and regression.

We have looked at different methods such asPerceptron algorithm, linear least squares method,logistic regression, Fisher linear discriminant etc.

We presented all the algorithms for 2-class problemsin case of classification and for learning a real-valuedfunction in case of regression.

As we briefly saw in the previous lecture, we cangeneralize these easily.

PR NPTEL course – p.4/

Recap

As we saw, learning a vector-valued function usingleast squares approach is a simple extension.

PR NPTEL course – p.5/

Recap

As we saw, learning a vector-valued function usingleast squares approach is a simple extension.

Given training data,

X

i

, y

i

, i

, n

where

X

i

d

and

y

i

y

1 i

, y

m i

m

, we want to learn

W

j

,

b

j

,

j

, m

so that

y

j

W

T

j

X

b

j

, j

, m

We can easily do this by essentially solving

m

number

linear least squares regression problems.

PR NPTEL course – p.7/

We also considered the multiclass case of linear leastsquares approach.

PR NPTEL course – p.8/

We also considered the multiclass case of linear leastsquares approach.

We can, in principle, solve a multiclass problem bylearning many 2-class classifiers.

we saw that we can do this by ‘one Vs rest’ or ‘one Vsone’ approach.

PR NPTEL course – p.10/

We also considered the multiclass case of linear leastsquares approach.

We can, in principle, solve a multiclass problem bylearning many 2-class classifiers.

we saw that we can do this by ‘one Vs rest’ or ‘one Vsone’ approach.

While these are often used, they are not fullysatisfactory extensions of linear discriminant functionsto the multiclass case.

PR NPTEL course – p.11/

Similar problem exists when we try ‘

C

i

Vs

C

j

’

approach

???

R

2

R

1

R

3

C

2

C

3

C

1

C

1

C

3

C

2

PR NPTEL course – p.13/

As we saw, we can formulate linear discriminantbased classifier for the multi-class case as follows.

PR NPTEL course – p.14/

As we saw, we can formulate linear discriminantbased classifier for the multi-class case as follows.

We will have

K

functions,

g

s

,

s

, K

, given by

g

s

X

W

T

s

X

b

s

Now the classifier would assign class

C

j

to

X

if

g

j

X

g

s

X

s

(We have some rule for breaking ties).

PR NPTEL course – p.16/

As we saw, we can formulate linear discriminantbased classifier for the multi-class case as follows.

We will have

K

functions,

g

s

,

s

, K

, given by

g

s

X

W

T

s

X

b

s

Now the classifier would assign class

C

j

to

X

if

g

j

X

g

s

X

s

(We have some rule for breaking ties).

This is same as the way we generalized Bayesclassifier to multi-class case.

PR NPTEL course – p.17/

To learn a linear classifier for the

K

-class case, we

need to learn the

K

functions

g

s

.

The simplest way to do this is to make the class labelto be a vector of

K

components.

PR NPTEL course – p.19/

To learn a linear classifier for the

K

-class case, we

need to learn the

K

functions

g

s

.

The simplest way to do this is to make the class labelto be a vector of

K

components.

If

X

i

C

j

then

y

i

would be a

K

-vector with

j

th

component one and all others zero.

PR NPTEL course – p.20/