Linear Discriminant Functions - Pattern Recognition - Lecture Slides, Slides of Engineering Dynamics

Download Linear Discriminant Functions - Pattern Recognition - Lecture Slides and more Slides Engineering Dynamics in PDF only on Docsity!

Recap^ •^ We have been considering linear discriminantfunctions.

Recap^ •^ We have been considering linear discriminantfunctions.^ •^ Such a linear classifier is given by^ h(X)^

′ d ∑= 1 if^ wφi i= (X) +^ w>^0 i^0 =^0 Otherwisewhere φare fixed functions.i

Perceptron^ •^ Perceptron is the earliest such classifier.

Perceptron^ •^ Perceptron is the earliest such classifier.^ •^ Assuming augumented feature vector,^ h(X) =^ sgn(W

T^ X).

Perceptron^ •^ Perceptron is the earliest such classifier.^ •^ Assuming augumented feature vector,^ h(X) =^ sgn(W

T^ X).

-^ ‘find weighted sum and threshold’

Perceptron Learning Algorithm^ •^ A simple iterative algorithm.

Perceptron Learning Algorithm^ •^ A simple iterative algorithm.^ •^ Each iteration, we locally try to correct errors.Let^ ∆W^ (k) =^ W

(k^ + 1)^ −^ W^ (k

). Then ∆W^ (k)^ = 0

T^ if W (k)X (k)^ >^ 0 &^ y(k) = 1

,^ or T^ W (k)X(k)^ <^ 0 & y(k) = 0 =^ X(k)^ if^ W

T^ (k)X(k)^ ≤^ 0 & y(k) = 1 =^ −^ X(k)^ if^ W

T^ (k)X(k)^ ≥^ 0 & y(k) = 0^ PR NPTEL course – p.10/

Perceptron: Geometric view The algorithm has a simple geometric view. Consider thefollowing data set.

-^ Now the correction made to

W^ (k)^ can be seen as^ PR NPTEL course – p.13/

-^ We showed that: if the training set is linearlyseparable, the the algorithm would find a separatinghyperplane in finitely many iterations.

Perceptron^ •^ A simple ‘device’: Weighted sum and threshold.

Perceptron^ •^ A simple ‘device’: Weighted sum and threshold.^ •^ A simple learning machine. (A neuron model).

-^ Perceptron is an interesting algorithm to learn linearclassifiers.

-^ Perceptron is an interesting algorithm to learn linearclassifiers. •^ Works only when data is linearly separable.