Prepare for your exams
Get points
Guidelines and tips
Sell on Docsity
Docsity AI

Prepare for your exams

Study with the several resources on Docsity

Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips

Sell on Docsity

Docsity AI

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search for your university

Find the specific documents for your university's exams

Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents

20 Points

For each uploaded document

Answer questions

5 Points

For each given answer (max 1 per day)

All the ways to get free points

Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

Optimal Feature Generation-Recognizing Patterns and Classifying Them-Lecture Slides, Slides of Pattern Classification and Recognition

Banasthali Vidyapith Pattern Classification and Recognition

This lecture is related to Pattern Classification and Recognition. It was delivered by Sahayu Agendra at Banasthali Vidyapith. It includes: Optimal, Feature, Generation, Fisher, Linear, Discrimination, Scatter, Matrices, Transformation, Criterion, Diagnolizes, Simultaneously

Typology: Slides

2011/2012

Uploaded on 07/17/2012

bandhula 🇮🇳

4.7

(10)

91 documents

1 / 15

This page cannot be seen from the preview

Don't miss anything!

Optimal Feature Generation

In general, feature generation is a problem-dependent

task. However, there are a few general directions

common in a number of applications. We focus on three

such alternatives.

Optimized features based on Scatter matrices (Fisher’s

linear discrimination).

• The goal: Given an original set of mmeasurements

,compute ,bythelinear transformation

so that the J3scattering matrix criterion involving Sw,S

is maximized. ATis an matrix.

x 

y

xAy T



xm

docsity.com

Discover Slides of Pattern Classification and Recognition Banasthali Vidyapith

Partial preview of the text

Download Optimal Feature Generation-Recognizing Patterns and Classifying Them-Lecture Slides and more Slides Pattern Classification and Recognition in PDF only on Docsity!

Optimal Feature Generation

In general, feature generation is a problem-dependenttask.

However,

there

are

a

few

general

directions

common in a number of applications. We focus on threesuch alternatives.

Optimized

features

based

Scatter

matrices

(Fisher’s

linear discrimination).

The goal: Given an original set

measurements

, compute

, by the linear transformation

so that the

J

scattering matrix criterion involving

S

, S

is maximized.

A

is an

matrix.

x



y

x A



xm 

The basic steps in the proof:

J

= trace{

S

A

S

A, S

= A

T^ S

A

J

A

)=trace{(

A

T^ S

A

T^ S

A

Compute

A

so that

J

A

)^

is maximum.

The solution:
- Let

B

the

matrix

that

diagonalizes

simultaneously matrices

S

, S

, i.e:

B

T^ S

B = I , B

T^ S

B = D

where

B

is a

matrix and

D

diagonal matrix.



ℓ

<M-

, choose the

ℓ

eigenvectors corresponding to

the

ℓ

largest eigenvectors.



this

case,

J 3,y

3,x

that

there

loss

information.

Geometric

interpretation.

The

vector

the

projection of

onto the subspace spanned by the

eigenvectors of

(^1) 

Principal Components Analysis(The Karhunen – Loève transform):

The goal: Given an original set of

measurements

compute for an orthogonal

A

,^

so that the elements of

are

optimally mutually uncorrelated.That is 

Sketch of the proof:

     y

x

A

y

^

j i j y i y E 



, 0 ) ( ) (

A R A A x x A E y y E R x



docsity.com

Define

The Karhunen – Loève transform minimizes thesquare error: 

The error is:

can

also

shown

that

this

the

minimum

mean

square

error

compared

any

other

representation of

by an

ℓ-dimensional vector.

  

(^10)

) (

ˆ^

 i

i a i y

^

^

  

  







) (

ˆ^

m i

i a i y E x x E



^

^



m i

x

E



In other words,

is the projection of

into

the

subspace

spanned

the

principal

eigenvectors. However, for Pattern Recognitionthis is not the always the best solution.

ˆ x

x

Subspace Classification. Following the idea of projecting ina

subspace,

the

subspace

classification

classifies

unknown

to the class whose subspace is closer to

The following steps are in order:

For each class, estimate the autocorrelation matrix

R

, i

and compute the

largest eigenvalues. Form

A

, by i

using respective eigenvectors as columns.

Classify

to the class

, i

for which the norm of the

subspace projection is maximumAccording to Pythagoras theorem, this corresponds tothe subspace to which

is closer.

x

j i x A x A

T j

T i^

Independent Component Analysis (ICA)In

contrast

to

PCA,

where

the

goal

was

to

produce

uncorrelated

features,

the

goal

in

ICA

is

to

produce

statistically

independent

features.

This

is

a

much

stronger requirement, involving higher to second orderstatistics. In this way, one may overcome the problemsof PCA, as exposed before.

The goal: Given

, compute

so

that

the

components

of

are

statistically

independent.

In

order

the

problem

to

have

a

solution, the following assumptions must be valid:

Assume

that

indeed

generated

linear

combination of independent components

x

   y x W



y Φ



Common’s

method:

Given

,^

and

under

the

previously

stated

assumptions,

the

following

steps

are adopted:

Step 1: Perform PCA on

Step 2: Compute a unitary matrix,

, so that the fourth

order cross-cummulants of the transform vectorare zero. This is equivalent to searching for an

that

makes the squares of the auto-cummulants maximum,where,

is the 4

order auto-cumulant.

x

x A



ˆ A

y A

ˆ ˆ 

ˆ A

^

(^2) 

ˆˆ

) (

ˆ) (

max

 



i y k

TA A

^

4 k

Step 3:

A hierarchy of components: which

ℓ^

to use? In PCA

one

chooses

the

principal

ones.

In

ICA

one

can

choose the ones with the least resemblance to theGaussian pdf.

^

T  A A



Optimal Feature Generation-Recognizing Patterns and Classifying Them-Lecture Slides, Slides of Pattern Classification and Recognition

Related documents

Partial preview of the text

Download Optimal Feature Generation-Recognizing Patterns and Classifying Them-Lecture Slides and more Slides Pattern Classification and Recognition in PDF only on Docsity!

Optimal Feature Generation

In general, feature generation is a problem-dependenttask.

However,

there

are

a

few

general

directions

common in a number of applications. We focus on threesuch alternatives.

J

S

, S

A

x

y

J

S

S

S

A

S

A, S

= A

T^ S

A

J

A

A

T^ S

A

A

T^ S

A

A

J

A

)^

B

S

, S

B

T^ S

B = I , B

T^ S

B = D

B

D

Principal Components Analysis(The Karhunen – Loève transform):

A

,^

x

A

y

y

^

ℓ-dimensional vector.

x

x

E

ˆ x

x

R

A

x

x

x

x

j i x A x A

Independent Component Analysis (ICA)In

contrast

to

PCA,

where

the

goal