Principal Component Analysis (PCA) for Dimensionality Reduction, Slides of Engineering Dynamics

Download Principal Component Analysis (PCA) for Dimensionality Reduction and more Slides Engineering Dynamics in PDF only on Docsity!

• In this course so far, we have discussed variousstrategies for learning classifiers.

• In this course so far, we have discussed variousstrategies for learning classifiers. - We have considered Bayes classifier and differentmethods of estimating class conditional densities toimplement it.

• In this course so far, we have discussed variousstrategies for learning classifiers. - We have considered Bayes classifier and differentmethods of estimating class conditional densities toimplement it. - Then we studied methods (Perceptron, Least squaresetc.) of learning linear classifiers and regressionmodels. - After that we studdied some general techniques(neural networks, SVMs) for learning nonlinearclassifiers.

• However, we have not discussed anything regardingwhat features one should use.

• However, we have not discussed anything regardingwhat features one should use. - Patterns are represented as feature vectors. - As mentioned at the beginning, the features used areproblem-dependent.

• However, we have not discussed anything regardingwhat features one should use. - Patterns are represented as feature vectors. - As mentioned at the beginning, the features used areproblem-dependent. - One heuristically decides to measure quantities thatseem reasonable and which are easy to measure.

Feature Selection

• One strategy is to start with many differentmeasurements/features and then find a good subsetof features.

Feature Selection

• One strategy is to start with many differentmeasurements/features and then find a good subsetof features. - This is known as feature selection problem.

Feature Selection

• One strategy is to start with many differentmeasurements/features and then find a good subsetof features. - This is known as feature selection problem. - Which is a ‘small’ subset of features that has ‘best’correlation with class labels? - There are some general techniques to address suchissues.

Feature Selection

• One strategy is to start with many differentmeasurements/features and then find a good subsetof features. - This is known as feature selection problem. - Which is a ‘small’ subset of features that has ‘best’correlation with class labels? - There are some general techniques to address suchissues. - Due to lack of time we would not consider it in thiscourse.

• There are also techniques to transform the originalfeature vector into a new feature vector. - We may want to do this to improve the features (e.g.,make them uncorrelated).

• There are also techniques to transform the originalfeature vector into a new feature vector. - We may want to do this to improve the features (e.g.,make them uncorrelated). - Or we may want to do this to reduce thedimensionality of the feature vector without loosingtoo much information.

• There are also techniques to transform the originalfeature vector into a new feature vector. - We may want to do this to improve the features (e.g.,make them uncorrelated). - Or we may want to do this to reduce thedimensionality of the feature vector without loosingtoo much information. - One such technique is the Principal ComponentAnalysis (PCA). - This is a general-purpose method useful in manyproblems of data analysis and machine learning.

Principal Component Analysis

• We can think of PCA as a useful linear transformationof the feature vector.