Dimensionality Reduction, Data Compression-Machine Learning and Artificial Intelligence-Lecture Slides, Slides of Machine Learning

Download Dimensionality Reduction, Data Compression-Machine Learning and Artificial Intelligence-Lecture Slides and more Slides Machine Learning in PDF only on Docsity!

Dimensionality

Reduc1on

Mo1va1on I:

Data Compression

Machine Learning

Data Compression (inches) (cm) Reduce data from 2D to 1D

Data Compression

Reduce data from 3D to 2D

Dimensionality

Reduc1on

Mo1va1on II:

Data Visualiza1on

Machine Learning

Data Visualiza2on Country Canada 1.6 1. China 1.7 0. India 1.6 0. Russia 1.4 0. Singapore 0.5 1. USA 2 1. … … …

Data Visualiza2on

Principal Component Analysis (PCA) problem formula2on

Principal Component Analysis (PCA) problem formula2on Reduce from 2-­‐dimension to 1-­‐dimension: Find a direc1on (a vector ) onto which to project the data so as to minimize the projec1on error. Reduce from n-­‐dimension to k-­‐dimension: Find vectors onto which to project the data, so as to minimize the projec1on error.

PCA is not linear regression

Dimensionality

Reduc1on

Principal Component

Analysis algorithm

Machine Learning

Principal Component Analysis (PCA) algorithm

Reduce data from 2D to 1D Reduce data from 3D to 2D

Principal Component Analysis (PCA) algorithm

Reduce data from -­‐dimensions to -­‐dimensions

Compute “covariance matrix”:

Compute “eigenvectors” of matrix :

[U,S,V] = svd(Sigma);

Principal Component Analysis (PCA) algorithm summary Ader mean normaliza1on (ensure every feature has zero mean) and op1onally feature scaling:

Sigma =

[U,S,V] = svd(Sigma);

Ureduce = U(:,1:k);

z = Ureduce’*x;

Dimensionality

Reduc1on

Reconstruc1on from

compressed

representa1on

Machine Learning