Project Part 1 [10 points] Feature Extraction, Density Estimation and Bayesian
Classification (Due October 6, 11:59pm)
This part of the project uses a subset of images (with modifications) from the MNIST
dataset. The original MNIST dataset (http://yann.lecun.com/exdb/mnist/) contains
70,000 images of handwritten digits, divided into 60,000 training images and 10,000
testing images. We use only images for digit “3” and digit “7” in this project. The data is
stored in “.mat” files. You may use the following piece of code to read the dataset in
Python (or you may use the load filename command in Matlab, since these are .mat files):
import scipy.io
data = scipy.io.loadmat(‘matlabfile.mat’)
Following are the statistics for the data you are going to use:
Number of samples in the training set: "3": 5713; "7": 5835
Number of samples in the testing set : "3": 1428; "7": 1458
You will practice doing the following three tasks in this project:
Task 1. Feature extraction and normalization
In the .mat file, each image is stored as a 28x28 array of pixels. The pixel values range
from 0 to 255. For each image i, compute two features:
1. Skewness of image ki – This is measure of symmetry of the pixel values for i. For
more information, visit
https://www.itl.nist.gov/div898/handbook/eda/section3/eda35b.htm
2. Ratio of brighter to darker pixels ri using a threshold T.
We further normalize the features in the following way, before starting any subsequent
tasks. Using the feature representations of all the training images from both classes (each
image i is now viewed as a 2-d vector, Xi = [ki, ri]t, as explained), compute the mean Mj
and standard deviation Sj, j=1,2, for the first and the second feature, respectively. Mj and
Sj will be used to normalize all the feature vectors (both training and testing): for each
feature vector Xi representing image i, a normalized feature vector Yi will be computed as
Yi = [ y1i, y2i]t = [ (ki – M1)/S1, (ri – M2)/S2]t
This Yi is the final feature representation for image i and will be used for subsequent
steps.
Task 2. Density estimation
