Answer Key for Practice Exam 1
CS 591Q/791V - Pattern Recognition
Posted on: March 21, 2009
1. (a) Bootstrapping: A technique to estimate the variance in the error rate of a pattern classifier. In this
method, the test dataset consisting of, say, Ndata points, D={x1, x2, . . . xn}, is sampled multiple times.
Each sample set, DS, is obtained by drawing Npoints at random from D,with replacement, so that some
points in Dmay be replicated in DS, whereas other points in Dmay be absent from DS. This process is
repeated Ltimes to generate Lsample sets each of size N. The performance of the classifier is then evaluated
on each of these Lsample sets thereby allowing one to estimate the variance in error between the different
bootstrap data sets. [See pg. 23].
(b) Parzen Window: The kernel function used to estimate the value of the density function at a particular
point in the feature space by using the evidence of all the training pointpoints is called a Parzen window. If
φis a kernel function such that φ(u)≥0 and Rφ(u)du= 1, then the density at xcan be estimated as,
ˆp(x) = 1
N
N
X
i=1
1
hDφx−xi
h,
where, {x1,x2,...xN}are the training points, Dis the dimensionality of each xiand his the width of the
Parzen window. [See pg. 123].
(c) Fisher’s linear discriminant: Fisher’s linear discriminant is a classification method that projects high-
dimensional data (x) onto a line and performs classification in this one-dimensional space (y). The projection
maximizes the distance between the means of the classes while minimizing the variance within each class.
The projection vector, w, is obtained by maximizing the Fisher’s criterion. [See pp. 186-189].
2. The estimated density at xcan be computed as,
ˆp(x) = 1
N
N
X
i=1
1
hφx−xi
h,(1)
where, the xis are the points sampled from the unknown density p(x), N= 13, h= 1, and φis the kernel
function.
The uniform kernel function takes the following form [pg. 123]:
φ(u) = (1,|u| ≤ 1/2,
0,otherwise.(2)
Now, φx−xi
h=φ(x−xi), i= 1,2,...13, since h= 1. Thus,
φ(x−xi) = 1,if |x−xi| ≤ 1/2 based on equation (2).(3)
Therefore, the sum
13
X
i=1
φ(x−xi),
1
