Download CS 591Q/791V - Pattern Recognition Homework 3 - Prof. Arun Ross and more Assignments Computer Science in PDF only on Docsity!
Homework 3
CS 591Q/791V - Pattern Recognition
Instructor: Dr. Arun Ross
Due Date: April 3, 2008
Note: You are permitted to discuss the following questions with others in the class.
However, you must write up your own solutions to these questions. Any indication to the
contrary will be considered an act of academic dishonesty. Code developed as part of this
assignment should be placed in a zip file and sent to arun.ross at mail.wvu.edu with the
subject line “CS 591Q/791V : Homework 3”. Also, include a hard-copy of your code when
you submit the homework.
- [10 points] Consider two one-dimensional normal distributions p(x|ω 1 ) ∼ N (− 0. 5 , 1) and
p(x|ω 2 ) ∼ N (+0. 5 , 1) and P (ω 1 ) = P (ω 2 ) = 0.5. Calculate the Bhattacharyya bound for
the error of a Bayesian classifier. How would you go about computing the true error rate of
the Bayesian classifier?
- [20 points] Consider two Cauchy distributions in one dimension:
p(x|ωi) =
πb
x−ai b
) 2 ,^ i^ = 1,^2.
(a) Plot the two distributions in the same graph by assuming a 1 = 10, a 2 = 20, and b = 6.
(b) Assuming equal prior probabilities for both the categories and a 0-1 loss function for
misclassification, show that the minimum probability of error is given by
Perror =
π
tan
− 1
a 2 − a 1
2 b
(c) What is the maximum value of Perror and under what conditions can this occur? Ex-
plain.
- [10 points] Let the components of the vector x = (x 1 , x 2 ,... , xd)
t be ternary valued (i.e.,
xi ∈ {− 1 , 0 , 1 }). For the j
th class, ωj , let:
pij = P r[xi = 1|ωj ],
qij = P r[xi = 0|ωj ],
rij = P r[xi = − 1 |ωj ].
Show that a minimum probability of error decision rule can be derived that involves dis-
criminant functions gj (x) that are quadratic functions of the components xi.
- [10 points] Consider a set of n i.i.d. samples (one-dimensional training patterns), D =
{x 1 , x 2 ,... xn}, that are drawn from the following distribution:
p(x|λ) =
e
−λ λ
x
x!
(Poisson Distribution).
Derive the maximum likelihood estimate of λ, i.e., ̂λmle.
- [10 points] Consider a set of n i.i.d. samples (one-dimensional training patterns), D =
{x 1 , x 2 ,... xn}, that are drawn from the following distribution (Rayleigh distribution):
p(x|θ) =
2 θxe
−θx^2 , x ≥ 0 ,
0 , otherwise.
Derive the maximum likelihood estimate of θ, i.e., θ̂mle.
- Consider a two-category (ω 1 and ω 2 ) classification problem with equal priors. Each feature
is a two-dimensional vector x = (x 1 , x 2 )
t
. The class-conditional densities are:
p(x|ω 1 ) ∼ N (μ 1 = [0, 0]
t , Σ 1 = 2I),
p(x|ω 2 ) ∼ N (μ 2 = [1, 2]
t , Σ 2 = I).
Generate 50 bivariate random training samples from each of the two densities.
(a) [10 points] Find the values for the maximum likelihood estimates of μ 1 , μ 2 , Σ 1 , and
Σ 2 using these training samples (see page 89, equations (18) and (19)).
(b) [10 points] Compute the Bayes decision boundary using the estimated parameters and
plot it along with the training points.
(c) [10 points] Compute the Bayes decision boundary using the true parameters and plot
it on the same graph.
(d) [10 points] Repeat (a) - (c) after generating 100, 500 and 1000 random training samples
from each of the two densities. Comment on your result.
