CS 4803A/8803A: Pattern Recognition
Problem Set 3
Date: Feb 20, 2003
Due: Feb 28th, 2003 - which is Friday before Spring Break.
But will be accepted until Monday 10th.
1. It is suspected that a certain four-sided die is unfair. By rolling the die many times, we wish
to estimate the probability of each of the four outcomes. Let {Pi}, i = 1, . . . , 4denote the
unknown probabilities.
Suppose the die is rolled Ntimes. If the probabilities to be estimated were known, then the
probability of observing exactly y11’s, y22’s, . . ., and y44’s in the Nrolls would be:
P(y1, . . . , y4;P1, . . . , P4) = N!
y1!y2!· · · y4!Py1
1Py2
2· · · Py4
4.
This distribution is called the multinomial distribution.
Suppose exactly yioccurrences of iare observed in the Nrolls, where i= 1,...,4. Find the
maximum likelihood estimates {ˆ
Pi}for the probabilities {Pi}, i = 1, . . . , 4in terms of the
yi’s.
2. DHS Problem 3.2
3. DHS Problem 3.15
4. DHS Problem 3.11 (Symbolic math - but cool to see that the obvious Gaussian model is in fact
the best Gaussian model for minimizing KL- divergence.)
5. This one can be tricky. We’ll do it in class but first do it yourself ...
We return to the fishing scenario in Problem Set 2, Problem 7. By now you’ve constructed
the optimal Bayesian classifier to decide if a fish is carp or bass, based on the uncertain class
information you received from the two experts. You draw two fish from the pond which weigh
1.5 pounds and 5 pounds, respectively. If you applied your classifier independently to each
fish, they would both be labeled bass. However, this may not seem right, and indeed it isn’t
(remember: only one expert is right about the pond). In this problem we will figure out what
the optimal classification of the two fish should be.
(a) The weight of a fish in the pond can be considered a random variable. If I draw two fish,
the weights correspond to two random variables. Are these two variables independent,
given your limited knowledge of the pond? Motivate your answer mathematically. (Re-
member: independence is not a property of the fish but of your knowledge about them.)
1
