ENEE 633 / CMSC 828C : Statistical Pattern Recognition
Assignment 1
1. Consider a classification problem in which the loss incurred when an input vector from
class Ckis classified as belonging to class Cjis given by the loss matrix Lkj, and for which the
loss incurred in selecting the reject option is λ. Find the decision criterion that will give the
minimum expected loss. Verify that this reduces to the reject criterion discussed in Section
1.5.3 when the loss matrix is given by Lkj =1−Ikj. What is the relationship between λand the
rejection threshold θ? (Q 1.24 from Bishop)
2. Consider the minimax criterion for a two category classification problem.
(a) Fill in the steps of the derivation of Eq. 23
(b) Explain why the overall Bayes risk must be concave down as a function of the prior P(w1),
as shown in Fig. 2.4.
(c) Assume we have one-dimensional Gaussian distributions p(x|wi)∼N(µi,σ2
i),i=1,2, but
completely unknown prior probabilities. Use the minimax criterion to find the optimal decision
point x∗in terms of µiand σiunder a zero-one risk.
(d) For the decision point x∗you found in (c), what is the overall minimax risk ? Express this
risk in terms of an error function erf(.).
(e) Assume p(x|w1)∼N(0,1)and p(x|w2)∼N(1
2,1
2), under a zero-one loss. Find x∗and the
overall minimax loss.
(d) Assume p(x|w1)∼N(5,1)and p(x|w2)∼N(6,1). Without performing any explicit
calculations, determine x∗for the minimax criterion. Explain your reasoning. (Q 2.4 from
Duda, Hart and Stock)
3. Let the conditional densities for a two category one-dimensional problem be given by
the Cauchy distribution described as
p(x|wi) = 1
πb
˙
1
1+x−ai
b2,i=1,2,a2>a1(1)
(a) By explicit integration, check that the distributions are indeed normalized.
(b) Assuming P(w1) = P(w2), show that P(w1|x) = P(w2|x)if x= (a1+a2)/2, that is,
the minimum error decision boundary is a point midway between the peaks of the two
distributions, regardless of b.
(c) Plot P(w1|x)for the case a1=3, a2=5 and b=1.
(d) How do P(w1|x)and P(w2|x)behave as x→ −∞?x→+∞?. Explain. (Q 2.8 from Duda,
Hart and Stock)
4. Let wmax(x)be the state of nature for which P(wmax |x)≥P(wi|x)for all i, i=1...c.
(a) Show that P(wmax|x)≥1
c.
(b) Show that for the minimum-error-rate decision rule the average probability of error is given
by
P(error) = 1−ZP(wmax|x)p(x)dx(2)
(c) Use these two results to show that P(error)≤(c−1)
c.
(d) Describe a situation for which P(error) = (c−1)
c. (Q 2.12 from Duda, Hart and Stock)
