CS446: Pattern Recognition and Machine Learning Fall 2008
Problem Set 5
Handed Out: November 6, 2008 Due: November 20, 2008
•Feel free to talk to other members of the class in doing the homework. I am more concerned that
you learn how to solve the problem than that you demonstrate that you solved it entirely on your
own. You should, however, write down your solution yourself. Please try to keep the solution brief
and clear.
•Feel free to send me email or come to ask questions regarding this handout or conceptual issues.
•Please, no handwritten solutions.
•The homework is due at 4:00 pm on the due date. Email your write-up to the TA. Please put
“<userid>CS446 hw5 submission” as the subject line of the email when you submit your homework
1. [Probability - 10 points]
Define the following boolean, random variables:
•h= pizza is hot
•t= pizza is tasty
Assume we know that P(h) = 0.2, P(t) = 0.4, and P(t|h) = 0.95. Show and explain
your calculations for both part (a) and part (b).
(a) What is P(h|¬t)?
(b) Given that the pizza is tasty, what’s the probability that it is hot?
2. [Learning Threshold Functions - 30 points]
Consider the Boolean function fT H (3,7). This is a threshold function defined on the 7
dimensional Boolean cube as follows: given an instance x,fT H(3,7) (x) = 1 if and only
if 3 or more of x’s components are 1.
(a) Show that fTH (3,7) has a linear decision surface over the 7 dimensional Boolean
cube.
(b) Assume that you are given data sampled according to the uniform distribution
over the Boolean cube {0,1}7and labeled according to fT H (3,7). Use na¨ıve Bayes
to learn a hypothesis that predicts these labels. What is the hypothesis generated
by the na¨ıve Bayes algorithm? (You may assume that you have seen all the data
required to get accurate estimates of the probabilities).
(c) Show that the hypothesis produced in (b) does not represent this function.
(d) Are the na¨ıve Bayes assumptions satisfied by fT H(3,7) ? Justify your answer.
1
