CS446: Pattern Recognition and Machine Learning Fall 2008
Problem Set 4
Handed Out: October 9, 2008 Due: October 20, 2008
•Feel free to talk to other members of the class in doing the homework. I am more concerned that
you learn how to solve the problem than that you demonstrate that you solved it entirely on your
own. You should, however, write down your solution yourself. Please try to keep the solution brief
and clear.
•Feel free to send me email or come to ask questions regarding this handout or conceptual issues.
•Please, no handwritten or hard copy solutions.
•The homework is due at 4:00 pm on the due date. Email your write-up to the TA. Please put
“<userid>CS446 hw4 submission” as the subject line of the email when you submit your homework
1. [Computing Margins - 35 points]
The margin of a set of points {x1, x2,...,xm}is defined to be the distance of the closest
point to the hyperplane w·xc−θ= 0.
The margin γis thus:
γ=mini
w·xi−θ
kwk
(a) In the file hw4 1a.dat we have given you set of 50 points in {0,1}20: 20 dimen-
sional binary vectors.
Using the hyperplane w∈ <20:
w=<−1,1,1,0,1,0,0,0,0,0,−1,1,−1,0,1,1,0,0,0,−1>, θ = 1.5
Classify each of the points in hw4 1a.dat and compute the margin of this data
set with respect to this hyperplane. (No code is needed for any of these problems
though. Do the calculations however you please.) [5 points]
(b) Now using the sparse disjunction x3∨x6∨x9∨x12:
i. Come up with a linear threshold function that represents this concept. Using
this hyperplane, label hw4 1a.dat and compute the margin. [5 points]
ii. What is the minimum distance between a positive and negative example. [5
points]
(c) Now using the denser disjunction x2∨x4∨x6∨x8∨x10 ∨x12 ∨x14 ∨x16 ∨x18:
i. Come up with a linear threshold function that represents this concept. Using
this hyperplane, label hw4 1a.dat and compute the margin. [5 points]
ii. What is the minimum distance between a positive and negative example. [5
points]
1
