CS 680
Homework 2 : SVMs
(due October 4th)
1. SVM with no bias term [35 pts].
Formulate a soft-margin SVM without the bias term, i.e. f(x) = wTx. Derive the
saddle point conditions, KKT conditions and the dual. Compare it to the standard
SVM formulation. What is the implication of the difference on the design of SMO-like
algorithms? Recall that SMO algorithms work by iteratively optimizing two variables
at a time. Hint: consider the difference in the constraints.
Discuss the merit of the bias-less formulation as the dimensionality of the data (or the
feature space) is varied. When using this SVM formulation it may be useful to add a
constant to the kernel matrix. Explain why this can be beneficial.
2. Soft-margin for separable data [15 pts].
Consider training a soft-margin SVM with Cset to some positive constant. Suppose
the training data is linearly separable. Since increasing the ξican only increase the
objective of the primal problem (which we are trying to minimize), at the optimal
solution to the primal problem, all the training examples will have functional margin
at least 1 and all the ξiwill be equal to zero. True or false? Explain! Given a
linearly separable dataset, is it necessarily better to use a a hard margin SVM over a
soft-margin SVM?
3. In-bound SVs in soft-margin SVMs [20 pts].
Examples xiwith αi>0 are called support vectors (SVs). For soft-margin SVM we
distinguish between in bound SVs, for which 0 < αi< C, and bound SVs for which
αi=C. Show that in-bound SVs lie exactly on the margin. Argue that bound SVs
can lie both on or in the margin, and that they will “usuallly” lie in the margin. Hint:
use the KKT conditions.
4. Margin of optimal margin hyperplanes [10 pts].
In class we saw that the geometric margin of an optimal margin hyperplane, denoted
by ρ, is equal to 2/||w||. Show that for an optimal margin hyperplane
2ρ−2=
n
X
i=1
αi
and
4ρ−2= 2W(α) = ||w||2,
where W(α) is the dual function to be optimized.
