CS 680
Homework 1 (due September 18th)
1. Ridge regression [10 pts].
The ridge regression optimization can be solved by gradient descent. Write this algo-
rithm in the primal and dual formulations.
2. A kernel for discrete data [10 pts].
Let Ω be a set, and let A, B be subsets of Ω. Show that the following functions are
kernels:
K1(A, B) = |A∩B|,
and
K2(A, B) = 2|A∩B|,
where |·|is the cardinality of a set.
3. Is this a kernel? [35 pts]
Let K1and K2be kernels over X × X. Let f:X 7→ R, let abe a positive number,
let φbe a function φ:X 7→ Rm, and let K3be a kernel over Rm×Rm. For each of
the functions Kdefined below, state whether it is a kernel. If you think it is a kernel,
prove it by explicitly showing the feature map for which Kis a dot product; otherwise
prove it is not a kernel.
(a) K(x,x0) = K1(x,x0) + K2(x,x0)
(b) K(x,x0) = K1(x,x0)−K2(x,x0)
(c) K(x,x0) = aK1(x,x0)
(d) K(x,x0) = −aK1(x,x0)
(e) K(x,x0) = K1(x,x0)K2(x,x0)
(f) K(x,x0) = K3(φ(x), φ(x0))
(g) K(x,x0) = f(x)f(x0)
(h) K(x,x0) = K1(x,x0)
√K1(x,x)K1(x0,x0)
4. Positivity on the diagonal [5 pts]. Prove that every kernel satisfies K(x,x)≥
0∀x∈ X.
5. Kernels with a vanishing diagonal [5 pts]. Prove that a kernel satisfying K(x,x) =
0∀x∈ X is identically zero. You can use the Cauchy-Schwarz inequality which states
that for all x,x0in some vector space X
|hx,x0i| ≤ ||x|| · ||x0||,
with equality occurring if and only if xand x0are co-linear (i.e. there exists λsuch
that x0=λx).
