EE364b Prof. S. Boyd
EE364b Homework 5
1. Distributed lasso. Consider the ℓ1-regularized least-squares (‘lasso’) problem
minimize f(z) = (1/2)
"A10B1
0A2B2#
x1
x2
y
−"c1
c2#
2
2
+λ
x1
x2
y
1
,
with optimization variable z= (x1, x2, y)∈Rn1×Rn2×Rp. We can think of xias the
local variable for system i, for i= 1,2; yis the common or coupling variable.
(a) Primal decomposition. Explain how to solve this problem using primal decompo-
sition, using (say) the subgradient method for the master problem.
(b) Dual decomposition. Explain how to solve this problem using dual decomposition,
using (say) the subgradient method for the master problem. Give a condition (on
the problem data) that allows you to guarantee that the primal variables x(k)
i
converge to optimal values.
(c) Numerical example. Generate some numerical data as explained below, and solve
the problem (using CVX) to find the optimal value p⋆. Implement primal and
dual decomposition (as in parts (a) and (b)), using CVX to solve the subproblems,
and the subgradient method for the master problem in both cases. For primal
decomposition, plot the relative suboptimality (f(z(k))−p⋆)/p⋆versus iteration.
For dual decomposition, plot the relative consistency residual ky(k)
1−y(k)
2k2/ky⋆k2
versus iteration, where y⋆is an optimal value of yfor the problem. In each case,
you needn’t worry about attaining a relative accuracy better than 0.001, which
corresponds to 0.1%.
Generating the data. Generate Ai,Bi, and ciwith entries from a standard Gaus-
sian, with dimensions n1= 100, n2= 200, p= 10, m1= 250, and m2= 300
(these last two are the dimensions of c1and c2). Check that the condition you
gave in part (b) is satisfied. Choose λ= 0.1λmax, where λmax is the value of λ
above which the solution is z⋆= 0. (See homework 2, exercise 3.) To get reason-
able convergence (say, in a few tens of iterations), you may need to play with the
subgradient step size.
You are of course welcome (even, encouraged) to also try your distributed lasso
solver on problem instances other than the one generated above.
2. Cutting plane for nonexpansive operators. Suppose F:Rn→Rnis nonexpansive, and
let x⋆be any fixed point. For x∈Rn, let z= (1/2)(x+F(x)) (which would be the
next iterate in damped iteration, with θ= 1/2). Show that
(F(x)−x)T(x⋆−z)≥0.
1
docsity.com
