Homework 5: Methods for Simple Constrained Problems
Due November 14, 2012 in class
NOTE: All norms are Euclidean unless clearly indicated otherwise.
Exercise 1: (20 points)
Consider the gradient projection method with the stepsize α= 1/L for a function f
with Lipshitz continuous gradients, where Lis the Lipschitz constant (see the notes
on First-Order Methods). Show that
xk+1 = argmin
y∈X
Q(y;xk),
where Q(y;x) is the quadratic over-estimate of f, i.e.,
Q(y;x) = f(x) + ∇f(x)0(y−x) + L
2ky−xk2for all x, y ∈Rn.
Exercise 2: (40 =20 I + 20 II, each (a)(b) is 10 points)
This exercise will require the use of Matlab to develop a code for solving the following
problem:
minimize f(x1, x2) = 1
2(x2
1+ax2
2)
subject to (x1, x2)∈[0,10] ×[0,10],
where ais a parameter to be specified shortly.
(I) Solve the problem by using the first-order gradient method with initial point
x0= (10,10) and a stepsize αk=1
L, where Lis the Lipshcitz gradient constant
for the given function. Using Matlab, run simulations to solve the following
instances of the problem:
(a) a= 100.
(b) a= 10000.
(I) Repeat part (I) by using the Nesterov first method instead, with the same initial
point as in part (I).
For both parts (I) and (I) to report your simulation results, do the following:
(1) Report the value of the constant Lthat you have computed.
(2) Run the algorithm for 1000 iterations in each part (a) and (b).
(3) Provide a plot showing the function values obtained by the algorithm in cases (a)
and (b) as the iteration index is varying k= 1,...,1000.
1
