Subgradient Methods for Constrained Problems
•projected subgradient method
•projected subgradient for dual
•subgradient method for constrained optimization
Prof. S. Boyd, EE364b, Stanford University
docsity.com
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Subgradient Methods for Constrained Problems-Software and Convex Optimization-Lecture Slides, Slides of Convex Optimization
Dr. Hanumant Chawd delivered this lecture at Alagappa University for Convex Optimization course. Its main points are: Subgradient, Methods, Constrained, Problems, Optimization, Euclidean, Projection, Linear, Equality
Typology: Slides
2011/2012
1 / 14
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Subgradient Methods for Constrained Problems-Software and Convex Optimization-Lecture Slides and more Slides Convex Optimization in PDF only on Docsity!
Subgradient Methods for Constrained Problems
projected subgradient method
projected subgradient for dual
subgradient method for constrained optimization
Prof. S. Boyd, EE364b, Stanford University
docsity.com
Projected subgradient method
solves constrained optimization problem
minimize
f
x
subject to
x
∈ C
where
f
R
n
R
C ⊆
R
n
are convex
projected subgradient method
is given by
x
(
k
+1)
P
x
(
k
)
α
k
g
(
k
)
P
is (Euclidean) projection on
C
, and
g
(
k
)
∂f
x
(
k
)
Prof. S. Boyd, EE364b, Stanford University
1
docsity.com
Linear equality constraints
minimize
f
x
subject to
Ax
b
projection of
z
onto
x
Ax
b
is
P
z
z
A
T
AA
T
−
1
Az
b
I
A
T
AA
T
−
1
A
z
A
T
AA
T
−
1
b
projected subgradient update is (using
Ax
(
k
)
b
x
(
k
+1)
P
x
(
k
)
α
k
g
(
k
)
x
(
k
)
α
k
I
A
T
AA
T
−
1
A
g
(
k
)
x
(
k
)
α
k
P
N
(
A
)
g
(
k
)
Prof. S. Boyd, EE364b, Stanford University
3
docsity.com
Example: Least
l
1
-norm
minimize
x
1
subject to
Ax
b
subgradient of objective is
g
sign
x
projected subgradient update is
x
(
k
+1)
x
(
k
)
α
k
I
A
T
AA
T
−
1
A
sign
x
(
k
)
Prof. S. Boyd, EE364b, Stanford University
4
docsity.com
Projected subgradient for dual problem
(convex) primal:
minimize
f
0
x
subject to
f
i
x
i
,... , m
solve dual problem
maximize
g
λ
subject to
λ
via projected subgradient method:
λ
(
k
+1)
λ
(
k
)
α
k
h
h
g
λ
(
k
)
Prof. S. Boyd, EE364b, Stanford University
6
docsity.com
Subgradient of negative dual function
assume
f
0
is strictly convex, and denote, for
λ
x
∗
λ
) = argmin
z
f
0
z
λ
1
f
1
z
λ
m
f
m
z
so
g
λ
f
0
x
∗
λ
λ
1
f
1
x
∗
λ
λ
m
f
m
x
∗
λ
a subgradient of
g
at
λ
is given by
h
i
f
i
x
∗
λ
projected subgradient method for dual:
x
(
k
)
x
∗
λ
(
k
)
λ
(
k
+1)
i
λ
(
k
)
i
α
k
f
i
x
(
k
)
Prof. S. Boyd, EE364b, Stanford University
7
docsity.com
Example
minimize strictly convex quadratic (
P
) over unit box:
minimize
x
T
P x
q
T
x
subject to
x
2 i
i
,... , n
L
x, λ
x
T
P
diag
λ
x
q
T
x
T
λ
x
∗
λ
P
diag
λ
−
1
q
projected subgradient for dual:
x
(
k
)
P
diag
λ
(
k
)
−
1
q,
λ
(
k
+1)
i
λ
(
k
)
i
α
k
x
(
k
)
i
2
Prof. S. Boyd, EE364b, Stanford University
9
docsity.com
problem instance with
n
, fixed step size
α
f
⋆
x
(
k
)
is a nearby feasible point for
x
(
k
)
5
10
15
20
25
30
35
40
−10 −20 −30 −40 −
0
k
upper and lower bounds
f
0
(˜
x
(
k
)
)
g
(
λ
(
k
)
)
Prof. S. Boyd, EE364b, Stanford University
10 docsity.com
Convergence
assumptions:
there exists an optimal
x
⋆
; Slater’s condition holds
g
(
k
)
2
G
x
(1)
x
⋆
2
R
typical result
: for
α
k
α
k
∞ i
=
α
i
, we have
f
(
k
)
best
f
⋆
Prof. S. Boyd, EE364b, Stanford University
12 docsity.com
Example: Inequality form LP
LP with
n
variables,
m
inequalities,
f
⋆
α
k
/k
for optimality step, Polyak’s step size for feasibility step
0
500
1000
1500
2000
2500
10
−
10
−
10
0
10
1
k
f
)k(
best
f −
⋆
Prof. S. Boyd, EE364b, Stanford University
13 docsity.com