Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Dr. Hanumant Chawd delivered this lecture at Alagappa University for Convex Optimization course. Its main points are: Localization, Cutting, Plane, Methods, Algorithms, Stopping, Criteria, Epigraph, Computation, Subgradient, Methods
Typology: Slides
1 / 29
This page cannot be seen from the preview
Don't miss anything!
cutting-plane oracle
finding cutting-planes
localization algorithms
specific cutting-plane methods
epigraph cutting-plane method
lower bounds and stopping criteria
Prof. S. Boyd, EE364b, Stanford University
docsity.com
based on idea of ‘localizing’ desired point in some set, which becomessmaller at each step
like subgradient methods, require computation of a subgradient ofobjective or constraint functions at each step
in particular, directly handle nondifferentiable convex (and quasiconvex)problems
typically require more memory and computation per step thansubgradient methods
but can be much more efficient (in theory and practice) thansubgradient methods
Prof. S. Boyd, EE364b, Stanford University
1 docsity.com
if
a
T
x
b
x
is on boundary of halfspace that is cut) cutting-plane is
called
neutral cut
if
a
T
x > b
x
lies in interior of halfspace that is cut), cutting-plane is
called
deep cut
x
x
Prof. S. Boyd, EE364b, Stanford University
3 docsity.com
minimize convex
f
n
is set of optimal points (minimizers)
given
x
, find
g
∂f
x
from
f
z
f
x
g
T
z
x
we conclude
g
T
z
x
f
z
f
x
i.e.
, all points in halfspace
g
T
z
x
are
worse
than
x
, and in
particular not optimal
so
g
T
z
x
is (neutral) cutting-plane at
x
a
g
b
g
T
x
Prof. S. Boyd, EE364b, Stanford University
4 docsity.com
suppose we know a number
f
with
f
x
f
f
⋆
e.g.
, the smallest value of
f
found so far in an algorithm)
from
f
z
f
x
g
T
z
x
, we have
f
x
g
T
z
x
f
f
z
f
f
⋆
z
so we have deep cut
g
T
z
x
f
x
f
Prof. S. Boyd, EE364b, Stanford University
6 docsity.com
find
x
subject to
f
i
x
i
,... , m
f
1
,... , f
m
convex;
is set of feasible points
if
x
not feasible, find
j
with
f
j
x
, and evaluate
g
j
∂f
j
x
since
f
j
z
f
j
x
g
Tj
z
x
f
j
x
g
Tj
z
x
f
j
z
z
i.e.
, any feasible
z
satisfies the inequality
f
j
x
g
Tj
z
x
this gives a deep cut
Prof. S. Boyd, EE364b, Stanford University
7 docsity.com
basic (conceptual) localization (or cutting-plane) algorithm:
given
initial polyhedron
0
z
Cz
d
known to contain
k
repeat
Choose a point
x
(
k
+1)
in
k
Query the cutting-plane oracle at
x
(
k
+1)
If
x
(
k
+1)
, quit
Else, add new cutting-plane
a
Tk
z
b
k
k
k
z
a
Tk
z
b
k
If
k
, quit
k
k
Prof. S. Boyd, EE364b, Stanford University
9 docsity.com
P
k
x
(
k
+1)
x
(
k
+1)
a
k
a
k
P
k
k
gives our uncertainty of
x
⋆
at iteration
k
want to pick
x
(
k
+1)
so that
k
is as small as possible, no matter
what cut is made
want
x
(
k
+1)
near center of
(
k
)
Prof. S. Boyd, EE364b, Stanford University
10 docsity.com
minimize convex
f
k
is interval
obvious choice for query point:
x
(
k
+1)
midpoint
k
bisection algorithm
given
interval
0
l, u
containing
x
⋆
repeat
x
l
u
f
′
x
f
′
x
l
x
; else
u
x
Prof. S. Boyd, EE364b, Stanford University
12 docsity.com
P
k
P
k
x
(
k
+1)
Prof. S. Boyd, EE364b, Stanford University
13 docsity.com
methods vary in choice of query point
center of gravity (CG) algorithm
x
(
k
+1)
is center of gravity of
k
maximum volume ellipsoid (MVE) cutting-plane method
x
(
k
+1)
is center of maximum volume ellipsoid contained in
k
Chebyshev center cutting-plane method
x
(
k
+1)
is Chebyshev center of
k
analytic center cutting-plane method (ACCPM)
x
(
k
+1)
is analytic center of (inequalities defining)
k
Prof. S. Boyd, EE364b, Stanford University
15 docsity.com
take
x
(
k
+1)
k
(center of gravity) CG
k
P
k
x dx
P
k
dx
theorem.
if
n
convex,
x
cg
g
vol
x
g
T
x
x
cg
/e
vol
vol
(independent of dimension
n
hence in CG algorithm,
vol
k
k
vol
0
Prof. S. Boyd, EE364b, Stanford University
16 docsity.com
advantages of CG-method
guaranteed convergence
affine-invariance
number of steps proportional to dimension
n
, log of uncertainty
reduction
disadvantages
finding
x
(
k
+1)
k
is
much harder
than original problem
(but, can modify CG-method to work with approximate CG computation) Prof. S. Boyd, EE364b, Stanford University
18 docsity.com
x
(
k
+1)
is center of maximum volume ellipsoid in
k
(can compute as convex problem)
affine-invariant
can show
vol
k
/n
vol
k
hence can bound number of steps:
k
n
log(
R/r
log(
/n
n
2
log(
R/r
if cutting-plane oracle cost is not small, MVE is a good practical method
Prof. S. Boyd, EE364b, Stanford University
19 docsity.com
