Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Prof. Devilaal Chandra delivered this lecture for Convex Optimization course at Alagappa University. Its main points are: Interior, Point, Methods, Logarithmic, Barrier, Function, Feasibility, Generalized, Inequalities
Typology: Slides
1 / 32
This page cannot be seen from the preview
Don't miss anything!
Convex Optimization — Boyd & Vandenberghe
inequality constrained minimization
logarithmic barrier function and central path
barrier method
feasibility and phase I methods
complexity analysis via self-concordance
generalized inequalities
12–
minimize
f
0
x
subject to
f
i
x
i
,... , m
Ax
b
f
i
convex, twice continuously differentiable
p
×
n
with
rank
p
we assume
p
⋆
is finite and attained
we assume problem is strictly feasible: there exists
x
with
x
dom
f
0
f
i
x
i
,... , m,
x
b
hence, strong duality holds and dual optimum is attained
Interior-point methods
12–
reformulation of (1) via indicator function:
minimize
f
0
x
mi
=
−
f
i
x
subject to
Ax
b
where
−
u
if
u
−
u
otherwise (indicator function of
−
approximation via logarithmic barrier
minimize
f
0
x
/t
mi
=
log(
f
i
x
subject to
Ax
b
an equality constrained problem
for
t >
/t
) log(
u
is a
smooth approximation of
−
approximation improves as
t
u
−
3
−
2
−
1
0
1
−
5 0 5
10
Interior-point methods
12–
docsity.com
logarithmic barrier function
φ
x
m
i
=
log(
f
i
x
dom
φ
x
f
1
x
,... , f
m
x
convex (follows from composition rules)
twice continuously differentiable, with derivatives
φ
x
m
i
=
f
i
x
f
i
x
2
φ
x
m
i
=
f
i
x
2
f
i
x
f
i
x
T
m
i
=
f
i
x
2
f
i
x
Interior-point methods
12–
x
x
⋆
t
if there exists a
w
such that
t
f
0
x
m
i
=
f
i
x
f
i
x
T
w
Ax
b
therefore,
x
⋆
t
minimizes the Lagrangian
x, λ
⋆
t
, ν
⋆
t
f
0
x
m
i
=
λ
⋆i
t
f
i
x
ν
⋆
t
T
Ax
b
where we define
λ
⋆i
t
tf
i
x
⋆
t
and
ν
⋆
t
w/t
this confirms the intuitive idea that
f
0
x
⋆
t
p
⋆
if
t
p
⋆
g
λ
⋆
t
, ν
⋆
t
x
⋆
t
, λ
⋆
t
, ν
⋆
t
f
0
x
⋆
t
m/t
Interior-point methods
12–
x
x
⋆
t
λ
λ
⋆
t
ν
ν
⋆
t
satisfy
f
i
x
i
,... , m
Ax
b
λ
λ
i
f
i
x
/t
i
,... , m
x
vanishes:
f
0
x
m
i
=
λ
i
f
i
x
T
ν
difference with KKT is that condition 3 replaces
λ
i
f
i
x
Interior-point methods
12–
example
minimize
c
T
x
subject to
a
Ti
x
b
i
i
,... , m
objective force field is constant:
0
x
tc
constraint force field decays as inverse distance to constraint hyperplane:
i
x
a
i
b
i
a
Ti
x
i
x
2
dist
x,
i
where
i
x
a
Ti
x
b
i
−
c
−
3
c
t
= 1
t
= 3
Interior-point methods
12–
given
strictly feasible
x
,
t
:=
t
(0)
0
,
μ >
1
, tolerance
ǫ >
0
.
repeat
Centering step.
Compute
x
⋆
(
t
)
by minimizing
tf
0
φ
, subject to
Ax
=
b
.
Update.
x
:=
x
⋆
(
t
)
.
Stopping criterion.
quit
if
m/t < ǫ
.
Increase
t
.
t
:=
μt
.
terminates with
f
0
x
p
⋆
ǫ
(stopping criterion follows from
f
0
x
⋆
t
p
⋆
m/t
centering usually done using Newton’s method, starting at current
x
choice of
μ
involves a trade-off: large
μ
means fewer outer iterations,
more inner (Newton) iterations; typical values:
μ
several heuristics for choice of
t
(0)
Interior-point methods
12–
inequality form LP
m
inequalities,
n
variables)
Newton iterations
duality gap
μ
= 2
μ
= 50
μ
= 150
0
20
40
60
80
10
−
6
10
−
4
10
−
2
10
0
10
2
μ
Newton iterations
0
40
80
120
160
200
0
80 60 40 20
140120100
starts with
x
on central path (
t
(0)
, duality gap
terminates when
t
8
(gap
−
6
centering uses Newton’s method with backtracking
total number of Newton iterations not very sensitive for
μ
Interior-point methods
12–
geometric program
m
inequalities and
n
variables)
minimize
log
5 k
=
exp(
a
T 0
k
x
b
0
k
subject to
log
5 k
=
exp(
a
Tik
x
b
ik
i
,... , m
Newton iterations
duality gap
μ
= 2
μ
= 50
μ
= 150
0
20
40
60
80
100
120
10
−
6
10
−
4
10
−
2
10
0
10
2
Interior-point methods
12–
feasibility problem:
find
x
such that
f
i
x
i
,... , m,
Ax
b
phase I
: computes strictly feasible starting point for barrier method
basic phase I method
minimize (over
x
s
s
subject to
f
i
x
s,
i
,... , m
Ax
b
if
x
s
feasible, with
s <
, then
x
is strictly feasible for (2)
if optimal value
p
⋆
of (3) is positive, then problem (2) is infeasible
if
p
⋆
and attained, then problem (2) is feasible (but not strictly);
if
p
⋆
and not attained, then problem (2) is infeasible
Interior-point methods
12–
sum of infeasibilities phase I method
minimize
T
s
subject to
s
f
i
x
s
i
i
,... , m
Ax
b
for infeasible problems, produces a solution that satisfies many moreinequalities than basic phase I method example
(infeasible set of 100 linear inequalities in 50 variables)
b
i
−
a
Ti
x
max
number
−
1
−
0
.
5
0
0
.
5
1
1
.
5
0
60 40 20
number
−
1
−
0
.
5
0
0
.
5
1
1
.
5
0
60 40 20
b
i
−
a
Ti
x
sum
left: basic phase I solution; satisfies 39 inequalitiesright: sum of infeasibilities phase I solution; satisfies 79 inequalities Interior-point methods
12–
same assumptions as on page 12–2, plus:
sublevel sets (of
f
0
, on the feasible set) are bounded
tf
0
φ
is self-concordant with closed sublevel sets
second condition
holds for LP, QP, QCQP
may require reformulating the problem,
e.g.
minimize
n i
=
x
i
log
x
i
subject to
F x
g
minimize
n i
=
x
i
log
x
i
subject to
F x
g,
x
needed for complexity analysis; barrier method works even whenself-concordance assumption does not apply
Interior-point methods
12–
Newton iterations per centering step:
from self-concordance theory
Newton iterations
μtf
0
x
φ
x
μtf
0
x
φ
x
γ
c
bound on effort of computing
x
x
⋆
μt
starting at
x
x
⋆
t
γ
c
are constants (depend only on Newton algorithm parameters)
from duality (with
λ
λ
⋆
t
ν
ν
⋆
t
μtf
0
x
φ
x
μtf
0
x
φ
x
μtf
0
x
μtf
0
x
m
i
=
log(
μtλ
i
f
i
x
m
log
μ
μtf
0
x
μtf
0
x
μt
m
i
=
λ
i
f
i
x
m
m
log
μ
μtf
0
x
μtg
λ, ν
m
m
log
μ
m
μ
log
μ
Interior-point methods
12–
