Download Constrained Minimization: Algorithms and Complexity - Prof. Sergiy Butenko and more Study notes Systems Engineering in PDF only on Docsity!
1/
ISEN 629: Engineering Optimization
Lecture 18
Sergiy Butenko
Industrial and Systems Engineering
Texas A&M University
Fall 2007
2/
Constrained minimization
Consider the problem
min f (x),
s.t. f j (x) ≤ 0 , j = 1,... , m,
x ∈ Q,
where Q is a compact convex set and f (x), f j (x) are Lipschitz
continuous on Q.
Denoting by
f (x) = max
1 ≤j≤m
f j (x), we obtain the equivalent problem
min f (x),
s.t.
f (x) ≤ 0 ,
x ∈ Q,
Then f (x),
f (x) are convex and Lipschitz continuous on Q.
Constrained minimization
Let us introduce the parametric function:
f (t; x) = max{f (x) − t,
f (x)}
and denote by
f
∗
(t) = min
x∈Q
f (t; x). (1)
Note that f
∗
(t) is nonincreasing in t.
Lemma
Let x
∗ be the optimal solution of problem (∗) with t
∗ = f (x
∗ )
being the optimal objective value. Then
f
∗
(t) ≤ 0 for all t ≥ t
∗
,
f
∗
(t) > 0 for all t < t
∗
.
Thus, t
∗ is the smallest root of function f
∗ (t) and it corresponds
to the optimal value of (∗).
Constrained minimization
To solve (∗), we will use models of both f (x) and
f (x) (which will
be used to introduce a model function for f (t, x)).
For a sequence X = {x k : k ≥ 0 }, denote by
f k
(X ; x) = max
0 ≤j≤k
[f (x j
) + g (x j
T (x − x j
)] ≤ f (x),
f k
(X ; x) = max
0 ≤j≤k
[
f (x j
) + ¯g (x j
T (x − x j
)] ≤
f (x),
where g (x j ) ∈ ∂f (x j ) and ¯g (x j
f (x j
5/
Constrained minimization
The models of f (x) and
f (x) can be used to introduce the model
for f (t; x):
f k (X ; t, x) = max{
f k (X ; x) − t,
f k (X , x)} ≤ f (t; x),
f
∗
k
(X ; t) = min
x∈Q
f k
(X ; t, x) ≤ f
∗ (t).
Then
f
∗
k
(X ; t) is nonincreasing in t and its smallest root t
∗
k
(X )
does not exceed t
∗ .
Lemma (3.3.4)
t
∗
k
(X ) = min{
f k (X ; x) :
f k (X ; x) ≤ 0 , x ∈ Q}.
Denote by
f
∗
k
(X ; t) = min
0 ≤j≤k
f k (X ; t, x j
the record value of our parametric model.
6/
Constrained level method
- Choose x 0
∈ Q, t 0
< t
∗ , κ ∈ (0, 1 /2) and > 0.
- k-th iteration (k ≥ 0):
(a) Keep generating sequence X = {x j : j ≥ 0 } by the level
method applied to function f (t k
; x). If the internal termination
criterion
ˆ f
∗
j
(X ; t k ) ≥ (1 − κ)f
∗
j
(X ; t k )
†
holds, then stop the internal process and set j(k) = j.
Global stop: f
∗
j
(X ; t k
) ≤ .
(b) Set t k+ = t
∗
j(k)
(X ).
† same as f
∗
j
(X ; t k
f
∗
j
(X ; t k
) ≤ δ with δ = κf
∗
j
(X ; t k
Constrained level method
Let the global stop condition be satisfied:
f
∗
j
(X ; t k
Then there exists j
∗
such that
f (t k , x j
∗ ) = f
∗
j
(X ; t k
Therefore, we have
f (t k
, x j
∗ ) = max{f (x j
∗ ) − t k
f (x j
Since t k
≤ t
∗ , we conclude that
f (x j
∗ ) ≤ t
∗
f (x j
Constrained level method: complexity
To derive the analytical complexity bound for the constrained level
method, we need to estimate
- the complexity of the master process (i.e., the maximum
number of iterations indexed by k);
- the complexity of step 1(a).
13/
Constrained level method
- Choose x 0 ∈ Q, t 0 < t
∗
, κ ∈ (0, 1 /2) and > 0.
- k-th iteration (k ≥ 0):
(a) Keep generating sequence X = {x j : j ≥ 0 } by the level
method applied to function f (t k ; x). If the internal termination
criterion
ˆ f
∗
j
(X ; t k ) ≥ (1 − κ)f
∗
j
(X ; t k )
holds, then stop the internal process and set j(k) = j.
Global stop: f
∗
j
(X ; t k ) ≤ .
(b) Set t k+
= t
∗
j(k)
(X ).
Remaining issues to discuss:
Computing
f
∗
j
(X ; t k ) - reduces to LP if Q is a polytope
[previous lecture].
Finding the root t
∗
j(k)
(X ).
14/
Constrained level method: finding t
∗
j(k)
(X )
Recall Lemma (3.3.4): t
∗
k
(X ) = min{ ˆ f k
(X ; x) : ˇf k
(X ; x) ≤ 0 , x ∈ Q}.
Thus, finding t
∗
j(k)
(X ) is equivalent to solving the problem
min{
f k
(X ; x) :
f k
(X ; x) ≤ 0 , x ∈ Q},
which is equivalent to
min t,
s.t. f (x j ) + g (x j
T
(x − x j ) ≤ t, j = 0,... , k,
f (x j ) + ¯g (x j
T
(x − x j ) ≤ 0 , j = 0,... , k,
x ∈ Q.
If Q is a polytope, this is an LP.
If Q is more complicated, we can use interior point methods.
15/
Constrained level method
Remark: Since
f (x) = max
1 ≤i≤m
f i (x), we can use
f k
(X ; x) = max
0 ≤j≤k
max
1 ≤i≤m
[f i
(x j
) + g i
(x j
T (x − x j
)] ≤
f (x),
where g i (x j ) ∈ ∂f i (x j ), instead of
f k (X ; x) = max
0 ≤j≤k
[
f (x j ) + ¯g (x j
T
(x − x j
)] ≤
f (x).
In practice, this complete model accelerates the convergence of the
process at expense of increased per-iteration complexity.
