Download Barrier Methods - Lecture Notes - Continuous Optimization Methods | IOE 511 and more Study notes Systems Engineering in PDF only on Docsity!
13 Barrier Methods
Consider the constrained optimization problem (P):
(P) min f (x) s.t. gi (x) ≤ 0 , i = 1,... , m x ∈ R n^ ,
whose feasible region we denote by S = {x ∈ R n^ : gi (x) ≤ 0 , i = 1,... , m}. Barrier methods are also, like penalty methods, designed to solve (P) by instead solving a sequence of specially constructed unconstrained optimization problems.
In a barrier method, we presume that we are given a point x 0 that lies in the interior of the feasible region S, and we impose a very large cost on feasible points that lie ever closer to the boundary of S, thereby creating a “barrier” to exiting the feasible region.
Definition. A barrier function for (P) is any function b(x) : R n^ → R that satisfies
- b(x) ≥ 0 for all x that satisfy g(x) < 0, and
- b(x) → ∞ as lim (^) x max (^) i {gi (x)} → 0.
The idea in a barrier method is to dissuade points x from ever approaching the boundary of the feasible region. We consider solving:
B(c) min f (x) + (^1) c b(x) s.t. g(x) < 0 , x ∈ R n^.
for a sequence of ck → +∞. Note that the constraints “g(x) < 0” are effectively unimportant in B(c), as they are never binding in B(c).
Example:
b(x) =
∑^ m
i=
−gi (x)
Suppose g(x) = (x − 4 , 1 − x)T^ , x ∈ R 1. Then
b(x) =
4 − x
x − 1
Let r(c, x) = f (x) + (^1) c b(x). Let the sequence {ck } satisfy ck+1 > ck and ck → ∞ as k → ∞. Let xk denote the exact solution to B(ck ), and let x!^ denote any optimal solution of (P).
The following Lemma presents some basic properties of barrier methods.
Lemma 63 (Barrier Lemma) 1. r(ck , xk ) ≥ r(ck+1 , xk+1 )
- b(xk ) ≤ b(xk+1 )
- f (xk ) ≥ f (xk+1 )
- f (x!^ ) ≤ f (xk ) ≤ r(ck , xk ).
Proof:
r(ck , xk ) = f (xk ) +
ck b(xk ) ≥ f (xk ) +
ck+ b(xk )
≥ f (xk+1 ) +
ck+ b(xk+1 ) = r(ck+1 , xk+1 )
f (xk ) +
ck
b(xk ) ≤ f (xk+1 ) +
ck
b(xk+1 )
and f (xk+1 ) +
ck+ b(xk+1 ) ≤ f (xk ) +
ck+ b(xk ).
Summing and rearranging, we have ( 1 ck
ck+
b(xk ) ≤
ck
ck+
b(xk+1 ).
Since ck < ck+1 , it follows that b(xk+1 ) ≥ b(xk ).
- From the proof of (1.),
f (xk ) +
ck+
b(xk ) ≥ f (xk+1 ) +
ck+
b(xk+1 ).
But from (2.), b(xk ) ≤ b(xk+1 ). Thus f (xk ) ≥ f (xk+1 ).
- f (x!^ ) ≤ f (xk ) ≤ f (xk ) + (^) c^1 k b(xk ) = r(ck , xk ).
Let N (x, !) denote the ball of radius! centered at the point x. The next result concerns convergence of the barrier method.
Theorem 64 (Barrier Convergence Theorem). Suppose f (x), g(x), and b(x) are continuous functions. Let {xk }, k = 1,... , ∞, be a sequence of solutions of B(ck ). Suppose there exists an optimal solution x!^ of (P) for which N (x!^ , !) ∩ {x | g(x) < 0 } (= ∅ for every! > 0. Then any limit point ¯x of {xk } solves (P).
Proof: Let ¯x be any limit point of the sequence {xk }. From the continuity of f (x) and g(x), limk→∞ f (xk ) = f (¯x) and limk→∞ g(xk ) = g(¯x) ≤ 0. Thus ¯x is feasible for (P).
For any! > 0, there exists ˜x such that g(˜x) < 0 and f (˜x) ≤ f (x!^ ) + !. For each k,
f (x!^ ) +! +
ck b(˜x) ≥ f (˜x) +
ck b(˜x) ≥ r(ck , xk ).
Therefore for k sufficiently large, f (x!^ ) + 2! ≥ r(ck , xk ), and since r(ck , xk ) ≥ f (x!^ ) from (4.) of the Barrier Lemma, then f (x!^ ) + 2! ≥ lim k→∞
r(ck , xk ) ≥ f (x!^ ).
Therefore we can interpret the u (^) k as a sort of vector of Karush-Kuhn-Tucker multipliers. In fact, we have:
Lemma 65 Let (P) satisfy the conditions of the Barrier Convergence Theorem. Suppose γ(y) is continuously differentiable and let u (^) k be defined by (32). Then if xk → ¯x , and x¯ satisfies the linear independence condition for gradient vectors of active constraints, then u (^) k → ¯u, where ¯u is a vector of Karush-Kuhn-Tucker multipliers for the optimal solution ¯x of (P).
Proof: Let xk → x¯ and let I = {i | g (^) i (¯x) = 0} and N = {i | g (^) i (¯x) < 0 }. For all i ∈ N ,
[u (^) k ]i =
ck
∂γ(g(xk )) ∂yi → 0 as k → ∞,
since ck → ∞ and g (^) i (xk ) → gi (¯x) < 0, and ∂γ( ∂gy(¯ (^) ix ))is finite. Also [u (^) k ]i ≥ 0 for all i, and k sufficiently large.
Suppose u (^) k → u¯ as k → ∞. Then ¯u ≥ 0, and ¯u (^) i = 0 for all i ∈ N. From the continuity of all functions involved, (33) implies that
∇f (¯x) +
∑^ m
i=
u ¯ (^) i ∇gi (¯x) = 0, u¯ ≥ 0 , ¯u T^ g(¯x) = 0.
Thus ¯u is a vector of Kuhn-Tucker multipliers. It remains to show that u (^) k → u¯ for some unique ¯u. The proof that u (^) k → u¯ for some unique ¯u is exactly as in Lemma 56.
