Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
It gsimplifies optimization techniques
Typology: Lecture notes
1 / 44
This page cannot be seen from the preview
Don't miss anything!
Ling-Chieh Kung
Department of Information Management National Taiwan University
I (^) Motivating examples. I (^) Convex analysis. I (^) Single-variate NLPs. I (^) Multi-variate convex analysis.
In an NLP which has an optimal solution, there may exist no extreme point optimal solution.
I (^) For example:
min x 1 ≥ 0 ,x 2 ≥ 0
x^21 + x^22
s.t. x 1 + x 2 ≥ 4.
I (^) The optimal solution x∗^ = (2, 2) is not an extreme point. I (^) The two extreme points are not optimal.
I (^) No one has invented an efficient algorithm for solving general NLPs (i.e., finding a global optimum). I (^) For an NLP: I (^) We want to have a condition that makes a local minimum always a global minimum. I (^) We want to have a condition that guarantees an extreme point optimal solution (when there is an optimal solution). I (^) To answer these questions, we need convex analysis. I (^) Let’s define convex sets and convex and concave functions. I (^) Then we define convex programs and show that they have the first desired property.
I (^) Let’s start by defining convex sets and convex functions:
A set F ⊆ Rn^ is convex if
λx 1 + (1 − λ)x 2 ∈ F
for all λ ∈ [0, 1] and x 1 , x 2 ∈ F.
For a convex domain F ⊆ Rn, a function f : Rn^ → R is convex over F if f
λx 1 + (1 − λ)x 2
≤ λf (x 1 ) + (1 − λ)f (x 2 )
for all λ ∈ [0, 1] and x 1 , x 2 ∈ F.
For a convex (concave) function f over a convex domain F , a local minimum (maximum) is a global minimum (maximum).
f (x) = x^3 + x^2 − x. f (x, y) = x^2 + y^2.
Proof. Suppose a local minimum x′^ is not a global minimum and there exists x′′^ such that f (x′′) < f (x′). Consider a small enough λ > 0 such that ¯x = λx′′^ + (1 − λ)x′^ satisfies f (¯x) > f (x′). Such ¯x exists because x′^ is a local minimum. Now, note that
f (¯x) = f (λx′′^ + (1 − λ)x′)
f (x′) = λf (x′) + (1 − λ)f (x′) λf (x′′) + (1 − λ)f (x′),
which violates the fact that f (·) is convex. Therefore, by contradiction, the local minimum x′^ must be a global minimum.
I (^) Now we know if we minimize a convex function over a convex feasible region, a local minimum is a global minimum. I (^) What may happen if we minimize a concave function? I (^) One “goes down” on a concave function if she moves “towards its boundary”. I (^) We thus have the following proposition:
For any concave function that has a global minimum over a convex feasible region, there exists a global minimum that is an extreme point.
Proof. Beyond the scope of this course.
I (^) Now we know when we minimize f (·) over a convex feasible region F : I (^) If f (·) is convex, search for a local minimum. I (^) If f (·) is concave, search among the extreme points of F. I (^) For any LP, we have both!
The feasible region of an LP is convex.
Proof. First, note that the feasible region of an LP is the intersection of several half spaces (each one is determined by an inequality constraint) and hyperplanes (each one is determined by an equality constraint). It is trivial to show that half spaces and hyperplanes are always convex. It then remains to show that the intersection of convex sets are convex, which is left as an exercise.
I (^) Consider a general NLP
min x∈Rn^ f (x)
s.t. gi(x) ≤ bi ∀i = 1, ..., m.
I (^) If the feasible region F = {x ∈ Rn|gi(x) ≤ bi∀i = 1, ..., m} is convex and f is convex over F , a local minimum is a global minimum. I (^) In this case, the NLP is called a convex program (CP).
An NLP is a CP if its feasible region is convex and its objective function is convex over the feasible region.
I (^) Efficient algorithms exist for solving CPs. I (^) The subject of formulating and solving CPs is Convex Programming.
I (^) When is an NLP a CP?
For an NLP min x∈Rn
f (x)
∣gi(x)^ ≤^ bi∀i^ = 1, ..., m
if f and gis are all convex functions, the NLP is a CP.
Proof. We only need to prove that the feasible region is convex, which is implied if Fi = {x ∈ Rn|gi(x) ≤ bi} is convex for all i. For two points x 1 , x 2 ∈ Fi and an arbitrary λ ∈ [0, 1], we have
gi(λx 1 + (1 − λ)x 2 ) ≤ λgi(x 1 ) + (1 − λ)gi(x 2 ) ≤ λbi + (1 − λ)bi = bi,
which implies that Fi is convex. Repeating this argument for all i completes the proof.
I (^) Motivating examples. I (^) Convex analysis. I (^) Solving single-variate NLPs. I (^) Multi-variate convex analysis.
I (^) Here we discuss how to analytically solve single-variate NLPs. I (^) “Analytically solving a problem” means to express the solution as a function of problem parameters symbolically. I (^) Even though solving problems with only one variable is restrictive, we will see some useful examples in the remaining semester. I (^) We will focus on twice differentiable functions and try to utilize convexity (if possible).
