Download Optimization Problems on Nonlinear Programming | IOE 611 and more Study notes Systems Engineering in PDF only on Docsity!
Convex optimization problems
! (^) optimization problem in standard form
! convex optimization problems
! linear optimization
! quadratic optimization
! geometric programming
! (^) quasiconvex optimization
! (^) generalized inequality constraints
! semidefinite programming
! vector optimization
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Optimization problem in standard form
minimize f 0 (x)
subject to f i (x) ≤ 0 , i = 1,... , m
h i
(x) = 0, i = 1,... , p
! x ∈ R
n is the optimization variable
! (^) f 0
: R
n → R is the objective or cost function
! f i
: R
n → R, i = 1,... , m, are the inequality constraint
functions
! h i
: R
n → R are the equality constraint functions
optimal value:
p
! = inf{f 0 (x) | f i (x) ≤ 0 , i = 1,... , m, h i (x) = 0, i = 1,... , p}
! p
! = ∞ if problem is infeasible (no x satisfies the constraints)
! p
! = −∞ if problem is unbounded below
Optimal and locally optimal points
! x is feasible if x ∈ dom f 0 and it satisfies the constraints
! a feasible x is optimal if f 0 (x) = p
! ; X opt is the set of optimal
points
! x is locally optimal if there is an R > 0 such that x is
optimal for
minimize (over z) f 0 (z)
subject to f i (z) ≤ 0 , i = 1,... , m,
h i (z) = 0, i = 1,... , p
‖z − x‖ 2
≤ R
examples (with n = 1, m = p = 0)
! f 0 (x) = 1/x, dom f 0
= R
++ : p
! = 0, no optimal point
! (^) f 0 (x) = − log x, dom f 0
= R
++ : p
! = −∞
! f 0 (x) = x log x, dom f 0
= R
++ : p
! = − 1 /e, x = 1/e is
optimal
! f 0 (x) = x
3 − 3 x, p
! = −∞, local optimum at x = 1
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Implicit constraints
The standard form optimization problem has an implicit
constraint
x ∈ D =
m ⋂
i=
dom f i
p ⋂
i=
dom h i
! (^) we call D the domain of the problem
! (^) the constraints f i
(x) ≤ 0, h i
(x) = 0 are the explicit constraints
! a problem is unconstrained if it has no explicit constraints
(m = p = 0)
example:
minimize f 0 (x) = −
k
i=
log(b i − a
T
i
x)
is an unconstrained problem with implicit constraints
a
T
i
x < b i , i = 1,... , k
Standard form convex optimization problem
Example
minimize f 0 (x) = x
2
1
2
2
subject to f 1 (x) = x 1 /(1 + x
2
2
h 1 (x) = (x 1
2 = 0
! (^) f 0
is convex; feasible set {(x 1
, x 2
) | x 1
= −x 2
≤ 0 } is convex
! not a convex problem (according to our definition): f 1 is not
convex, h 1 is not affine
! (^) equivalent (but not identical) to the convex problem
minimize x
2
1
2
2
subject to x 1
x 1
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Local and global optima
Any locally optimal point of a convex problem is (globally) optimal
proof: suppose x is locally optimal and y is feasible with
f 0 (y ) < f 0 (x).
“x locally optimal” means there is an R > 0 such that
z feasible, ‖z − x‖ 2 ≤ R =⇒ f 0 (z) ≥ f 0 (x).
Consider z = θy + (1 − θ)x with θ = R/(2‖y − x‖ 2
! (^) ‖y − x‖ 2
R, so 0 < θ < 1 / 2
! (^) z is a convex combination of two feasible points, hence also
feasible
! ‖z − x‖ 2 = R/2 and
f 0 (z) ≤ θf 0 (x) + (1 − θ)f 0 (y ) < f 0 (x),
which contradicts our assumption that x is locally optimal
Optimality criterion for differentiable f
0
x is optimal if and only if it is feasible and
∇f 0
(x)
T (y − x) ≥ 0 for all feasible y
Optimality criterion for differentiable f
0
x is optimal if and only if it is feasible and
∇f 0 (x)
T (y − x) ≥ 0 for all feasible y
PSfrag replacements
−∇f 0 (x)
X
x
if nonzero, ∇f 0 (x) defines a supporting hyperplane to feasible set X at x
Convex optimization problems 4 – 9
if nonzero, ∇f 0 (x) defines a supporting hyperplane to feasible set
X at x
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Optimality criterion: special cases
! unconstrained problem: x is optimal if and only if
x ∈ dom f 0
, ∇f 0
(x) = 0
! equality constrained problem
minimize f 0 (x) subject to Ax = b
x is optimal if and only if there exists a ν such that
x ∈ dom f 0 , Ax = b, ∇f 0 (x) + A
T ν = 0
! minimization over nonnegative orthant
minimize f 0 (x) subject to x + 0
x is optimal if and only if
x ∈ dom f 0 , x + 0 ,
∇f 0 (x) i ≥ 0 x i
∇f 0
(x) i
= 0 x i
Equivalent convex problems
! epigraph form: standard form convex problem is equivalent
to
minimize (over x, t) t
subject to f 0 (x) − t ≤ 0
f i (x) ≤ 0 , i = 1,... , m
Ax = b
! minimizing over some variables
minimize f 0 (x 1 , x 2
subject to f i (x 1 ) ≤ 0 , i = 1,... , m
is equivalent to
minimize
f 0 (x 1
subject to f i (x 1 ) ≤ 0 , i = 1,... , m
where
f 0 (x 1 ) = inf x 2 f 0 (x 1 , x 2
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Linear program (LP)
minimize c
T x + d
subject to Gx - h
Ax = b
! convex problem with affine objective and constraint functions
! (^) feasible set is a polyhedron
Linear program (LP)
minimize c
T
x + d
subject to Gx! h
Ax = b
- convex problem with affine objective and constraint functions
- feasible set is a polyhedron
PSfrag replacements
P
x
!
−c
Convex optimization problems 4 – 17
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Examples
diet problem: choose quantities x 1 ,... , x n of n foods
! (^) one unit of food j costs c j , contains amount a ij of nutrient i
! (^) healthy diet requires nutrient i in quantity at least b i
to find cheapest healthy diet,
minimize c
T x
subject to Ax + b, x + 0
piecewise-linear minimization
minimize max i=1,...,m (a
T
i
x + b i
equivalent to an LP
minimize t
subject to a
T
i
x + b i ≤ t, i = 1,... , m
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Chebyshev center of a polyhedron
Chebyshev center of
P = {x | a
T
i
x ≤ b i , i = 1,... , m}
is center of largest inscribed ball
B = {x c
Chebyshev center of a polyhedron
Chebyshev center of
P = {x | a
T
i
x ≤ b i , i = 1 ,... , m}
is center of largest inscribed ball
B = {x c
PSfrag replacements
x cheb x cheb
T
i
x ≤ b i for all x ∈ B if and only if
sup{a
T
i
(x c
T
i
x c
- hence, x c , r can be determined by solving the LP
maximize r
subject to a
T
i
x c
- r‖a i ‖ 2 ≤ b i , i = 1 ,... , m
Convex optimization problems 4 – 19
! a
T
i
x ≤ b i for all x ∈ B if and only if
sup{a
T
i
(x c
T
i
x c
2 ≤ b i
! hence, x c , r can be determined by solving the LP
maximize xc ,r r
subject to a
T
i
x c
2 ≤ b i
, i = 1,... , m
Quadratically constrained quadratic program (QCQP)
minimize (1/2)x
T P 0 x + q
T
0
x + r 0
subject to (1/2)x
T P i x + q
T
i
x + r i ≤ 0 , i = 1,... , m
Ax = b
! P i
∈ S
n
; objective and constraints are convex quadratic
! (^) if P 1
,... , P
m
∈ S
n
++
, feasible region is intersection of m
ellipsoids and an affine set
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Second-order cone programming
minimize f
T x
subject to ‖A i x + b i
2 ≤ c
T
i
x + d i , i = 1,... , m
Fx = g
(A
i
∈ R
n i ×n , F ∈ R
p×n )
! inequalities are called second-order cone (SOC) constraints:
(A
i
x + b i
, c
T
i
x + d i
) ∈ second-order cone in R
n i
! for n i = 0, reduces to an LP; if c i = 0, reduces to a QCQP
! more general than QCQP and LP
Robust linear programming
the parameters in optimization problems are often uncertain, e.g.,
in an LP
minimize c
T x
subject to a
T
i
x ≤ b i
, i = 1,... , m,
there can be uncertainty in c, a i
, b i
two common approaches to handling uncertainty (in a i , for
simplicity)
! (^) deterministic model: constraints must hold for all a i
∈ E
i
minimize c
T x
subject to a
T
i
x ≤ b i for all a i
∈ E
i , i = 1,... , m,
! (^) stochastic model: a i is random variable; constraints must hold
with probability η
minimize c
T x
subject to prob(a
T
i
x ≤ b i
) ≥ η, i = 1,... , m
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Deterministic approach via SOCP
! (^) choose an ellipsoid as E i
E
i = {¯a i
+ P
i u | ‖u‖ 2 ≤ 1 } (¯a i
∈ R
n , P i
∈ R
n×n )
center is ¯a i , semi-axes determined by singular values/vectors
of P i
! robust LP
minimize c
T x
subject to a
T
i
x ≤ b i
∀a i
∈ E
i
, i = 1,... , m
is equivalent to the SOCP
minimize c
T x
subject to ¯a
T
i
x + ‖P
T
i
x‖ 2 ≤ b i , i = 1,... , m
(follows from sup ‖u‖ 2 ≤ 1
(¯a i
+ P
i u)
T x = ¯a
T
i
x + ‖P
T
i
x‖ 2
Geometric program in convex form
change variables to y i = log x i , and take logarithm of cost,
constraints
! monomial f (x) = cx
a 1
1
· · · x
an
n
transforms to
log f (e
y 1 ,... , e
yn ) = a
T y + b (b = log c)
! (^) posynomial f (x) =
K
k=
c k
x
a 1 k
1
x
a 2 k
2
· · · x
a nk n transforms to
log f (e
y 1 ,... , e
y n ) = log
K ∑
k=
e
a
T
k
y +b k
(b k = log c k
! (^) geometric program transforms to convex problem
minimize log
K
k=
exp(a
T
0 k
y + b 0 k
subject to log
K
k=
exp(a
T
ik
y + b ik
≤ 0 , i = 1,... , m
Gy + d = 0
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Design of cantilever beam
Design of cantilever beam
PSfrag replacements
F
segment 4 segment 3 segment 2 segment 1
• N segments with unit lengths, rectangular cross-sections of size w
i
× h
i
• given vertical force F applied at the right end
design problem
minimize total weight
subject to upper & lower bounds on w
i
, h
i
upper bound & lower bounds on aspect ratios h
i
/w
i
upper bound on stress in each segment
upper bound on vertical deflection at the end of the beam
! N segments with unit lengths, rectangular cross-sections of
size w i × h i
! (^) given vertical force F applied at the right end
design problem
minimize total weight
subject to upper & lower bounds on w i , h i
upper bound & lower bounds on aspect ratios h i /w i
upper bound on stress in each segment
upper bound on vertical deflection at the end of the beam
variables: w i , h i for i = 1,... , N
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Objective and constraint functions
! total weight w 1 h 1
- · · · + w N h N is posynomial
! aspect ratio h i /w i and inverse aspect ratio w i /h i are
monomials
! (^) maximum stress in segment i is given by 6iF /(w i h
2
i
), a
monomial
! the vertical deflection y i and slope v i of central axis at the
right end of segment i are defined recursively as
v i
= 12(i − 1 /2)
F
Ew i
h
3
i
y i = 6(i − 1 /3)
F
Ew i h
3
i
for i = N, N − 1 ,... , 1, with v N+
= y N+
= 0 (E is Young’s
modulus)
v i and y i are posynomial functions of w , h
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Formulation as a GP
minimize w 1 h 1
subject to w
− 1
max
w i ≤ 1 , w min w
− 1
i
≤ 1 , i = 1,... , N
h
− 1
max
h i ≤ 1 , h min h
− 1
i
≤ 1 , i = 1,... , N
S
− 1
max
w
− 1
i
h i
≤ 1 , S
min w i h
− 1
i
≤ 1 , i = 1,... , N
6 iF σ
− 1
max
w
− 1
i
h
− 2
i
≤ 1 , i = 1,... , N
y
− 1
max
y 1
note
! (^) we write w min
≤ w i
≤ w max
and h min
≤ h i
≤ h max
w min /w i ≤ 1 , w i /w max ≤ 1 , h min /h i ≤ 1 , h i /h max
! we write S min ≤ h i /w i
≤ S
max as
S
min w i /h i ≤ 1 , h i /(w i
S
max
! (^) The number of monomials appearing in y 1
grows
approximately as N
2 .
Examples
!
|x| is quasiconvex on R
! ceil(x) = inf{z ∈ Z | z ≥ x} is quasilinear
! log x is quasilinear on R ++
! f (x 1 , x 2 ) = x 1 x 2 is quasiconcave on R
2
++
! (^) linear-fractional function
f (x) =
a
T x + b
c
T x + d
, dom f = {x | c
T x + d > 0 }
is quasilinear
! distance ratio
f (x) =
‖x − a‖ 2
‖x − b‖ 2
, dom f = {x | ‖x − a‖ 2 ≤ ‖x − b‖ 2
is quasiconvex
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Internal rate of return
! cash flow x = (x 0 ,... , x n ); x i is payment in period i (to us if
x i
! we assume x 0 < 0 and x 0
! present value of cash flow x, for interest rate r :
PV(x, r ) =
n ∑
i=
(1 + r )
−i x i
! (^) internal rate of return is smallest interest rate for which
PV(x, r ) = 0:
IRR(x) = inf{r ≥ 0 | PV(x, r ) = 0}
IRR is quasiconcave: superlevel set is intersection of halfspaces
IRR(x) ≥ R ⇐⇒
n ∑
i=
(1 + r )
−i x i ≥ 0 for 0 ≤ r ≤ R
Properties
modified Jensen inequality: for quasiconvex f
0 ≤ θ ≤ 1 =⇒ f (θx + (1 − θ)y ) ≤ max{f (x), f (y )}
first-order condition: differentiable f with cvx domain is
quasiconvex iff
f (y ) ≤ f (x) =⇒ ∇f (x)
T (y − x) ≤ 0
Properties
modified Jensen inequality: for quasiconvex f
0 ≤ θ ≤ 1 =⇒ f (θx + ( 1 − θ)y) ≤ max{f (x), f (y)}
first-order condition: differentiable f with cvx domain is quasiconvex iff
f (y) ≤ f (x) =⇒ ∇f (x)
T (y − x) ≤ 0
PSfrag replacements
x
∇f (x)
sums of quasiconvex functions are not necessarily quasiconvex
Convex functions 3 – 26
sums of quasiconvex functions are not necessarily quasiconvex
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Quasiconvex optimization
minimize f 0 (x)
subject to f i (x) ≤ 0 , i = 1,... , m
Ax = b
with f 0
: R
n → R quasiconvex, f 1 ,... , f m convex
can have locally optimal points that are not (globally) optimal
Quasiconvex optimization
minimize f 0 (x)
subject to f i (x) ≤ 0 , i = 1 ,... , m
Ax = b
with f 0 : R
n → R quasiconvex, f 1 ,... , f m convex
can have locally optimal points that are not (globally) optimal
PSfrag replacements
(x, f 0 (x))
Convex optimization problems 4 – 14
(Generalized) linear-fractional program
minimize f 0 (x)
subject to Gx - h
Ax = b
linear-fractional program
f 0 (x) =
c
T x + d
e
T x + f
, dom f 0 (x) = {x | e
T x + f > 0 }
! a quasiconvex optimization problem; can be solved by
bisection
! (^) also, if feasible, equivalent to the LP (variables y , z)
minimize c
T y + dz
subject to Gy - hz
Ay = bz
e
T y + fz = 1
z ≥ 0
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Generalized linear-fractional program
f 0 (x) = max
i=1,...,r
c
T
i
x + d i
e
T
i
x + f i
, dom f 0 (x) = {x | e
T
i
x+f i
0 , i = 1,... , r }
a quasiconvex optimization problem; can be solved by bisection
example: Von Neumann model of a growing economy
maximize (over x, x
) min i=1,...,n x
i
/x i
subject to x
! (^) x, x
∈ R
n : activity levels of n sectors, in current and next
period
! (Ax) i , (Bx
) i : produced, resp. consumed, amounts of good i
! (^) x
i
/x i : growth rate of sector i
allocate activity to maximize growth rate of slowest growing sector
Convexity of vector-valued functions
f : R
n → R
m is K -convex if dom f is convex and
f (θx + (1 − θ)y ) - K θf (x) + (1 − θ)f (y )
for x, y ∈ dom f , 0 ≤ θ ≤ 1
example f : S
m → S
m , f (X ) = X
2 is S
m
-convex
proof: for fixed z ∈ R
m , z
T X
2 z = ‖Xz‖
2
2
is convex in X , i.e.,
z
T (θX + (1 − θ)Y )
2 z ≤ θz
T X
2 z + (1 − θ)z
T Y
2 z
for X , Y ∈ S
m , 0 ≤ θ ≤ 1
therefore (θX + (1 − θ)Y )
2
2
2
IOE 611: Nonlinear Programming, Winter 2008 4. Convex optimization problems Page 4–
Generalized inequality constraints
convex problem with generalized inequality constraints
minimize f 0 (x)
subject to f i (x) - K i
0 , i = 1,... , m
Ax = b
! f 0
: R
n → R convex; f i
: R
n → R
k i K i -convex w.r.t. proper
cone K i
! (^) same properties as standard convex problem (convex feasible
set, local optimum is global, etc.)
conic form problem: special case with affine objective and
constraints
minimize c
T x
subject to Fx + g - K
Ax = b
extends linear programming (K = R
m
) to nonpolyhedral cones
