Inequality Constraints - Nonlinear Programming - Lecture Slides, Slides of Computer Science

These are the Lecture Slides of Nonlinear Programming which includes Convex Cost, Linear Constraints, Duality Theorem, Linear Programming Duality, Quadratic Programming Duality, Linear Inequality, Constrained Problem, Minimize, Feasible etc.Key important points are: Inequality Constraints, Necessary Conditions, Sufficiency Conditions, Linear Constraints, Minimize, Continuously Differentiable, Activeinequality, Treated As Equations, Inactive Constraints, Lagrange Multipliers

Typology: Slides

2012/2013

Uploaded on 03/27/2013

ekana
ekana 🇮🇳

4

(44)

370 documents

1 / 8

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
NONLINEAR PROGRAMMING
LECTURE 13: INEQUALITY CONSTRAINTS
LECTURE OUTLINE
Inequality Constrained Problems
Necessary Conditions
Sufciency Conditions
Linear Constraints
Inequality constrained problem
minimize f(x)
subject to h(x)=0, g(x) 0
where f : n →, h : n →
m, g : n →
r are
continuously differentiable. Here
h =(h1, ..., hm), g =(g1, ..., gr
).
Docsity.com
pf3
pf4
pf5
pf8

Partial preview of the text

Download Inequality Constraints - Nonlinear Programming - Lecture Slides and more Slides Computer Science in PDF only on Docsity!

NONLINEAR PROGRAMMING

LECTURE 13: INEQUALITY CONSTRAINTS

LECTURE OUTLINE

  • Inequality Constrained Problems
  • Necessary Conditions
  • Sufficiency Conditions
  • Linear Constraints

Inequality constrained problem

minimize f (x)

subject to h(x) = 0, g(x) ≤ 0

where f : n^ → , h : n^ → m, g : n^ → r^ are

continuously differentiable. Here

h = (h 1 , ..., hm), g = (g 1 , ..., gr ).

TREATING INEQUALITIES AS EQUATIONS

  • Consider the set of active inequality constraints

A(x) = j | gj (x) = 0.

  • If x∗^ is a local minimum:

− The active inequality constraints at x∗^ can be

treated as equations

− The inactive constraints at x∗^ don’t matter

  • Assuming regularity of x∗^ and assigning zero

Lagrange multipliers to inactive constraints,

m

i=

r

j=

∇f (x∗ ) + λ∗ i ∇hi(x μ∗ j ∇gj (x∗^ ) = 0,

∗ j = 0,^ ∀^ j /∈^ A(x μ ∗^ ).

  • Extra property: μ^ ∗

j ≥^0 for all^ j.

Intuitive reason: Relax jth constraint, gj (x) ≤ uj ,

∗ μ j = −(∆cost due to uj )/uj

PROOF OF KUHN-TUCKER CONDITIONS

Use equality-constraints result to obtain all the

conditions except for μ^ ∗

j ≥^0 for^ j^ ∈^ A(x

∗). Intro-

duce the penalty functions

g^ + j (x) = max^0 , gj^ (x)^ ,^ j^ = 1,... , r,

and for k = 1, 2 ,.. ., let xk^ minimize

r

k k )^2

f (x) +

||h(x)||^2 +

gj^ +(x) +

||x − x∗||^2

j=

over a closed sphere of x such that f (x∗) ≤ f (x).

Using the same argument as for equality con-

straints,

λ^ ∗ i =^ lim^ khi(x

k (^) ), i = 1,... , m, k→∞

μ^ ∗ j =^ lim^ kgj

+(xk (^) ), j = 1,... , r. k→∞

Since g^ ∗

j

+(xk ) ≥ 0 , we obtain μ

j ≥^0 for all^ j.

LINEAR CONSTRAINTS

• Consider the problem mina′

j†x≤bj†, j=1,...,r^

f (x).

• Remarkable property: No need for regularity.

• Proposition: If x∗^ is a local minimum, there exist

μ^ ∗^ ∗^ ∗

1 ,... , μr^ with^ μj ≥^0 ,^ j^ = 1,... , r, such that

r ∇f (x^ ∗ ) + μ∗^ ∗^ ∗ j aj^ = 0,^ μj = 0,^ ∀^ j /∈^ A(x^ ). j=

• Proof uses Farkas Lemma: Consider the cones

C and C⊥

a 2

Σ μjaj,^ μj ≥^0 }

C⊥^ = {y | aj'y ≤ 0, j=1,...,r}

a 1

0 C =^ {x^ |^ x =^ j=

r

x ∈ C iff x′^ y ≤ 0 , ∀ y ∈ C⊥^.

PROOF OF LAGRANGE MULTIPLIER RESULT

a 2

r

Σj=1 μjaj,^ μj ≥^0 }

a 1

Cone generated by a (^) j , j ∈ A(x*^ )

− ∇f(x^ ) x

Constraint set

C = {x | x =

{x | aj'x ≤ bj, j = 1,...,r}

The local min x^ ∗ of the original problem is also a local min for the problem mina′ j†x≤bj†, j∈A(x∗^ )^

f (x). Hence

∇f (x^ ∗ )′(x − x∗ ) ≥ 0 , ∀ x with a ∗ j

′ (^) x ≤ bj , j ∈ A(x ).

Since a constraint a′ j x ≤ bj , j ∈ A(x∗) can also be ex-

pressed as a′ j (x − x∗) ≤ 0, we have

∇f (x^ ∗ )′^ y ≥ 0 , ∀ y with a ∗ j

′ (^) y ≤ 0 , j ∈ A(x ).

From Farkas’ lemma, −∇f (x∗) has the form

μ^ ∗^ ∗ j aj^ ,^ for some^ μj ≥^ 0,^ j^ ∈^ A(x

j∈A(x∗^ )

Let μ^ ∗ j = 0 for^ j /∈^ A(x

CONVEX COST AND LINEAR CONSTRAINTS

Let f : n^ →  be convex and cont. differentiable,

and let J be a subset of the index set { 1 ,... , r}.

Then x∗^ is a global minimum for the problem

minimize f (x)

subject to aj^ ′^ x ≤ bj , j = 1,... , r,

if and only if x∗^ is feasible and there exist scalars

μ j , j ∈ J, such that

∗ μ j ≥ 0 , j ∈ J,

j = 0,^ ∀^ j^ ∈^ J^ with^ j /∈^ A(x

μ ∗^ ),

∗ (^) = arg min a′j†^ x≤bj j /∈J†

f (x) + j∈J

x μ ∗ j (a′j x − bj ).

  • Proof is immediate if J = { 1 ,... , r}.
  • Example: Simplex Constraint.