Introduction to Duality - Nonlinear Programming - Lecture Slides, Slides of Computer Science

These are the Lecture Slides of Nonlinear Programming which includes Convex Cost, Linear Constraints, Duality Theorem, Linear Programming Duality, Quadratic Programming Duality, Linear Inequality, Constrained Problem, Minimize, Feasible etc.Key important points are: Introduction to Duality, Convex Cost, Linear Constraints, Duality Theorem, Linear Programming Duality, Quadratic Programming Duality, Linear Inequality, Constrained Problem, Minimize, Feasible

Typology: Slides

2012/2013

Uploaded on 03/27/2013

ekana
ekana 🇮🇳

4

(44)

370 documents

1 / 8

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
NONLINEAR PROGRAMMING
LECTURE 14: INTRODUCTION TO DUALITY
LECTURE OUTLINE
Convex Cost/Linear Constraints
Duality Theorem
Linear Programming Duality
Quadratic Programming Duality
Linear inequality constrained problem
minimize f(x)
subject to aj
x bj
, j =1,...,r,
where f is convex and continuously differentiable
over n.
Docsity.com
pf3
pf4
pf5
pf8

Partial preview of the text

Download Introduction to Duality - Nonlinear Programming - Lecture Slides and more Slides Computer Science in PDF only on Docsity!

NONLINEAR PROGRAMMING

LECTURE 14: INTRODUCTION TO DUALITY

LECTURE OUTLINE

• Convex Cost/Linear Constraints

• Duality Theorem

• Linear Programming Duality

• Quadratic Programming Duality

Linear inequality constrained problem

minimize f (x)

subject to a

j ′ x ≤ bj , j = 1,... , r,

where f is convex and continuously differentiable

over 

n

LAGRANGE MULTIPLIER RESULT

Let J ⊂ { 1 ,... , r}. Then x

is a global min if and

only if x

is feasible and there exist μ

j

≥ 0 , j ∈ J,

such that μ

j

= 0 for all j ∈ J /∈ A(x

), and

∗ ∗ ′ x = arg min f (x) + μj (aj x − bj ). a′^ x≤bj j j / † j∈J ∈J†

Proof: Assume x

is global min. Then there exist

∗ ∗ μ j

≥ 0 , such that μ

j (a ′ ∗

− bj ) = 0 for all j and

j x r (^) ∗ ∇f (x ∗ ) + j= μ j

aj = 0, implying

r ∗ ∗ ′ x = arg min f (x) + μj (aj x − bj ). x∈n† j= ∗

Since μ

j (a ′ j x ∗

− bj ) = 0 for all j,

r ∗ ∗ ′ f (x ) = min f (x) + μj (aj x − bj ). x∈n† j= ∗

Since μ

j (a ′ j

x − bj ≤ 0 ,

j

x − bj ) ≤ 0 if a

′ r ∗ ∗ ′ f (x ) ≤ min f (x) + μj (aj x − bj ) a′^ x≤bj j j / † j= ∈J† ∗ ′ ≤ min f (x) + μj (aj x − bj ). a′^ x≤bj j j / † j∈J ∈J†

THE DUAL PROBLEM

• Consider the problem

min f (x) x∈X, a′ j x≤bj , j=1,...,r

where f is convex and cont. differentiable over 

n

and X is polyhedral.

• Define the dual function q : 

r  → [−∞, ∞) r j= q(μ) = inf f (x) + μj (aj x − bj ) x∈X L(x, μ) = inf x∈X

and the dual problem

max μ≥ 0 q(μ).

• If X is bounded, the dual function takes real

values. In general, q(μ) can take the value −∞.

The “effective” constraint set of the dual is

Q = μ | μ ≥ 0 , q(μ) > −∞.

DUALITY THEOREM

(a) If the primal problem has an optimal solution,

the dual problem also has an optimal solution and

the optimal values are equal.

(b) x

is primal-optimal and μ

is dual-optimal if

and only if x

is primal-feasible, μ

≥ 0 , and

∗ ∗ ∗ ∗ f (x ) = L(x , μ ) = min x∈X L(x, μ ).

Proof: (a) Let x

be a primal optimal solution. For

all primal feasible x, and all μ ≥ 0 , we have μ

′ j x − j (a ′

bj ) ≤ 0 for all j, so

r j= f (x) + μj (a j q(μ) ≤ inf x − bj ) x∈X, a′ j x≤bj , j=1,...,r ∗ ≤ inf ). x∈X, a′ j x≤bj , j=1,...,r f (x) = f (x (*) ∗

By L-Mult. Th., there exists μ

≥ 0 such that μ

j (a ′ j x ∗ −

bj ) = 0 for all j, and x

∗ = arg minx∈X L(x, μ ∗

), so

∗ ) + r j= q(μ μ ∗ ) = L(x ∗ ∗ ) = f (x ∗ ′ j (aj x ∗ − bj ) = f (x ∗ , μ ).

THE DUAL OF A LINEAR PROGRAM

• Consider the linear program

minimize c

′ x

subject to ei

′ x = di, i = 1,... , m, x ≥ 0

• Dual function

n m m q(λ) = inf cj − λieij xj + λidi. x≥ 0 j=1 i=1 i= m

• If cj −

i=

λieij ≥ 0 for all j, the infimum is

m

attained for x = 0, and q(λ) = λidi. If cj −

i= m

λieij < 0 for some j, the expression in braces

i=

can be arbitrarily small by taking xj suff. large, so

q(λ) = −∞. Thus, the dual is

m

maximize λidi

i= m

subject to λieij ≤ cj , j = 1,... , n.

i=

THE DUAL OF A QUADRATIC PROGRAM

• Consider the quadratic program

1

minimize

2 x ′ Qx + c ′ x

subject to Ax ≤ b,

where Q is a given n×n positive definite symmetric

matrix, A is a given r × n matrix, and b ∈ 

r

and

c ∈  n

are given vectors.

• Dual function:

q(μ) = inf x∈n 1 2 x ′ Qx + c ′ x + μ ′ (Ax − b).

The infimum is attained for x = −Q

− 1 (c + A ′

μ), and,

after substitution and calculation,

2 μ ′ AQ − 1 A ′ μ − μ ′ (b + AQ − 1 c) − 1 q(μ) = − 1 2 c ′ Q − 1 c.

• The dual problem, after a sign change, is

minimize 1

2 μ ′ P μ + t ′ μ

subject to μ ≥ 0 ,

where P = AQ

− 1 A ′

and t = b + AQ

− 1 c.