Linear Programming: Finding Lower and Upper Bounds using Duality Theory - Prof. J. S. Prov, Study notes of Statistics

Download Linear Programming: Finding Lower and Upper Bounds using Duality Theory - Prof. J. S. Prov and more Study notes Statistics in PDF only on Docsity!

Lecture 6:

Duality Theory

Reading: Chapter 4

Bounds on the Objective Function Consider the canonical max LP:

(P )

max z = c 1 x 1 + c 2 x 2 +... + cnxn a 11 x 1 + a 12 x 2 +... + a 1 nxn ≤ b 1 a 21 x 1 + a 22 x 2 +... + a 2 nxn ≤ b 2 ... ... am 1 x 1 + am 2 x 2 +... + amnxn ≤ bm x 1 ≥ 0 x 2 ≥ 0... xn ≥ 0

or

max z = cx Ax ≤ b x ≥ 0 Let z∗ denote the (as yet unknown) optimal solution objec- tive value for this LP.

Question: How can we find lower and upper bounds on z∗?

Example Consider Woody’s LP: max z = 35x 1 + 60x 2 + 75x 3 8 x 1 + 12x 2 + 16x 3 ≤ 120 15 x 2 + 20x 3 ≤ 60 3 x 1 + 6 x 2 + 9 x 3 ≤ 48 x 1 ≥ 0 x 2 ≥ 0 x 3 ≥ 0

Lower bound: the feasible solution x 1 = 11, x 2 = x 3 = 1 has objective function value 520, which is then a lower bound on z∗. Upper bound: We can construct the following surrogate constraint

15 / 4 × (8x 1 + 12x 2 + 16x 3 ≤ 120) ⇒ 30 x 1 + 45x 2 + 60 x 3 ≤ 450 1 / 5 × (15x 2 + 20x 3 ≤ 60) ⇒ 3 x 2 + 4 x 3 ≤ 12 2 × (3x 1 + 6x 2 + 9x 3 ≤ 48) ⇒ 6 x 1 + 12x 2 + 18 x 3 ≤ 96 1 × (−x 1 ≤ 0) ⇒ −x 1 7 × (−x 3 ≤ 0) ⇒ − 7 x 3 ≤ 0 Adding rows: 35 x 1 + 60x 2 + 75 x 3 ≤ 558 The number 558 − 520 gives you a rough idea of how close you are to finding the true value z∗.

The Primal and Dual LPs

greatest lower bound (primal LP): find the feasi- ble solution with the largest objective function value.

least upper bound (dual LP): construct the implied constraint for the objective function having the small- est bound on z.

Example

y 1 × (8x 1 + 12x 2 + 16x 3 ≤ 120) ⇒ (8y 1 )x 1 + (12y 1 )x 2 + (16y 1 )x 3 ≤ 120 y 1 y 2 × (15x 2 + 20x 3 ≤ 60) ⇒ (15y 2 )x 2 + (20y 2 )x 3 ≤ 60 y 2 y 3 × (3x 1 + 6x 2 + 9x 3 ≤ 48) ⇒ (3y 3 )x 1 + (6y 3 )x 2 + (9y 3 )x 3 ≤ 48 y 3 u 1 × (−x 1 ≤ 0) ⇒ −u 1 x 1 ≤ 0 u 2 × (−x 2 ≤ 0) ⇒ −u 2 x 2 ≤ 0 u 3 × (−x 3 ≤ 0) ⇒ −u 3 x 3 ≤ 0

Adding these, we get the surrogate constraint:

(8y 1 + 3y 3 − u 1 )x 1 + (12y 1 + 15y 2 + 6y 3 − u 2 )x 2

• (16y 1 +20y 2 +9y 3 −u 3 )x 3 ≤ 120 y 1 +60y 2 +48y 3

y and u satisfy (B1), (B2) if and only if 8 y 1 +3y 3 −u 1 = 35, 12 y 1 +15y 2 +6y 3 −u 2 = 60, and 16y 1 +20y 2 +9y 3 −u 3 = 75, or equivalently if the y’s satisfy

8 y 1 + 3y 3 ≥ 35 12 y 1 + 15y 2 + 6y 3 ≥ 60 16 y 1 + 20y 2 + 9y 3 ≥ 75 y 1 ≥ 0 , y 2 ≥ 0 , y 3 ≥ 0

and we want to choose y so as to minimize

120 y 1 + 60y 2 + 48y 3

Dual Linear Programs

The dual to a canonical max LP

max z = c 1 x 1 + c 2 x 2 +... + cnxn a 11 x 1 + a 12 x 2 +... + a 1 nxn ≤ b 1 a 21 x 1 + a 22 x 2 +... + a 2 nxn ≤ b 2 (P ) ... ... ... ... am 1 x 1 + am 2 x 2 +... + amnxn ≤ bm x 1 ≥ 0 x 2 ≥ 0... xn ≥ 0

is the canonical min LP

min w = b 1 y 1 + b 2 y 2 +... + bmym a 11 y 1 + a 21 y 2 +... + am 1 ym ≥ c 1 a 12 y 1 + a 22 y 2 +... + am 2 ym ≥ c 2 (D) ... ... ... ... a 1 ny 1 + a 2 ny 2 +... + amnym ≥ cn y 1 ≥ 0 y 2 ≥ 0... ym ≥ 0

Weak Duality Theorem

Theorem 6.1 Let x¯ = (¯x 1 ,... , x¯n) be a feasible so- lution to (P ) with objective function value z¯, and let y ¯ = (¯y 1 ,... , y¯m) be a feasible solution to (D) with objec- tive function value w¯. Then z¯ ≤ w¯, and if z¯ = ¯w then x ¯ and y¯ are both optimal to their respective LPs.

Proof: Since we know that ¯z provides an lower bound on z∗ and ¯w provides a upper bound on that same num- ber, then clearly ¯z ≤ w¯. Further, the only way that ¯z = ¯w is when they are both equal to z∗, that is, ¯x and ¯y are optimal to (P ) and (D), respectively.

Question: When can we actually get an ¯x and ¯y such that the upper and lower bounds given above are the same?

A Fundamental Insight Lemma 6.2 Suppose we solve (P ) using the Simplex Method, starting with tableau basis z x s rhs xB ¯ 0 A I b z 1 −c 0 0

where s is the vector of slack variables. A basic tableau for (P ) corresponding to basis (xB, xN ) will look like:

basis z x s rhs xB 0 A¯ S¯ ¯b z 1 −c + ¯yA y¯ z¯ 0

Then

• The associated BFS x¯ = (¯xB, x¯N ) = (¯b, 0) is a feasible solution for (P ) whenever ¯b ≥ 0.
• The primal objective function value for x¯ is z¯ 0 = yb¯.
• If we let y¯ be the dual solution value, then the slack variables values for the dual problem will be u ¯ = −c + ¯yA

Therefore,

• If the objective function row is nonnegative, then y¯ is a feasible solution for (D).
• The dual objective function value for y¯ is z¯ 0.

Example

The dual for Woody’s LP is

(D)

min w = 120y 1 + 60y 2 + 48y 3 8 y 1 + 3 y 3 ≥ 35 12 y 1 + 15y 2 + 6 y 3 ≥ 60 16 y 1 + 20y 2 + 9 y 3 ≥ 75 y 1 ≥ 0 , y 2 ≥ 0 , y 3 ≥ 0

Now the optimal simplex tableau for Woody’s problem is

basis z x 1 x 2 x 3 s 1 s 2 s 3 rhs s 2 0 0 0 − 10 15 / 4 1 − 10 30 x 2 0 0 1 2 − 1 / 4 0 2 / 3 2 x 1 0 1 0 − 1 1 / 2 0 − 1 12 z 1 0 0 10 5 / 2 0 5 540

so if we set

(y 1 , y 2 , y 3 ) = (5/ 2 , 0 , 5)

then it can be checked that the slack variables for (D) are

(u 1 , u 2 , u 3 ) = (0, 0 , 10)

Thus y = (5/ 2 , 0 , 5) is feasible to (D) and has objective function value w = 540, the same as that for the optimal primal solution.

The Symmetry of the Primal and Dual LPs

Consider the two dual LPs (in matrix form):

(P )

max z = cx Ax ≤ b x ≥ 0

(D)

min w = yb yA ≥ c y ≥ 0

Theorem 6.3 The dual of (D) is (P ).

Proof: First put (D) in canonical max form, by negating the objective (to put it into max form) and all of the con- straints (to put them into ≤ form):

(D′)

max −w = −yb −yA ≤ −c y ≥ 0

or

max ¯z = ¯cx¯ A^ ¯x¯ ≤ ¯b x ¯ ≥ 0

where ¯x = yT^ , ¯c = −bT^ , A¯ = − A¯T^ , and ¯b = −¯cT^.

Now take the dual of (D′):

min ¯w = ¯y¯b y ¯ A¯ ≥ ¯c y ¯ ≥ 0

or

min −cx −Ax ≥ −b x ≥ 0

which is exactly (P ).

Duality for General LPs

Let’s consider the dual program for a general LP whose objective is to be maximized. The Fundamental Duality Equation

w = z +

∑n j=

ujxj +

∑m i=

yisi

applies to a general LP by setting

si = bi − ∑n j=

aijxj

uj = cj − ∑n i=

ajiyi

Now the nature of the xj’s, yi’s, si’s and uj’s will depend upon what the nature of the variables and constraints are for P. What we know, however, is that w comprises an upper bound on z if

∑^ n j=

ujxj + ∑m i=

yisi ≥ 0 ,

or more precisely,

ujxj ≥ 0 , j = 1,... , n yisi ≥ 0 , i = 1,... , m.

The Dual of the General LP

The dual of a general LP will always have as variables y 1 ,... , ym, one for each constraint, objective function value

w = b 1 y 1 +... + bmym

and as constraints

aj 1 yi +... + ajmym

    

    

cj, j = 1,... , n.

In terms of the uj’s, this is equivalent to

uj

    

Now let’s see what this means in terms of (F DE) implying w ≥ z. For the variables:

• If xj ≥ 0, then in order that ujxj ≥ 0 we must have uj ≥ 0, that is, ∑m i=

yiaij ≥ cj.

• If xj ≤ 0, then in order that ujxj ≥ 0 we must have uj ≤ 0, that is, ∑m i=

yiaij ≤ cj.

• If xj is unrestricted, in order that ujxj ≥ 0 we must have uj = 0, that is, ∑^ m i=

yiaij = cj.

Summary of Dual Construction for a General LP Maximization Problem

Minimization Problem

objective:

max

z^

= c^1

x^1

c^2

x^2

...

cn

xn

objective:

min

w

=

b^1

y^1

b^2

y^2

...

bm

ym

constraint:

ai

x 1

• 1 a

xi 2

• 2

...

ain

xn

 

≤ bi ≥ bi = bi

variable:

yi

 

≥ 0 ≤ 0 unrestricted

variable:

xj

 

≥

0 ≤

0 unrestricted

constraint:

a^1

yj

• 1 a

y 2 j

• 2

...

a mj

y m

 

≥ cj ≤ cj = cj

Corollary:

The dual of the dual is the primal.

Proof:

The relationship between the constraints/dual variables and variables/dual constraints

goes both ways in the above diagram.

Example: The Dual to the Standard Form LP

The dual to the standard equality LP

max z = c 1 x 1 + c 2 x 2 +... + cnxn a 11 x 1 + a 12 x 2 +... + a 1 nxn = b 1 a 21 x 1 + a 22 x 2 +... + a 2 nxn = b 2 (P S) ... ... ... am 1 x 1 + am 2 x 2 +... + amnxn = bm x 1 ≥ 0 x 2 ≥ 0... xn ≥ 0.

is the inequality form min LP

min w = b 1 y 1 + b 2 y 2 +... + bmym a 11 y 1 + a 21 y 2 +... + am 1 ym ≥ c 1 a 12 y 1 + a 22 y 2 +... + am 2 ym ≥ c 2 (DS) ... ... ... ... a 1 ny 1 + a 2 ny 2 +... + amnym ≥ cn (yi unrestricted)

Now an optimal tableau for (P S) will look like

basis z x rhs xB 0 S∗A S∗b z 1 −c + y∗A y∗b

where S∗^ is the inverse of the optimal basis matrix B∗, y∗^ = cB∗S∗^ (no sign restriction), and −c + y∗A ≥ 0. It follows that y∗^ is optimal to (DS).