Download Dual of a Linear Program, Lecture Notes - Computer Science and more Study notes Linear Programming in PDF only on Docsity!
How to take the Dual of a Linear Program
S´ebastien Lahaie∗
October 1, 2008
The purpose of these notes is to: (1) explain how to recognize a linear program, (2) show how to take the dual of a linear program, and (3) list the fundamental results relating a linear program and its dual. These notes do not provide any proofs and do not explain any of the deep geometric insight behind linear programming duality. The main goal is to explain in detail the mechanical procedure of taking the dual. There are other ways to take the dual than the one described herein, but this is my favorite. I find that it is readily memorized once it has been practiced on a few linear programs. It is perhaps slower and more tedious than other methods you might find in textbooks, but it is easy to memorize, and the intermediate steps produce useful information as a by-product.
1. Formulation
A linear program is a formulation of an optimization problem: a minimization or maximiza- tion of an objective function over some domain. The objective function is linear, and the domain, or feasible set, is defined by linear constraints. Rather than give the generic form of a linear program, here is a specific linear program that we will work with throughout this primer. Once we’ve studied this example, it should be quite clear what a linear program looks like in general.
max x 1 ≥ 0 , x 2 ≤ 0 , x 3
v 1 x 1 + v 2 x 2 + v 3 x 3 (1)
such that a 1 x 1 + x 2 + x 3 ≤ b 1 (2) x 1 + a 2 x 2 = b 2 (3) a 3 x 3 ≥ b 3 (4)
The variables here are x 1 , x 2 , x 3. The remaining terms are constants (e.g., v 1 , a 2 , b 3 ). A linear program consists of an objective function (1), and a set of inequality and equality constraints (2–4). The objective function
f (x) = v 1 x 1 + v 2 x 2 + v 3 x 3
is a linear function. Formally, this means that for two vectors x^1 and x^2 and real-valued constants c 1 and c 2 we have f (c 1 x^1 + c 2 x^2 ) = c 1 f (x^1 ) + c 2 f (x^2 ). You can check that this
∗. Yahoo Research, New York, NY, 10018. [email protected]
indeed holds for our objective. The objective can either be a maximization or minimization. In our case, we have a maximization problem. The left-hand side of each constraint is also a linear function. The right-hand sides are all constants. (You may encounter some linear programs where the right-hand side for some constraint includes some variables, but the constraint can always be rearranged so that the right-hand side is constant.) In a linear program, we cannot have strict inequalities; so x 1 + x 2 < 3 would not be allowed, for example. Constraints that specify whether each variable is non-negative, non-positive, or unre- stricted are special and we usually list those under the max or min. This only holds for these kinds of constraints. A constraint of the form x 1 ≥ 2, for example, is not special, whereas x 1 ≥ 0 is. In our linear program, x 1 must be non-negative, x 2 must be non-positive, and x 3 is unrestricted (it helps to be explicit about this).
2. Primal and Dual
The linear program we start with is typically called the “primal”. To each linear program there is associated another linear program called its “dual”. Deriving the dual from the primal is a purely mechanical procedure. Let’s see how to derive the dual of the linear program of the previous section. There are seven steps. The first two steps put the primal in a “standard form”.
Step 1. If necessary, rewrite the objective as a minimization.
In our case the objective (1) is a maximization, so we rewrite it as
min x 1 ≥ 0 , x 2 ≤ 0 , x 3 −v 1 x 1 − v 2 x 2 − v 3 x 3
A solution that maximizes an objective also minimizes the negative of that objective, so this does not change the set of optimal solutions to the program.
Step 2. Rewrite each inequality constraint as a “less than or equal”, and rearrange each constraint so that the right-hand side is 0.
After this step our linear program now looks as follows.
min x 1 ≥ 0 , x 2 ≤ 0 , x 3 −v 1 x 1 − v 2 x 2 − v 3 x 3
s.t. a 1 x 1 + x 2 + x 3 − b 1 ≤ 0 (5) x 1 + a 2 x 2 − b 2 = 0 (6) −a 3 x 3 + b 3 ≤ 0 (7)
Note that the special constraints listed below the min do not change.
Step 3. Define a non-negative dual variable for each inequality constraint, and an unre- stricted dual variable for each equality constraint.
To constraints (5) and (7) we associate variables λ 1 ≥ 0 and λ 3 ≥ 0 respectively. To constraint (6) we associate λ 2 , which is unrestricted.
Step 4. For each constraint, eliminate the constraint and add the term (dual variable)*(left- hand side of constraint) to the objective. Maximize the result over the dual variables.
large), and thus the objective value is −∞. Since the outer player wants to maximize the value of the inner player’s minimization problem, it should therefore choose the values of λ 1 , λ 2 , λ 3 so that a 1 λ 1 + λ 2 − v 1 ≥ 0. The same reasoning applies to the other two terms. The outer player must choose its values so that λ 1 + a 2 λ 2 − v 2 ≤ 0 otherwise the inner player can make the term
x 2 (λ 1 + a 2 λ 2 − v 2 )
tend to −∞ by choosing x 2 → −∞. Finally, since x 3 is unrestricted, the only way to make sure the term x 3 (λ 1 − a 3 λ 3 − v 3 )
cannot approach −∞ is to choose the dual values such that λ 1 − a 3 λ 3 − v 3 = 0. After these changes we have a new linear program. Note that the primal variables no longer appear.
max λ 1 ≥ 0 ,λ 2 ,λ 3 ≥ 0
−b 1 λ 1 − b 2 λ 2 − b 3 λ 3
a 1 λ 1 + λ 2 − v 1 ≥ 0 (14) λ 1 + a 2 λ 2 − v 2 ≤ 0 (15) λ 1 − a 3 λ 3 − v 3 = 0 (16)
Step 7. If the linear program in step 1 was rewritten as a minimization, rewrite the result of the previous step as a minimization; otherwise, do nothing.
The result looks as follows. Optionally, the constraints can be also be rearranged in whichever form is most natural.
min λ 1 ≥ 0 ,λ 2 ,λ 3 ≥ 0 b 1 λ 1 + b 2 λ 2 + b 3 λ 3
a 1 λ 1 + λ 2 ≥ v 1 (17) λ 1 + a 2 λ 2 ≤ v 2 (18) λ 1 − a 3 λ 3 = v 3 (19)
This completes the process of taking the dual. As an exercise, you can check for yourself that by taking the dual of this new linear program, you recover the primal.
3. Key Results
A linear program may be infeasible, unbounded, or have a finite optimum. A progam is infeasible if there no solution that satisfies all the given constraints. For instance, suppose that in our original primal program we have a 1 = a 2 = a 3 = 1, and b 1 = −1 whereas b 2 + b 3 ≥ 1. Then clearly there is no solution to constraints (2–4). (Feasibility only has to do with the constraints, no the objective.) A linear program can alternatively be unbounded. This means that for any feasible solution, there is another feasible solution with strictly higher objective value, in the case of a maximization program, or strictly lower objective value, in the case of a minimization program. For instance, suppose that in our original primal we have a 1 = a 2 = 1 and
Finite optimum Unbounded Infeasible
Finite optimum Possible Impossible Impossible
Unbounded Impossible Impossible Possible
Infeasible Impossible Possible Possible
Table 1: Possible combinations for the primal and its dual.
a 3 = −1, whereas b 1 = b 2 = b 3 = 0, and finally the coefficients in the objective are all positive :v 1 , v 2 , v 3 > 0. Then note that c(0, 0 , 1) is a feasible solution for any c ∈ R. The objective value of this solution is cv 3 , which we can make abitrarily large by letting c → +∞. Techincally, this means there is no optimal solution, even though there are feasible solutions. If a program is feasible and bounded, it has a finite optimum (though it may not be unique). Table 1 lists the possible relationships between feasibility of a primal program and its dual. In particular, note that if the primal program is unbounded, the dual is infeasible, and if the dual is unbounded, the primal program is infeasible. But it may be the case that both are infeasible. The value of the objective function of the primal program at an optimum is denoted VP and the optimum value of the dual is denoted VD. The major result of linear programming is the following.
Theorem 1 (Strong duality) If a linear programming problem has an optimal solution, so does its dual, and VP = VD.
This result relates the values of the primal and dual programs, but not their solutions. The following result is written in reference to out running example. It should be straightforward to adapt it to other linear programs.
Theorem 2 (Complementary slackness) Let (x 1 , x 2 , x 3 ) and (λ 1 , λ 2 , λ 3 ) be feasible so- lutions to the primal and dual problem, respectively. They are optimal solutions for the two respective problems if and only if
λ 1 (a 1 x 1 + x 2 + x 3 − b 1 ) = 0 λ 2 (x 1 + a 2 x 2 − b 2 ) = 0 λ 3 (−a 3 x 3 + b 3 ) = 0
and
x 1 (a 1 λ 1 + λ 2 − v 1 ) = 0 x 2 (λ 1 + a 2 λ 2 − v 2 ) = 0 x 3 (λ 1 − a 3 λ 3 − v 3 ) = 0
These constraints are known as the complementary slackness conditions. Note that the first three constraints can be read off from (8–10), and the final three from (11–13). So we get the complementary slackness conditions as a by-product of our procedure for taking the dual.
