Download Qunatitative methods and more Study notes Economics in PDF only on Docsity!
Chapter 9
Linear programming
The nature of the programmes a computer scientist has to conceive often requires some knowl- edge in a specific domain of application, for example corporate management, network proto- cols, sound and video for multimedia streaming,... Linear programming is one of the necessary knowledges to handle optimization problems. These problems come from varied domains as production management, economics, transportation network planning,... For example, one can mention the composition of train wagons, the electricity production, or the flight planning by airplane companies. Most of these optimization problems do not admit an optimal solution that can be computed in a reasonable time, that is in polynomial time (See Chapter 3). However, we know how to ef- ficiently solve some particular problems and to provide an optimal solution (or at least quantify the difference between the provided solution and the optimal value) by using techniques from linear programming. In fact, in 1947, G.B. Dantzig conceived the Simplex Method to solve military planning problems asked by the US Air Force that were written as a linear programme, that is a system of linear equations. In this course, we introduce the basic concepts of linear programming. We then present the Simplex Method, following the book of V. Chv´atal [2]. If you want to read more about linear programming, some good references are [6, 1]. The objective is to show the reader how to model a problem with a linear programme when it is possible, to present him different methods used to solve it or at least provide a good ap- proximation of the solution. To this end, we present the theory of duality which provide ways of finding good bounds on specific solutions. We also discuss the practical side of linear programming: there exist very efficient tools to solve linear programmes, e.g. CPLEX [3] and GLPK [4]. We present the different steps leading to the solution of a practical problem expressed as a linear programme.
9.1 Introduction
A linear programme is a problem consisting in maximizing or minimizing a linear function while satisfying a finite set of linear constraints.
130 CHAPTER 9. LINEAR PROGRAMMING
Linear programmes can be written under the standard form: Maximize (^) ∑nj= 1 c (^) j x (^) j Subject to: (^) ∑nj= 1 ai j x (^) j ≤ bi for all 1 ≤ i ≤ m x (^) j ≥ 0 for all 1 ≤ j ≤ n.
All constraints are inequalities (and not equations) and all variables are non-negative. The variables x (^) j are referred to as decision variables. The function that has to be maximized is called the problem objective function. Observe that a constraint of the form (^) ∑nj= 1 ai j x (^) j ≥ bi may be rewritten as (^) ∑nj= 1 (−ai j )x (^) j ≤ −b (^) i. Similarly, a minimization problem may be transformed into a maximization problem: minimizing (^) ∑nj= 1 c (^) j x (^) j is equivalent to maximizing (^) ∑nj= 1 (−c (^) j )x (^) j. Hence, every maximization or minimization problem subject to linear constraints can be reformulated in the standard form (See Exercices 9.1 and 9.2.). A n-tuple (x 1 ,... , xn ) satisfying the constraints of a linear programme is a feasible solution of this problem. A solution that maximizes the objective function of the problem is called an optimal solution. Beware that a linear programme does not necessarily admits a unique optimal solution. Some problems have several optimal solutions while others have none. The later case may occur for two opposite reasons: either there exist no feasible solutions, or, in a sense, there are too many. The first case is illustrated by the following problem.
Maximize 3 x 1 − x 2 Subject to: x 1 + x 2 ≤ 2 − 2 x 1 − 2 x 2 ≤ − 10 x 1 , x 2 ≥ 0
which has no feasible solution (See Exercise 9.3). Problems of this kind are referred to as unfeasible. At the opposite, the problem
Maximize x 1 − x 2 Subject to: − 2 x 1 + x 2 ≤ − 1 −x 1 − 2 x 2 ≤ − 2 x 1 , x 2 ≥ 0
has feasible solutions. But none of them is optimal (See Exercise 9.3). As a matter of fact, for every number M, there exists a feasible solution x 1 , x 2 such that x 1 − x 2 > M. The problems verifying this property are referred to as unbounded. Every linear programme satisfies exactly one the following assertions: either it admits an optimal solution, or it is unfeasible, or it is unbounded. Geometric interpretation. The set of points in IRn^ at which any single constraint holds with equality is a hyperplane in IR n^. Thus each constraint is satisfied by the points of a closed half-space of IR n^ , and the set of feasible solutions is the intersection of all these half-spaces, a convex polyhedron P.
Because the objective function is linear, its level sets are hyperplanes. Thus, if the maximum value of cx over P is z ∗^ , the hyperplane cx = z∗^ is a supporting hyperplane of P. Hence cx = z∗ contains an extreme point (a corner) of P. It follows that the objective function attains its maximum at one of the extreme points of P.
132 CHAPTER 9. LINEAR PROGRAMMING
- Any feasible solution (x 1 , x 2 , x 3 , x 4 , x 5 , x 6 ) of (9.7) can be reduced by a simple removal of the slack variables into a feasible solution (x 1 , x 2 , x 3 ) of (9.4).
- This relationship between the feasible solutions of (9.4) and the feasible solutions of (9.7) allows to produce the optimal solution of (9.4) from the optimal solutions of (9.7) and vice versa.
The Simplex strategy consists in finding the optimal solution (if it exists) by successive improvements. If we have found a feasible solution (x 1 , x 2 , x 3 ) of (9.7), then we try to find a new solution ( x¯ 1 , x¯ 2 , x¯ 3 ) which is better in the sense of the objective function:
5 ¯x 1 + 4 ¯x 2 + 3 ¯x 3 ≥ 5 x 1 + 4 x 2 + 3 x 3. By repeating this process, we obtain at the end an optimal solution. To start, we first need a feasible solution. To find one in our example, it is enough to set the decision variables x 1 , x 2 , x 3 to zero and to evaluate the slack variables x 4 , x 5 , x 6 using (9.6). Hence, our initial solution,
x 1 = 0 , x 2 = 0 , x 3 = 0 , x 4 = 5 , x 5 = 11 , x 6 = 8 (9.8)
gives the result z = 0. We now have to look for a new feasible solution which gives a larger value for z. Finding such a solution is not hard. For example, if we keep x 2 = x 3 = 0 and increase the value of x 1 , then we obtain z = 5 x 1 ≥ 0. Hence, if we keep x 2 = x 3 = 0 and if we set x 1 = 1, then we obtain z = 5 (and x 4 = 3 , x 5 = 7 , x 6 = 5). A better solution is to keep x 2 = x 3 = 0 and to set x 1 = 2; we then obtain z = 10 (and x 4 = 1 , x 5 = 3 , x 6 = 2). However, if we keep x 2 = x 3 = 0 and if we set x 1 = 3, then z = 15 and x 4 = x 5 = x 6 = −1, breaking the constraint xi ≥ 0 for all i. The conclusion is that one can not increase x 1 as much as one wants. The question then is: how much can x 1 be raised (when keeping x 2 = x 3 = 0) while satisfying the constraints (x 4 , x 5 , x 6 ≥ 0)? The condition x 4 = 5 − 2 x 1 − 3 x 2 − x 3 ≥ 0 implies x 1 ≤ 52. Similarly, x 5 ≥ 0 implies x 1 ≤ (^114) and x 6 ≥ 0 implies x 1 ≤ 83. The first bound is the strongest one. Increasing x 1 to this bound gives the solution of the next step:
x 1 =
, x 2 = 0 , x 3 = 0 , x 4 = 0 , x 5 = 1 , x 6 =
which gives a result z = 252 improving the last value z = 0 of (9.8). Now, we have to find a new feasible solution that is better than (9.9). However, this task is not as simple as before. Why? As a matter of fact, we had at disposal the feasible solution of (9.8), but also the system of linear equations (9.6) which led us to a better feasible solution. Thus, we should build a new system of linear equations related to (9.9) in the same way as (9.6) is related to (9.8). Which properties should have this new system? Note first that (9.6) express the strictly positive variables of (9.8) in function of the null variables. Similarly, the new system has to express the strictly positive variables of (9.9) in function of the null variables of (9.9): x 1 , x 5 , x 6 (and z) in function of x 2 , x 3 and x 4. In particular, the variable x 1 , whose value just increased
9.2. THE SIMPLEX METHOD 133
from zero to a strictly positive value, has to go to the left side of the new system. The variable x 4 , which is now null, has to take the opposite move. To build this new system, we start by putting x 1 on the left side. Using the first equation of (9.6), we write x 1 in function of x 2 , x 3 , x 4 :
x 1 =
x 2 −
x 3 −
x 4 (9.10)
Then, we express x 5 , x 6 and z in function of x 2 , x 3 , x 4 by substituting the expression of x 1 given by (9.10) in the corresponding lines of (9.6).
x 5 = 11 − 4
x 2 −
x 3 −
x 4
− x 2 − 2 x 3
= 1 + 5 x 2 + 2 x 4 ,
x 6 = 8 − 3
x 2 −
x 3 −
x 4
− 4 x 2 − 2 x 3
x 2 −
x 3 +
x 4 ,
z = 5
x 2 −
x 3 −
x 4
x 2 +
x 3 −
x 4.
So the new system is
x 1 = 52 − 32 x 2 − 12 x 3 − 12 x 4 x 5 = 1 + 5 x 2 + 2 x 4 x 6 = 12 + 12 x 2 − 12 x 3 + 32 x 4 z = 252 − 72 x 2 + 12 x 3 − 52 x 4.
As done at the first iteration, we now try to increase the value of z by increasing a right variable of the new system, while keeping the other right variables at zero. Note that raising x 2 or x 4 would lower the value of z, against our objective. So we try to increase x 3. How much? The answer is given by (9.11) : with x 2 = x 4 = 0, the constraint x 1 ≥ 0 implies x 3 ≤ 5, x 5 ≥ 0 impose no restriction and x 6 ≥ 0 implies that x 3 ≤ 1. To conclude x 3 = 1 is the best we can do, and the new solution is
x 1 = 2 , x 2 = 0 , x 3 = 1 , x 4 = 0 , x 5 = 1 , x 6 = 0 (9.12)
and the value of z increases from 12.5 to 13. As stated, we try to obtain a better solution but also a system of linear equations associated to (9.12). In this new system, the (strictly) positive variables x 2 , x 4 , x 6 have to appear on the right. To build this new system, we start by handling the new left variable, x 3. Thanks to the third equation of (9.11) we rewrite x 3 and by substitution
9.2. THE SIMPLEX METHOD 135
Property 9.1. Any feasible solution of the equations of a dictionary is also a feasible solution of (9.16) and vice versa.
For example, for any choice of x 1 , x 2 ,... , x 6 and of z, the three following assertions are equivalent:
- (x 1 , x 2 ,... , x 6 , z) is a feasible solution of (9.6);
- (x 1 , x 2 ,... , x 6 , z) is a feasible solution of (9.11);
- (x 1 , x 2 ,... , x 6 , z) is a feasible solution of (9.13).
From this point of view, the three dictionaries (9.6), (9.11) and (9.13) contain the same information on the dependencies between the seven variables. However, each dictionary present this information in a specific way. (9.6) suggests that the values of the variables x 1 , x 2 and x 3 can be chosen at will while the values of x 4 , x 5 , x 6 and z are fixed. In this dictionary, the decision variables x 1 , x 2 , x 3 act as independent variables while the slack variables x 4 , x 5 , x 6 are related to each other. In the dictionary (9.13), the independent variables are x 2 , x 4 , x 6 and the related ones are x 3 , x 1 , x 5 , z.
Property 9.2. The equations of a dictionary have to express m variables among x 1 , x 2 ,... , xn+m , z in function of the n remaining others.
Properties 9.1 and 9.2 define what a dictionary is. In addition to these two properties, the dictionaries (9.6),(9.11) and (9.13) have the following property.
Property 9.3. When putting the right variables to zero, one obtains a feasible solution by eval- uating the left variables.
The dictionaries that have this last property are called feasible dictionaries. As a matter of fact, any feasible dictionary describes a feasible solution. However, all feasible solutions cannot be described by a feasible dictionary. For example, no dictionary describe the feasible solution x 1 = 1, x 2 = 0, x 3 = 1, x 4 = 2, x 5 = 5, x 6 = 3 of (9.4). The feasible solutions that can be described by dictionaries are referred as basic solutions. The Simplex Method explores only basic solutions and ignores all other ones. But this is valid because if an optimal solution exists, then there is an optimal and basic solution. Indeed, if a feasible solution cannot be improved by the Simplex Method, then increasing any of the n right variables to a positive value never increases the objective function. In such case, the objective function must be written as a linear function of these variables in which all the coefficient are non-positive, and thus the objective function is clearly maximum when all the right variables equal zero. For example, it was the case in (9.14).
9.2.3 Finding an initial solution
In the previous examples, the initialisation of the simplex method was not a problem. As a matter of fact, we carefully chosen problems with all b (^) i non negative. This way x 1 = 0 , x 2 = 0 ,
136 CHAPTER 9. LINEAR PROGRAMMING
· · · , xn = 0 was a feasible solution and the dictionary was easily built. These problems are called problems with a feasible origin. What happens when confronted with a problem with an unfeasible origin? Two difficulties arise. First, a feasible solution can be hard to find. Second, even if we find a feasible solution, a feasible dictionary has then to be built. A way to solve these difficulties is to use an other problem called auxiliary problem:
Minimise x 0 Subject to: (^) ∑nj= 1 a (^) i j x (^) j − x 0 ≤ b (^) i (i = 1 , 2 , · · · , m) x (^) j ≥ 0 ( j = 0 , 1 , · · · , n).
A feasible solution of the auxiliary problem is easily available: it is enough to set x (^) j = 0 ∀ j ∈ [ 1... n] and to give to x 0 a big enough value. It is now easy to see that the original problem has a feasible solution if and only if the auxiliary problem has a feasible solution with x 0 = 0. In other words, the original problem has a feasible solution if the optimal value of the auxiliary problem is null. Thus, the idea is to first solve the auxiliary problem. Let see the details on an example. Maximise x 1 − x 2 + x 3 Subject to : 2 x 1 − x 2 + 2 x 3 ≤ 4 2 x 1 − 3 x 2 + x 3 ≤ − 5 −x 1 + x 2 − 2 x 3 ≤ − 1 x 1 , x 2 , x 3 ≥ 0
Maximise −x 0 Subject to: 2 x 1 − x 2 + 2 x 3 − x 0 ≤ 4 2 x 1 − 3 x 2 + x 3 − x 0 ≤ − 5 −x 1 + x 2 − 2 x 3 − x 0 ≤ − 1 x 1 , x 2 , x 3 , x 0 ≥ 0
We introduce the slack variables. We obtain the dictionary:
x 4 = 4 − 2 x 1 + x 2 − 2 x 3 + x 0 x 5 = − 5 − 2 x 1 + 3 x 2 − x 3 + x 0 x 6 = − 1 + x 1 − x 2 + 2 x 3 + x 0 w = − x 0.
Note that this dictionary is not feasible. However it can be transformed into a feasible one by operating a simple pivot , x 0 entering the basis as x 5 exits it:
x 0 = 5 + 2 x 1 − 3 x 2 + x 3 + x 5 x 4 = 9 − 2 x 2 − x 3 + x 5 x 6 = 4 + 3 x 1 − 4 x 2 + 3 x 3 + x 5 w = − 5 − 2 x 1 + 3 x 2 − x 3 − x 5.
138 CHAPTER 9. LINEAR PROGRAMMING
The desired dictionary then is:
x 3 = 1. 6 − 0. 2 x 1 + 0. 2 x 5 + 0. 6 x 6 x 2 = 2. 2 + 0. 6 x 1 + 0. 4 x 5 + 0. 2 x 6 x 4 = 3 − x 1 − x 6 z = − 0. 6 + 0. 2 x 1 − 0. 2 x 5 + 0. 4 x 6
This strategy is known as the simplex method with two phases. During the first phase, we set and solve the auxiliary problem. If the optimal value is null, we do the second phase consisting in solving the original problem. Otherwise, the original problem is not feasible.
9.3 Duality of linear programming
Any maximization linear programme has a corresponding minimization problem called the dual problem. Any feasible solution of the dual problem gives an upper bound on the optimal value of the initial problem, which is called the primal. Reciprocally, any feasible solution of the primal provides a lower bound on the optimal value of the dual problem. Actually, if one of both problems admits an optimal solution, then the other problem does as well and the optimal solutions match each other. This section is devoted to this result also known as the Duality Theorem. Another interesting application of the dual problem is that, in some problems, the variables of the dual have some useful interpretation.
9.3.1 Motivations: providing upper bounds on the optimal value
A way to quickly estimate the optimal value of a maximization linear programme simply con- sists in computing a feasible solution whose value is sufficiently large. For instance, let us consider the following problem formulated in Problem 9.4. The solution ( 0 , 0 , 1 , 0 ) gives us a lower bound of 5 for the optimal value z ∗^. Even better, we get z∗^ ≥ 22 by considering the so- lution ( 3 , 0 , 2 , 0 ). Of course, doing so, we have no way to know how close to the optimal value the computed lower bound is.
Problem 9.4. Maximize 4 x 1 + x 2 + 5 x 3 + 3 x 4 Subject to: x 1 − x 2 − x 3 + 3 x 4 ≤ 1 5 x 1 + x 2 + 3 x 3 + 8 x 4 ≤ 55 −x 1 + 2 x 2 + 3 x 3 − 5 x 4 ≤ 3 x 1 , x 2 , x 3 , x 4 ≥ 0
The previous approach provides lower bounds on the optimal value. However, this intuitive method is obviously less efficient than the Simplex Method and this approach provides no clue about the optimality (or not) of the obtained solution. To do so, it is interesting to have upper bounds on the optimal value. This is the main topic of this section.
9.3. DUALITY OF LINEAR PROGRAMMING 139
How to get an upper bound for the optimal value in the previous example? A possible approach is to consider the constraints. For instance, multiplying the second constraint by 53 ,
we get that z ∗^ ≤ 2753. Indeed, for any x 1 , x 2 , x 3 , x 4 ≥ 0:
4 x 1 + x 2 + 5 x 3 + 3 x 4 ≤
x 1 +
x 2 + 5 x 3 +
x 4 = ( 5 x 1 + x 2 + 3 x 3 + 8 x 4 ) ×
≤ 55 ×
In particular, the above inequality is satisfied by any optimal solution. Therefore, z∗^ ≤ 2753. Let us try to improve this bound. For instance, we can add the second constraint to the third one. This gives, for any x 1 , x 2 , x 3 , x 4 ≥ 0:
4 x 1 + x 2 + 5 x 3 + 3 x 4 ≤ 4 x 1 + 3 x 2 + 6 x 3 − 3 x 4 ≤ ( 5 x 1 + x 2 + 3 x 3 + 8 x 4 ) + (−x 1 + 2 x 2 + 3 x 3 − 5 x 4 ) ≤ 55 + 3 = 58
Hence, z∗^ ≤ 58. More formally, we try to upper bound the optimal value by a linear combination of the constraints. Precisely, for all i, let us multiply the ith^ constraint by yi ≥ 0 and then sum the resulting constraints. In the previous two examples, we had (y 1 , y 2 , y 3 ) = ( 0 , 53 , 0 ) and (y 1 , y 2 , y 3 ) = ( 0 , 1 , 1 ). More generally, we obtain the following inequality:
y 1 (x 1 − x 2 − x 3 + 3 x 4 ) + y 2 ( 5 x 1 + x 2 + 3 x 3 + 8 x 4 ) + y 3 (−x 1 + 2 x 2 + 3 x 3 − 5 x 4 ) = (y 1 − 5 y 2 − y 3 )x 1 + (−y 1 + y 2 + 2 y 3 )x 2 + (−y 1 + 3 y 2 + 3 y 3 )x 3 + ( 3 y 1 + 8 y 2 − 5 y 3 )x 4 ≤ y 1 + 55 y 2 + 3 y 3
For this inequality to provide an upper bound of 4x 1 + x 2 + 5 x 3 + 3 x 4 , we need to ensure that, for all x 1 , x 2 , x 3 , x 4 ≥ 0,
4 x 1 + x 2 + 5 x 3 + 3 x 4 ≤ (y 1 − 5 y 2 − y 3 )x 1 + (−y 1 + y 2 + 2 y 3 )x 2 + (−y 1 + 3 y 2 + 3 y 3 )x 3 + ( 3 y 1 + 8 y 2 − 5 y 3 )x 4.
That is, y 1 − 5 y 2 − y 3 ≥ 4, −y 1 + y 2 + 2 y 3 ≥ 1, −y 1 + 3 y 2 + 3 y 3 ≥ 5, and 3y 1 + 8 y 2 − 5 y 3 ≥ 3. Combining all inequalities, we obtain the following minimization linear programme:
Minimize y 1 + 55 y 2 + 3 y 3 Subject to: y 1 − 5 y 2 − y 3 ≥ 4 −y 1 + y 2 + 2 y 3 ≥ 1 −y 1 + 3 y 2 + 3 y 3 ≥ 5 3 y 1 + 8 y 2 − 5 y 3 ≥ 3 y 1 , y 2 , y 3 ≥ 0 This problem is called the dual of the initial maximization problem.
9.3. DUALITY OF LINEAR PROGRAMMING 141
We deduce the following lemma.
Lemma 9.7. If D is the dual of a problem P, then the dual of D is P. Informally, the dual of the dual is the primal.
9.3.3 Duality Theorem
An important aspect of duality is that feasible solutions of the primal and the dual are related.
Lemma 9.8. Any feasible solution of Problem 9.6 yields an upper bound for Problem 9.5. In other words, the value given by any feasible solution of the dual of a problem is an upper bound for the primal problem.
Proof. Let (y 1 ,... , ym ) be a feasible solution of the dual and (x 1 ,... , xn ) be a feasible solution of the primal. Then,
n
j= 1
c (^) j x (^) j ≤
n
j=
m
i=
a (^) i j yi
x (^) j ≤
m
i= 1
n
j= 1
a (^) i j x (^) j
yi ≤
m
i= 1
b (^) i yi.
Corollary 9.9. If (y 1 ,... , ym ) is a feasible solution of the dual of a problem (Problem 9.6) and (x 1 ,... , xn ) is a feasible solution of the corresponding primal (Problem 9.5) such that ∑nj= 1 c (^) j x (^) j = ∑mi= 1 bi yi , then both solutions are optimal.
Corollary 9.9 states that if we find two solutions for the dual and the primal achieving the same value, then this is a certificate of the optimality of these solutions. In particular, in that case (if they are feasible), both the primal and the dual problems have same optimal value. For instance, we can easily verify that ( 0 , 14 , 0 , 5 ) is a feasible solution for Problem 9.4 with value 29. On the other hand, ( 11 , 0 , 6 ) is a feasible solution for the dual with same value. Hence, the optimal solutions for the primal and for the dual coincide and are equal to 29. In general, it is not immediate that any linear programme may have such certificate of opti- mality. In other words, for any feasible linear programme, can we find a solution of the primal problem and a solution of the dual problem that achieve the same value (thus, this value would be optimal)? One of the most important result of the linear programming is the duality theorem that states that it is actually always the case: for any feasible linear programme, the primal and the dual problems have the same optimal solution. This theorem has been proved by D. Gale, H.W. Kuhn and A. W. Tucker [5] and comes from discussions between G.B. Dantzig and J. von Neumann during Fall 1947.
Theorem 9.10 (D UALITY T HEOREM ). If the primal problem defined by Problem 9.5 admits an optimal solution (x ∗ 1 ,... , x ∗ n ), then the dual problem (Problem 9.6) admits an optimal solution (y∗ 1 ,... , y∗ m ), and n
j= 1
c (^) j x ∗ j =
m
i= 1
b (^) i y∗ i.
142 CHAPTER 9. LINEAR PROGRAMMING
Proof. The proof consists in showing how a feasible solution (y∗ 1 ,... , y∗ m ) of the dual can be obtained thanks to the Simplex Method, so that z∗^ = (^) ∑mi= 1 bi y∗ i is the optimal value of the primal. The result then follows from Lemma 9.8. Let us assume that the primal problem has been solved by the Simplex Method. For this purpose, the slack variables have been defined by
xn+i = bi −
n
j= 1
a (^) i j x (^) j for 1 ≤ i ≤ m.
Moreover, the last line of the last dictionary computed during the Simplex Method gives the optimal value z ∗^ of the primal in the following way: for any feasible solution (x 1 ,... , xn ) of the primal we have
z =
n
j= 1
c (^) j x (^) j = z∗^ +
n+m
i= 1
c¯i xi.
Recall that, for all i ≤ n + m, ¯ci is non-positive, and that it is null if xi is one of the basis variables. We set
y∗ i = − c¯n+i for 1 ≤ i ≤ m.
Then, by definition of the y∗ i ’s and the xn+i ’s for 1 ≤ i ≤ m, we have
z =
n
j= 1
c (^) j x (^) j = z∗^ +
n
i= 1
c¯i xi −
m
i= 1
y∗ i
b (^) i −
n
j= 1
a (^) i j x (^) j
z∗^ −
m
i= 1
y∗ i bi
n
j= 1
c ¯ (^) j +
m
i= 1
a (^) i j y∗ i
x (^) j.
Since this equation must be true whatever be the affectation of the xi ’s and since the ¯ci ’s are non-positive, this leads to
z ∗^ =
m
i= 1
y∗ i bi and
c (^) j = c¯ (^) j +
m
i= 1
a (^) i j y∗ i ≤
m
i= 1
ai j y∗ i for all 1 ≤ j ≤ n.
Hence, (y∗ 1 ,... , y∗ m ) defined as above is a feasible solution achieving the optimal value of the primal. By Lemma 9.8, this is an optimal solution of the dual.
9.3.4 Relation between primal and dual
By the Duality Theorem and Lemma 9.7, a linear programme admits a solution if and only if its dual admits a solution. Moreover, according to Lemma 9.8, if a linear programme is unbounded,
144 CHAPTER 9. LINEAR PROGRAMMING
Therefore, Inequalities 9.24 and 9.25 must be equalities. As the variables are positive, we further get that
for all i,
bi −
n
j= 1
a (^) i j x (^) j
yi = 0
and for all j,
m
i= 1
ai j yi − c (^) j
x (^) j = 0.
A product is equal to zero if one of its two members is null and we obtain the desired result.
Theorem 9.12. A feasible solution (x 1 ,... , xn ) of Problem 9.5 is optimal if and only if there is a feasible solution (y 1 ,... , ym ) of Problem 9.6 such that:
∑mi= ai j y^ i =^ c^ j i f^ x^ j >^0 yi = 0 i f (^) ∑mj= 1 a (^) i j x (^) j < b (^) i
Note that, if Problem 9.5 admits a non-degenerated solution (x 1 ,... , xn ), i.e., xi > 0 for any i ≤ n, then the system of equations in Theorem 9.12 admits a unique solution.
Optimality certificates - Examples. Let see how to apply this theorem on two examples. Let us first examine the statement that
x ∗ 1 = 2 , x ∗ 2 = 4 , x ∗ 3 = 0 , x ∗ 4 = 0 , x ∗ 5 = 7 , x ∗ 6 = 0
is an optimal solution of the problem
Maximize 18 x 1 − 7 x 2 + 12 x 3 + 5 x 4 + 8 x 6 Subject to: 2 x 1 − 6 x 2 + 2 x 3 + 7 x 4 + 3 x 5 + 8 x 6 ≤ 1 − 3 x 1 − x 2 + 4 x 3 − 3 x 4 + x 5 + 2 x 6 ≤ − 2 8 x 1 − 3 x 2 + 5 x 3 − 2 x 4 + 2 x 6 ≤ 4 4 x 1 + 8 x 3 + 7 x 4 − x 5 + 3 x 6 ≤ 1 5 x 1 + 2 x 2 − 3 x 3 + 6 x 4 − 2 x 5 − x 6 ≤ 5 x 1 , x 2 , · · · , x 6 ≥ 0
In this case, (9.26) says:
2 y∗ 1 − 3 y∗ 2 + 8 y∗ 3 + 4 y∗ 4 + 5 y∗ 5 = 18 − 6 y∗ 1 − y∗ 2 − 3 y∗ 3 + 2 y∗ 5 = − 7 3 y∗ 1 + y∗ 2 − y∗ 4 − 2 y∗ 5 = 0 y∗ 2 = 0 y∗ 5 = 0
As the solution ( 13 , 0 , 53 , 1 , 0 ) is a feasible solution of the dual problem (Problem 9.6), the pro- posed solution is optimal.
9.3. DUALITY OF LINEAR PROGRAMMING 145
Secondly, is x ∗ 1 = 0 , x ∗ 2 = 2 , x ∗ 3 = 0 , x ∗ 4 = 7 , x ∗ 5 = 0
an optimal solution of the following problem?
Maximize 8 x 1 − 9 x 2 + 12 x 3 + 4 x 4 + 11 x 5 Subject to: 2 x 1 − 3 x 2 + 4 x 3 + x 4 + 3 x 5 ≤ 1 x 1 + 7 x 2 + 3 x 3 − 2 x 4 + x 5 ≤ 1 5 x 1 + 4 x 2 − 6 x 3 + 2 x 4 + 3 x 5 ≤ 22 x 1 , x 2 , · · · , x 5 ≥ 0
Here (9.26) translates into:
− 3 y∗ 1 + 7 y∗ 2 + 4 y∗ 3 = − 9 y∗ 1 − 2 y∗ 2 + 2 y∗ 3 = 4 y∗ 2 = 0
As the unique solution of the system ( 3. 4 , 0 , 0. 3 ) is not a feasible solution of Problem 9.6, the proposed solution is not optimal.
9.3.5 Interpretation of dual variables
As said in the introduction of this section, one of the major interests of the dual programme is that, in some problems, the variables of the dual problem have an interpretation. A classical example is the economical interpretation of the dual variables of the following problem. Consider the problem that consits in maximizing the benefit of a company building some products. Each variable x (^) j of the primal problem measures the amount of product j that is built, and bi the amount of resource i (needed to build the products) that is available. Note that, for any i ≤ n, j ≤ m, a (^) i, j represents the number of units of resource i needed per unit of product j. Finally, c (^) j denotes the benefit (the price) of a unit of product j. Hence, by checking the units of measure in the constraints (^) ∑ a (^) i j yi ≥ c (^) j , the variable yi must represent a benefit per unit of resource i. Somehow, the variable yi measures the unitary value of the resource i. This is illustrated by the following theorem the proof of which is omitted.
Theorem 9.13. If Problem 9.5 admits a non degenerated optimal solution with value z∗^ , then there is ε > 0 such that, for any |t (^) i | ≤ ε (i = 1 ,... , m), the problem
Maximize (^) ∑nj= c (^) j x (^) j Subject to (^) ∑nj= 1 ai j x (^) j ≤ bi + ti (i = 1 ,... , m) x (^) j ≥ 0 ( j = 1 ,... , n)
admits an optimal solution with value z ∗^ + (^) ∑mi= 1 y∗ i t (^) i , where (y∗ 1 ,... , y∗ m ) is the optimal solution of the dual of Problem 9.5.
Theorem 9.13 shows how small variations in the amount of available resources can affect the benefit of the company. For any unit of extra resource i, the benefit increases by y∗ i. Sometimes, y∗ i is called the marginal cost of the resource i.
In many networks design problems, a clever interpretation of dual variables may help to achieve more efficient linear programme or to understand the problem better.
9.4. EXERCICES 147
a) to admit an optimal solution;
b) to be unfeasible;
c) to be unbounded.
Exercise 9.5. Prove or disprove: if the problem (9.1) is unbounded, then there exists an index k such that the problem:
Maximize xk Subject to: (^) ∑nj= 1 ai j x (^) j ≤ bi for 1 ≤ i ≤ m x (^) j ≥ 0 for 1 ≤ j ≤ n
is unbounded.
Exercise 9.6. The factory RadioIn builds to types of radios A and B. Every radio is produced by the work of three specialists Pierre, Paul and Jacques. Pierre works at most 24 hours per week. Paul works at most 45 hours per week. Jacques works at most 30 hours per week. The resources necessary to build each type of radio and their selling prices as well are given in the following table:
Radio A Radio B Pierre 1h 2h Paul 2h 1h Jacques 1h 3h Selling prices 15 euros 10 euros
We assume that the company has no problem to sell its production, whichever it is. a) Model the problem of finding a weekly production plan maximizing the revenue of Ra- dioIn as a linear programme. Write precisely what are the decision variables, the objective function and the constraints. b) Solve the linear programme using the geometric method and give the optimal production plan.
Exercise 9.7. The following table shows the different possible schedule times for the drivers of a bus company. The company wants that at least one driver is present at every hour of the working day (from 9 to 17). The problem is to determine the schedule satisfying this condition with minimum cost.
Time 9 – 11h 9 – 13h 11 – 16h 12 – 15h 13 – 16h 14– 17h 16 – 17h Cost 18 30 38 14 22 16 9
Formulate an integer linear programme that solves the company decision problem.
Exercise 9.8 (Chebyshev approximation). Data : m measures of points (xi , yi ) ∈ R n+^1 , i = 1 , ..., m. Objective: Determine a linear approximation y = ax + b minimizing the largest error of approx- imation. The decision variables of this problem are a ∈ R n^ and b ∈ R. The problem may be
148 CHAPTER 9. LINEAR PROGRAMMING
formulated as: min z = max i= 1 ,...,m
{|yi − ax (^) i − b|}.
It is unfortunately not under the form of a linear program. Let us try to do some transformations.
Questions:
- We call Min-Max problem the problem of minimizing the maximum of a set of numbers:
min z = max{c 1 x, ..., ck x}.
How to write a Min-Max problem as an LP?
- Can we express the following constraints
|x| ≤ b
or |x| ≥ b
in a LP (that is without absolute values)? If yes, how?
- Rewrite the problem of finding a Chebyshev linear approximation as an LP.
9.4.2 Simplex
Exercise 9.9. Solve with the Simplex Method the following problems:
a. Maximize 3 x 1 + 3 x 2 + 4 x 3 Subject to: x 1 + x 2 + 2 x 3 ≤ 4 2 x 1 + 3 x 3 ≤ 5 2 x 1 + x 2 + 3 x 3 ≤ 7 x 1 , x 2 , x 3 ≥ 0
