Download Integer Programming, Lecture Slides - Mathematics - Prof Raphael Hauser and more Slides Mathematics in PDF only on Docsity!
INTEGER PROGRAMMING
PART B, MATHEMATICAL INSTITUTE UNIVERSITY OF OXFORD, MICHAELMAS TERM 2010
RAPHAEL HAUSER∗
- Examples of Integer Programming Problems. To explain what integer programming problems are, it is best to start with a few illustrative examples.
Example 1 (The Assignment Problem). A team of n people have to carry out n jobs, each person carrying out exactly one job. If person i is assigned to job j, a cost cij is incurred. Which assignment minimises the total cost?
To formulate this problem mathematically, we introduce decision variables, con- straints, and an objective function:
Decision variables: For (i, j ∈ [1, n] := { 1 ,... , n}), we set
xij =
1 if person i carries out job j, 0 otherwise.
Constraints: Each person does exactly one job, that is,
∑^ n
j=
xij = 1 (i ∈ [1, n])
Each job is done by exactly one person,
∑^ n
i=
xij = 1 (j ∈ [1, n])
The decision variables are binary,
xij ∈ B := { 0 , 1 }, (i, j ∈ [1, n]).
Objective function: The total cost is
∑n i=
∑n j=1 cij^ xij^.
∗Mathematical Institute, 24-29 St Giles’, Oxford OX1 3LB, United Kingdom. Email: [email protected]
Fig. 1.1. How to assign workers to vehicles?
Thus, the assignment problem can be modelled as follows:
min x
∑n
i=
∑^ n
j=
cij xij
s.t.
∑^ n
j=
xij = 1 for i = 1,... , n,
∑^ n
i=
xij = 1 for j = 1,... , n,
xij ∈ B for i, j = 1,... , n.
More general integer programs are mathematical problems of the following form: Integer Programs:
max x cTx
s.t. Ax ≤ b, (componentwise) x ≥ 0 , (componentwise) x ∈ Zn,
where A is a matrix and b, c are vectors with rational coefficients, and where inequal- ities such as Ax ≤ b are to be understood as componentwise inequalities.
Binary Integer Programs:
max x cTx
s.t. Ax ≤ b x ∈ Bn
2 × 3 × 7 ×
Fig. 1.3. How to cut this paper roll so as to minimise waste for the given size orders?
integers, so that the model becomes
max
∑^ n
i=
cixi
s.t.
∑^ n
i=
aixi ≤ b,
x ≥ 0 , x ∈ Zn.
A generalisation of this problem in which more than one knapsack occur is called the Cutting Stock Problem. This kind of problem plays an important role in the pa- per industry. Another close relative is the Bin Packing Problem that occurs in the shipping industry.
Example 4 (The Set Covering Problem). In a city with m neighbourhoods [1, m] := { 1 ,... , m}, n potential locations [1, n] := { 1 ,... , n} for fire stations have been identified. Let Sj ⊆ [1, m] is the set of neighbourhoods that can be served from location j, and let us assume that establishing a fire station at location j incurs a cost cj. Where should one set up fire stations so as to minimise total set-up costs, and so that every neighbourhood can be served from at least one fire station.
It is tempting to formulate this situation as a combinatorial optimisation problem,
min T ⊆[1,n]
j∈T
cj :
j∈T
Sj = [1, m]
However, this form of the problem does not lend itself to being approached by the
considered location
considered location
considered location
considered location
fire station built
fire station built
Fig. 1.4. Where to build fire stations so that every neighbourhood is reachable within a given target time?
Fig. 1.5. Graph representation of the set covering problem from Figure 1.4.
algorithmic methods we will develop. We therefore reformulate the problem as a binary integer programme:
Let the incidence matrix A =
[
aij
]
be defined by
aij =
1 if i ∈ Sj , 0 otherwise.
We introduce indicator variables xj (j ∈ [1, n]) as follows,
xj =
1 if location j is selected, 0 otherwise
The covering constraint needs to be satisfied: At least one fire station must service neighbourhood i for each i,
∑^ n
j=
aij xj ≥ 1 , for each i ∈ [1, m].
c
S
S
Fig. 1.7. Subtours that don’t visit all nodes need to be disallowed.
Decision variables: For all i, j ∈ [1, n],
xij =
1 if the tour contains edge ij, 0 otherwise.
Constraints:
The salesman leaves city i exactly once
j:j 6 =i
xij = 1 (i = 1,... , n).
He arrives in city j exactly once
i:i 6 =j
xij = 1 (j = 1,... , n).
To eliminate solutions with subtours, introduce cut-set constraints:
i∈S
j /∈S
xij ≥ 1 ∀ S ⊂ V, S 6 = ∅.
Fig. 1.8. A facility location problem.
Thus, the TSP can be modelled as follows,
min x
∑n
i=
∑^ n
j=
cij xij
s.t.
j:j 6 =i
xij = 1 (i ∈ [1, n]),
∑
i:i 6 =j
xij = 1 (j ∈ [1, n]),
∑
i∈S
j /∈S
xij ≥ 1 (S ⊂ V, S 6 = ∅, V ),
x ∈ Bn×n.
Example 6 (Uncapacitated Facility Location (UFL)). A set of potential depot locationsN = { 1 ,... , n} has been identified. A set of clients M = { 1 ,... , m} is known, each of which buys products that could be delivered from one or several of the depot locations. The transportation costs for satisfying all of client i’s orders from depot j is cij. If depot j is in use, a fixed cost fj > 0 is incurred. Decide which depots to open and what proportion of the demand of client i to satisfy from depot j so as to minimise the total costs.
Modelling UFL as a mixed integer program (MIP): Decision variables: For each pair (i, j) ∈ M × N let xij ∈ [0, 1] be the proportion of the demand of of client i satisfied from depot j.
x
s
d
Fig. 1.9. A lot-sizing problem.
Decision variables: xt is the amount produced in period t (allowed to be fractional). yt = 1 if production occurs in period t, and yt = 0 otherwise. st is the amount of product held in storage during period t. Objective function: ∑^ n
t=
(ptxt + ftyt + htst).
Constraints: No product is ever thrown away,
st− 1 + xt = dt + st, ∀ t.
Production can only occur when the machine is switched on,
xt ≤ M yt ∀ t,
where M is a sufficiently large number, e.g. the total demand. Note that this is a big M type constraint.
Production and storage are nonnegative, xt, st ≥ 0 for all t. We thus arrive at the following MIP model for ULS,
min x,y,s
∑n
t=
ptxt +
∑^ n
t=
ftyt +
∑^ n
t=
htst
s.t. st− 1 + xt = dt + st (t ∈ [1, n]), xt ≤ M yt (t ∈ [1, n]), xt ≥ 0 (t ∈ [1, n]), yt ∈ B (t ∈ [1, n]), s 0 = 0, st ≥ 0 (t ∈ [1, n]).
Fig. 1.10. A problem of discrete alternatives in which a linear objective function has to be optimised over the union of two polyhedra. This example lends itself to the geometric understanding of big-M constraints.
Example 8 (Discrete Alternatives (Disjunctions)). In scheduling and other ap- plications, problems of the following type occur:
min x∈Rn^ cTx
s.t. 0 ≤ x ≤ u and either aT 1 x ≤ b 1 (1.1) or (inclusive) aT 2 x ≤ b 2 , (1.2)
where ai are vectors and bi scalars. Here “either... or” means that at least one of conditions (1.1) and (1.2) must hold.
A big M formulation can be used to model such problems as a MIPs: First, introduce extra decision variables for which of (1.1),(1.2) to impose:
y 1 =
1 if (1.1) is imposed, 0 if (1.1) is not imposed,
y 2 =
1 if (1.2) is imposed, 0 if (1.2) is not imposed.
Note that even if a condition is not imposed, it may still hold, but when it is imposed, it must hold.
Second, let M ≥ max
max 0 ≤x≤u aT i x − bi : i = 1, 2
, a bound that can easily be computed via LP, as we shall see later.
c
Fig. 2.1. The polyhedron is the enlarged (“relaxed”) domain of red dots.
Fig. 2.2. An example where the optimal integral solution (yellow dot) is far away from the optimal solution of the integer programming relaxation (white dot) and its nearest integer-feasible solution (red dot).
Proof. If the optimal objective value z∗^ of (IP) is achieved at the point x, then x is feasible for (IP), and hence it is also feasible for (LP). Therefore, ¯z ≥ cTx = z∗.
The consequence of the second effect is that it is much easier to solve the problem (LP) than (IP).
We will shortly turn to studying how to solve LPs. Before we do so, let us remark that solving the LP relaxation and rounding the optimal values of the decision variables to the nearest feasible integer valued feasible solution is not always a good approximation scheme for integer programming problems, as the graphical example of Figure 2.2 shows.
2.1. The Simplex Method Explained by Example. We will now discuss the simplex algorithm for solving general LPs. To motivate the method, we start with a simple example:
Example 9. Consider the LP instance
z = max x 5 x 1 + 4x 2 + 3x 3
s.t. 2 x 1 + 3x 2 + x 3 ≤ 5 4 x 1 + x 2 + 2x 3 ≤ 11 3 x 1 + 4x 2 + 2x 3 ≤ 8 x 1 , x 2 , x 3 ≥ 0.
Preliminary step I: introduce slack variables
z = max 5x 1 + 4x 2 + 3x 3 + 0x 4 + 0x 5 + 0x 6 s.t. 2 x 1 + 3x 2 + x 3 + x 4 = 5 4 x 1 + x 2 + 2x 3 + x 5 = 11 3 x 1 + 4x 2 + 2x 3 + x 6 = 8 x 1 , x 2 , x 3 , x 4 , x 5 , x 6 ≥ 0.
Preliminary step II: express in dictionary form
max z s.t. x 1 ,... , x 6 ≥ 0 , where x 4 = 5 − 2 x 1 − 3 x 2 − x 3 x 5 = 11 − 4 x 1 − x 2 − 2 x 3 x 6 = 8 − 3 x 1 − 4 x 2 − 2 x 3 z = 0 + 5x 1 + 4x 2 + 3x 3.
Step 0: x 1 , x 2 , x 3 = 0, x 4 = 5, x 5 = 11, x 6 = 8 is an initial feasible solution. x 1 , x 2 , x 3 are called the nonbasic variables and x 4 , x 5 , x 6 basic variables. That is, ba- sic variables are expressed in terms of nonbasic ones.
Step 1: We note that as long as x 1 is increased by at most
5 2 = min
all xi remain nonnegative, but z increases. Setting x 1 = 5/2 and substituting into the dictionary, we find x 2 , x 3 , x 4 = 0, x 5 = 1, x 6 = 1/2, z = 25/2 as an improved feasible solution. We call x 1 the pivot of the iteration.
We can now express the variables x 1 , x 5 , x 6 , z in terms of the new nonbasic vari- ables x 2 , x 3 , x 4 (those currently set to zero) to obtain a new dictionary. To do this, use line 1 (the line that relates x 1 and x 4 ) of the dictionary to express x 1 in terms of x 2 , x 3 , x 4 ,
x 1 =
5 − 3 x 2 − x 3 − x 4
and substitute the right hand side for x 1 in the remaining equations. The new dictio- nary then looks as follows,
which was obtained after two pivoting steps, could have been obtained directly from the input data of the original LP instance
(LPI) max 5x 1 + 4x 2 + 3x 3 + 0x 4 + 0x 5 + 0x 6 s.t. 2x 1 + 3x 2 + x 3 + x 4 = 5 4 x 1 + x 2 + 2x 3 + x 5 = 11 3 x 1 + 4x 2 + 2x 3 + x 6 = 8 x 1 , x 2 , x 3 , x 4 , x 5 , x 6 ≥ 0
if we had identified the relevant basic variables:
The constraints of (LPI) imply a functional dependence between the nonnegative decision variables xi, expressed by the linear system
Ax = b, (2.6)
where
A =
(^) , b =
The basic variables of dictionary (2.5) are x 3 , x 1 , x 5. Writing
xB :=
[
x 3 x 1 x 5
]T
, xN :=
[
x 2 x 4 x 6
]T
for the vectors of basic and nonbasic variables respectively, and extracting the sub- matrices
AB =
, AN =
consisting of the columns of A that correspond to basic and nonbasic variables in the same order, the linear system (2.6) can be written as
AB xB + AN xN = b.
Solving for the basic variables, we obtain
xB = A− B 1 (b − AN xN ). (2.7)
Likewise, the objective function can be written as
z = cT B xB + cT N xN ,
where
cB =
[
]T
, cN =
[
]T
and using (2.7), this can be expressed as
z = cT B A− B^1 b +
cT N − cT B A− B^1 AN
xN.
The dictionary (2.5) is therefore the same as the system of equations
xB = A− B 1 b − A− B^1 AN xN , z = cT B A− B^1 b +
cT N − cT B A− B^1 AN
xN.
More generally, let us consider a general LP instance
(P) max cTx s.t. Ax = b, x ≥ 0 ,
where A is a m × n matrix with m ≤ n.
Definition 2.3. i) A dictionary of (P) is a system of equations
xB = A− B 1 b − A− B^1 AN xN , z = cT B A− B^1 b +
cT N − cT B A− B^1 AN
xN ,
equivalent to
Ax = b, z = cTx,
where xB is a subvector (the so called basis) of size m of x such that the m×m matrix AB consisting of the columns of A that correspond to the components of xB is nonsingular, and where xN is the remaining part of X and AN the submatrix of A consisting of the columns corresponding to xN. ii) Furthermore, this is called a feasible dictionary if A− B^1 b ≥ 0 , so that x = (xB , xN ) = (A− B^1 b, 0) is a feasible (but generally suboptimal) solution for (P). (xB , xN ) is then called a basic feasible solution. Having formally defined the notion of dictionary, we are ready to describe the dictionary form of the simplex algorithm in full generality:
Algorithm 2.4.
- Choose a basic feasible solution (xB , xN ).
- Until cN − AT N A−BT cB ≤ 0 (componentwise), repeat: i) Choose i ∈ {ℓ ∈ N : cℓ > AT ℓ A−BT cB }, where Aℓ is the ℓ-th column of A. ii) If A− B^1 Ai ≤ 0 (componentwise) (P) is unbounded (objective → ∞), stop. Else choose j ∈ {ℓ ∈ B : δℓ/μℓ = min{δk/μk : μk > 0 }}, where δk is the k-th entry of A− B^1 b and μk the k-th entry of A− B 1 Ai. End. iii) Move i from N to B and j from B to N. End
- (xB , xN ) = (A− B^1 b, 0) is an optimal basic solution, stop.
previous section,
x 4 = 5 − 2 x 1 − 3 x 2 − x 3 x 5 = 11 − 4 x 1 − x 2 − 2 x 3 x 6 = 8 − 3 x 1 − 4 x 2 − 2 x 3 z = 0 + 5x 1 + 4x 2 + 3x 3 ,
can be written in tableau format as follows,
The tableau is obtained by moving all the xi in the dictionary to the left-hand side and constants to the right-hand side, then multiplying the last row by −1 and omitting all the variables (and in the case of z, its coefficient −1).
Note that the basic variables can be identified via an identity matrix that appears in the tableau. Furthermore, the rightmost entry of the last row equals the negative of the objective value at the current basic feasible dictionary. Obviously, an optimal dictionary corresponds to a tableau with all nonpositive entries on the last row (apart from possibly the rightmost entry).
Applying the same procedure to the second dictionary
x 1 =
x 2 −
x 3 −
x 4
x 5 = 1 + 5x 2 + 2x 4
x 6 =
x 2 −
x 3 +
x 4
z =
x 2 +
x 3 −
x 4 ,
we obtain the following tableau,
Here is how the first tableau can be directly transformed into the second one (it is straightforward to check that this is the tableau equivalent to chaning the dictionaries according to the rules of the simplex algorithm):
- Among the columns on the left, identify one whose last entry is positive.
We call this column the pivot column.
- For each row (apart from the last) for which the entry t in the pivot column is positive, look up the entry u in the rightmost column. If no such row exists, the problem is unbounded. If several such rows exist, pick the one for which t/u is small- est and call it the pivot row.
- Divide the pivot row by t,
- For all other rows i of the tableau, subtract the “rescaled” pivot row ti times, where ti is the row-i entry of the pivot colum,
2.4. First Graphic Interpretation of the Simplex Method. For the pur- poses of understanding how the simplex method works, it is helpful to interpret it geometrically. We give two such interpretation, the first being more natural, while the second is the historical precedent and gave the method its name.
Consider the polyhedron P = {x ∈ Rn^ : Ax = b, x ≥ 0 }
that constitutes the feasible domain of the LP problem
(P) max cTx s.t. Ax = b, x ≥ 0.
