Download Integer Programming, Lecture Notes - Mathematics - 3 and more Study notes Mathematics in PDF only on Docsity!
- Alternative Formulations of Integer Programmes. Recall our model of alternative disjunctions from Section 1,
min (x,y)∈R n+
c T^ x
s.t. a T i x − b (^) i ≤ M (1 − y (^) i ) (i = 1, 2), y 1 + y 2 = 1, 0 ≤ x ≤ u y ∈ B^2.
It would have been mathematically equivalent to replace the constraint
y 1 + y 2 = 1,
corresponding to the requirement that exactly one of the alternative constraints a T i x ≤ b (^) i (i = 1, 2) be imposed, by the constraint
y 1 + y 2 ≥ 1 ,
corresponding to the requirement that at least one of the alternative constraints be imposed.
Another example of an alternative formulation is the following:
Example 17. Recall our binary programming model of the travelling salesman problem,
min x
∑n
i=
∑^ n
j=
c (^) ij xij
s.t.
j:j" =i
xij = 1 (i ∈ [1, n]),
∑
i:i"=j
xij = 1 (j ∈ [1, n]),
∑
i∈S
j /∈S
xij ≥ 1 (S ⊂ V, S &= ∅),
x ∈ Bn×n^.
The cut-set constraints ∑
i∈S
j /∈S
xij ≥ 1 (S ⊂ V, S &= ∅)
were introduced to eliminate solutions that contain subtours.
Alternatively, this could be achieved using subtour elimination constraints: ∑
i∈S
j∈S
xij ≤ |S| − 1 ∀ S ⊂ N s.t. 2 ≤ |S| ≤ n − 1.
c
S
S
Fig. 5.1. Subtours that don’t visit all nodes need to be disallowed.
The new model
min x
∑n
i=
∑^ n
j=
c (^) ij xij
s.t.
j:j" =i
xij = 1 (i ∈ [1, n]),
∑
i:i"=j
xij = 1 (j ∈ [1, n]),
∑
i∈S
j∈S
xij ≤ |S| − 1 ∀ S ⊂ N s.t. 2 ≤ |S| ≤ n − 1 ,
x ∈ Bn×n^.
is mathematically equivalent to the old model – the set of feasible tours has not changed, and neither has the objective function – but from an algorithmic point of view the formulation is very different.
These examples raise the natural question as to whether it matters which for- mulation of an IP we use when several formulations are available. All integer pro- gramming problems have infinitely many different formulations that are all equivalent from a mathematical point of view in the sense that the optimal values of the decision variables are the same. However, the performance guarantee of algorithms can be very different for different formulations, so which formulation we use matters a great deal.
Deriving alternative formulations of an IP is usually a nontrivial task. Conceptu- ally, many general approaches for solving IPs – including branch-and-bound – attempt to generate better and better formulations until a trivial formulation is found – one for which the optimal solution is obvious.
5.1. Polytopes and Polyhedra. Before we discuss the business of alternative formulations further, we need to understand the geometric objects that are involved
Fig. 5.4. A convex set and its extreme points (in red).
Fig. 5.5. Polytopes can either be characterised by linear inequalities or by generators.
Theorem 5.2 (Carath´eodory’s Theorem). Let C be a bounded convex subset of R n^ , and let E ⊂ C be the set of extreme points. Then the following are true:
i) C = conv(E). ii) For every set D ⊂ C such that conv(D) = C we have C ⊂ D. iii) Any point in C can be written as a convex combination of at most n + 1 elements of E. It turns out that bounded polyhedrons are the same objects as polytopes. The characterisations in terms of constraints or generators are dual descriptions of one another 2 :
Theorem 5.3. Let P be a bounded subset of R n^. Then P is a polytope if and only if it is a polyhedron.
5.2. Geometric Interpretation of Alternative Formulations. Let us now consider the mixed integer programming problem
(MIP) max (x,y)∈R n+p^
c T x x + c T y y
s.t. Ax + By ≤ b x ∈ Zn^ ,
(^2) We note however that deriving one description explicitly from the other is a nontrivial compu- tational task
Fig. 5.6. The feasible set of the LP relaxation of an IP or MIP is a polyhedron.
Fig. 5.7. The feasible set of the MIP is the intersection of the polyhedron with the integer lattice.
where A and B are matrices, and let us denote the set of feasible solutions of (MIP) by
F :=
(x, y) ∈ R n+p^ : Ax + By ≤ b, x ∈ Zn^
We call Zn^ × R p^ the integer lattice for this problem.
Note that if we drop the integrality constraints x ∈ Zn^ , the set of points that satisfy the remaining constraints is a polyhedron:
P :=
(x, y) ∈ R n+p^ : Ax + By ≤ b
see Figure 5.6. The feasible set of the MIP is the intersection of this polyhedron with the integer lattice, F = P ∩ (Zn^ × R p^ ), see Figure 5.7.
We now note that a reformulation of (MIP) generally defines a different polyhe- dron
P˜ :=
(x, y) ∈ R n+p^ : Ax˜ + By˜ ≤ ˜b
The polyhedron P˜ also has the property that its intersection with the integer lattice yields the feasible set of (MIP), F = P˜ ∩ (Zn^ × R p^ ). In fact, there are infinitely many polyhedra with this property. But while the MIP described under an alternative for- mulation is the same, the LP relaxation is different, and algorithms such as LP based branch and bound behave differently.
programming problem
(MIP) z ∗^ = max (x,y)∈R n+p
c T x x + c T y y
s.t. Ax + By ≤ b x ∈ Zn^ ,
then any optimal basic feasible solution of the LP relaxation
(LP) z (^) LP = max (x,y)∈R n+p
c T x x + c T y y
s.t. (x, y) ∈ P
is an optimal solution of (MIP). Proof. First, recall that any basic feasible solution (x, y) of (LP) corresponds to a vertex of P and thus is among the extremal points E of P. Since P = conv(F ), Theorem 5.2 implies that x ∈ F , so that x is feasible for (MIP). Furthermore, (LP) being a relaxation of (MIP), we have z (^) LP = c T x x + c T y y ≥ z ∗^ , so that (x, y) must in fact be optimal for (MIP).
- Techniques for Finding Bounds on Integer Programmes. In our dis- cussion of the branch-and-bound approach we have learned that given an integer programming problem
(IP) z = max{c T^ x : x ∈ P ∩ Zn^ },
where P ⊆ R n^ is a given polyhedron, one often merely wishes to compute bounds
z ≤ z ∗^ ≤ z
on the optimal objective value z ∗^. Apart from helping in implicit enumeration, upper and lower bounds can also provide important stopping criteria:
- Having found a feasible point ˆx with objective value c T^ ˆx ∈ [z, z], one may decide to stop the algorithm if the gap z − z is smaller than a chosen error tolerance ε.
- In the extreme situation where ε = 0, the bounds z, z provide a certificate of optimality.
- Instead of predetermining an error tolerance ε, one may bound the maximum computation time, and in this case one is interested in the gap z − z to esti- mate the quality of the best found objective value c T^ ˆx as an approximation of the unknown value z ∗^.
Definition 6.1. A primal bound for problem (IP) is a lower bound z ≤ z ∗^. A dual bound is an upper bound z ≥ z ∗^.
We remark that Definition 6.1 refers to the maximisation problem (IP). For min- imisation problems upper bounds are primal and lower bounds are dual. This con- vention guarantees that if a maximisation problem is reformulated as a minimisation problem, the notions of primal and dual bounds do not change.
6.1. Finding Primal Bounds. The mechanism for finding primal bounds is conceptually simple: Any feasible point x ∈ P ∩ Zn^ provides a primal bound c T^ x on the optimal objective value z ∗^.
Example 19. The IP z = max x 1 − x 2 s.t. x 1 + 3x 2 ≤ 5 , x 1 , x 2 ≥ 0 , x ∈ Z^2
has x = (2, 1) among its feasible solutions, hence 1 is a lower bound on z ∗^ (which takes the value 5 in this example).
This may seem easy, but finding feasible solutions is often a difficult problem in itself. Typically, heuristics are used to overcome this problem.
Example 20. A feasible solution to the 0-1 Knapsack problem
max 5x 1 + 8x 2 + 17x 3 s.t. 4 x 1 + 3x 2 + 7x 3 ≤ 9 x ∈ B^3
can be found using the greedy heuristic, which is to grab objects that fit into the residual volume in order of decreasing value per unit volume.
- We note that c 2 /a 2 > c 3 /a 3 > c 1 /a 1.
- The residual volume is 9 , so we grab object 2 and set x 2 = 1.
- The residual volume (after packing object 2 is 6 = 9 − a 2. Object 3 does not fit, and we set x 3 = 0.
- The residual volume is 6 , object 1 fits and we set x 1 = 1.
6.2. Finding Dual Bounds. There are two broad classes of methods to gener- ate dual bounds:
i) Use a relaxation technique, such as LP relaxation, combinatorial relaxation, Lagrangian relaxation, semidefinite programming relaxation etc. (see the later parts of this course). ii) Exploit duality.
Definition 6.2. We say that the problem
(RP) z R^ = max{f (x) : x ∈ T ⊆ R n^ }
is a relaxation of the integer programming problem
(IP ) z ∗^ = max{c T^ x : x ∈ X ⊆ Zn^ }
if the following two conditions are satisfied,
Proposition 6.5. Let
(RP) z R^ = max{f (x) : x ∈ T ⊆ R n^ }
be a relaxation of the integer programming problem
(IP ) z ∗^ = max{c T^ x : x ∈ X ⊆ Zn^ }.
i) If x∗^ is an optimal solution of (RP) for which x∗^ ∈ X and f (x∗^ ) = c T^ x∗^ , then x∗^ is an optimal solution for (IP). ii) If (RP) is infeasible, then (IP) is infeasible. Proof. i) Since x∗^ ∈ X, c T^ x∗^ is a primal bound, and since x∗^ is optimal for (RP), f (x∗^ ) is a dual bound for (IP). Therefore, c T^ x∗^ ≤ z ∗^ ≤ f (x∗^ ) = c T^ x∗^ shows that x∗^ is optimal. ii) Since X ⊆ T , we have T = ∅ ⇒ X = ∅.
Example 21. Consider the binary programming problem
(BP) max 7x 1 + 4x 2 + 5x 3 + 2x 4 s.t. 3 x 1 + 3x 2 + 4x 3 + 2x 4 ≤ 6 x ∈ B^4.
The LP relaxation
(LP) max x∈R 4
7 x 1 + 4x 2 + 5x 3 + 2x 4
s.t. 3 x 1 + 3x 2 + 4x 3 + 2x 4 ≤ 6 xi ≤ 1 (i ∈ [1, 4]) xi ≥ 0 (i ∈ [1, 4])
has optimal solution x∗^ = (1, 1 , 0 , 0). Since x∗^ is binary and we did not change the objective function, x∗^ is also optimal for problem (BP).
6.2.3. Combinatorial relaxation. Sometimes hard integer programming prob- lems can be relaxed by well-solved combinatorial problems such as those we study in later lectures. This is called combinatorial relaxation.
Example 22. We will see that knapsack problems
(K) max
∑^ n
i=
c (^) i xi
s.t.
∑^ n
i=
a (^) i xi ≤ b,
x ≥ 0 , x ∈ Zn^ ,
are often well-solved using dynamic programming in the case where the volumina a (^) i (i = 1,... , n) and b are integers. In the case where a (^) i and b are rational, (K) can be relaxed by the problem
(RK) max
∑^ n
i=
c (^) i xi
s.t.
∑^ n
i=
.a (^) i /xi ≤ .b/,
x ≥ 0 , x ∈ Zn^ ,
which falls into the well-solved category.
Example 23. Let (V, A) be a directed graph with arc weights c (^) ij , (ij ∈ A), and consider the travelling salesman problem
min x
∑n
i=
∑^ n
j=
c (^) ij xij
s.t.
j:j" =i
xij = 1 (i ∈ [1, n]),
∑
i:i"=j
xij = 1 (j ∈ [1, n]),
∑
i∈S
j /∈S
xij ≥ 1 (S ⊂ V, S &= ∅),
x ∈ Bn×n^.
If we leave out the cut-set constraints the feasible set can only become larger, and we arrive at the assignment problem
min x
∑n
i=
∑^ n
j=
c (^) ij xij
s.t.
j:j" =i
xij = 1 (i ∈ [1, n]),
∑
i:i"=j
xij = 1 (j ∈ [1, n]),
x ∈ Bn×n^.
In later lectures we will see how the assignment problem can be solved efficiently.
6.2.4. IP duality. A conceptually different approach to obtaining dual bounds is via a dual problem, if one can be identified:
Definition 6.6. Consider an integer programming problem
(IP ) z ∗^ = max{c T^ x : x ∈ X ⊆ Zn^ },
(^1 )
3
4
5
6
7
1
2 3
4
5
6 7
Fig. 6.1. Example of an undirected graph with edge and node weights.
Fig. 6.2. The red edges form a matching.
a maximum cardinality matching
(M) max M⊆E {|M | : M is a matching}
and the problem of finding a minimum cardinality covering
(C) min R⊆V {|R| : R is a covering}.
We claim that (M) and (C) form a weak dual pair.
To prove this, let n := |V | and m := |E|, and consider the n × m node-edge incidence matrix A = (a (^) j,e ) which is defined as follows,
a (^) j,e =
1 if node j is incident to (an endpoint of ) edge e, 0 otherwise.
For example, The node-edge incidence matrix of the graph from Figure 6.1 is given by
A =
1 1 0 0 0 0 0 1 0 0 1 0 0 0 0 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1.
Using the concept of node-edge incidence matrix, (M) can now be reformulated as follows,
(M) z ∗^ = max{ 1 T^ x : Ax ≤ 1 , x ∈ Zm + },
where 1 = [ 1 ... 1 ]. To see this, lett Ai be the i-th row of A. Each component of the vector xi corresponds to an edge in the graph. Now Ai x ≤ 1 and x ∈ Zn + implies that
Fig. 6.3. The red nodes form a covering.
at most one of the xi corresponding to an edge incident to node i is nonzero, and that this nonzero entry is then equal to 1. Thus, all feasible x are binary and correspond to flag variables indicating which edges are selected, and these edges correspond to a matching.
Similarly, (C) can be reformulated as
(C) w ∗^ = min{ 1 T^ y : AT^ y ≥ 1 , y ∈ Zn + },
The argument to see this is similar, but here each component of y corresponds to a node in the graph. The LP relaxations of (M) and (C) are respectively the LP instances
(P) z LP^ = max{ 1 T^ x : Ax ≤ 1 , x ∈ R m + }, (D) w LP^ = min{ 1 T^ y : AT^ y ≥ 1 , y ∈ R n + },
which are duals of each other. By LP duality and the properties of relaxations, there- fore, for all x and y feasible for (M) and (C),
1 T^ x ≤ z ∗^ ≤ z LP^ = w LP^ ≤ w ∗^ ≤ 1 T^ y.
This shows our claim.
