Download Linear Programming: Maximizing Benefits for Penguins through LP and more Assignments Algorithms and Programming in PDF only on Docsity!
21: Linear Programming
Sariel Har-Peled
April 19, 2006
473G - Algorithms
1 Introduction and Motivation
In the VCR/guns/nuclear-bombs/napkins/star-wars/professors/butter/mice problem, the benevolent dictator, Biga Piguinus, of Penguina (a country in south Antarctica having 24 million penguins under its control) has to decide how to allocate her empire resources to the maximal benefits of her penguins. In particular, she has to decide how to allocate the money for the next year budget. For example, buying a nuclear bomb has a tremendous positive effect on security (the ability to destruct yourself completely together with your enemy is considered to induce a peaceful security feeling in most people). Guns on the other hand has lesser effect. Penguina (the state) has to supply a certain level of security. Thus, the allocation should be such that:
xgun + 1000 ∗ xnuclear−bomb ≥ 1000 ,
where xguns is the number of guns constructed, and xnuclear−bomb is the number of nuclear-bombs constructed. On the other hand, 1000 ∗ xgun + 1000000 ∗ xnuclear−bomb ≤ xsecurity
where xsecurity is the total Penguina is willing to spend on security, and 1000 is the price of producing a single gun, and 1000000 is the price of manufacturing one nuclear bomb. There are a lot of other constrains of this type, and Biga Piguinus would like to solve them, while minimizing the total money allocated for such spending (the less spent on budget, the larger the tax cut).
a 11 x 1 +... + a 1 n xn ≤ b 1 a 21 x 1 +... + a 2 n xn ≤ b 2
... am 1 x 1 +... + amn xn ≤ bm max c 1 x 1 +... + cn xn.
More formally, we have a (potentially large) number of variables: x 1 ,... , xn and a (potentially large) system of linear inequalities. We will refer to such an inequality as constraint. We would like to decide if there is an as- signment of values to x 1 ,... , xn where those inequalities are satisfied. Since there might be infinite number of such solutions, we decide that we want the solution that maximizes the quantity. See the instance on the right. The linear target function we are trying to maximize is known as the objective function of the linear program. Such a problem is an instance of linear programming. We refer to linear programming as LP.
1.1 History
Linear programming can be traced back to the early 19th century. It started in earnest in 1939 when L. V. Kantorovich noticed the importance of certain type of Linear Programming problems. Unfortunately, for several years, Kantorovich work was unknown in the west and unnoticed in the east. Dantzig, in 1947, invented the simplex method for solving LP problems for the US Air force planning problems. T. C. Koopmans, in 1947, showed that LP provide the right model for the analysis of classical economic theories. In 1975, both Koopmans and Kantorovich got the Nobel prize of economics. Dantzig probably did not get it because his work was too mathematical.
1.2 Network flow via linear programming
To see the impressive expression power of linear programming, we next show that network flow can be solved using linear programming. Thus, we are given an instance of max flow; namely, a network flow G = (V, E) with source s and sink t, and capacities c(·) on the edges. We would like to compute the maximum flow in G.
∀ (u → v) ∈ E 0 ≤ xu→v xu→v ≤ c(u → v)
∀v ∈ V \ {s, t}
(u→v)∈E
xu→v −
(v→w)∈E
xv→w ≤ 0 ∑
(u→v)∈E
xu→v −
(v→w)∈E
xv→w ≥ 0
maximizing
(s→u)∈E xs→u
To this end, for an edge (u → v) ∈ E, let xu→v be a variable which is the amount of flow assign to (u → v) in the maximum flow. We demand that 0 ≤ xu→v and xu→v ≤ c(u → v) (flow is non nega- tive on edges, and it comply with the capacity con- straints). Next, for any vertex v which is not the source or the sink, we require that
∑ (u→v)∈E^ xu→v^ = (v→w)∈E xv→w^ (this is conservation of flow). (Note, that an equality constraint a = b can be rewritten as two inequality constraints a ≤ b and b ≤ a.) Finally, under all these constraints, we are interest in the maximum flow. Namely, we would like to maximize the quantity
(s→u)∈E xs→u. Clearly, putting all this constraints together, we get the linear program depicted on the right.
2 The Simplex Algorithm via Geometry
Let us assume, that we have an instance of linear programming with n variables and m constraints. An assignment of values to the variables of x 1 = α 1 ,... , xn = αn, can be interpreted as a point (α 1 ,... , αn) ∈ IRn. As such, an inequality defines a region of space where the inequality holds, which is a half-space.
3 y + 2x ≤ 6
To realize what is such a half-space, consider the inequality 3y + 2 X ≤ 6. This in- equality holds with equality for 3y+ 2 x = 6, which is just a line in the two dimensional space. As such, the set of points that comply with this inequality are all the points in the plane below (or on) the line 3y + 2 x = 6. See figure on the right. In higher dimen- sions, the boundary of the feasible region of a single inequality is a half-space, with a boundary which is a hyperplane, where all the points on the hyperplane, are the points where quality holds.
Definition 2.1 A set S ⊆ IRn^ is convex, if for any two points x, y ∈ S , we have that the segment xy ⊆ S.
Observe, that a half-space is a convex region. Observe, that the intersection of a finite number of convex regions is convex (prove it). As such, the set of points in IRn^ that satisfy all the constraints of the linear program, referred to as the feasible regionlinear programming!feasible region, is the intersection of half- spaces, which is thus convex. This type of a convex region is known as a polytope; this is a convex region that its boundary is made out of finite number of linear surfaces. An example of a polytope in two dimensions is the region inside a triangle, and in three dimensions, consider a cube. The interesting features of the polytopes are the verticeslinear programming!vertices. A vertex is a point defined by n inequalities that hold at this point with equality. We need the following fact.
Lemma 2.2 Given n linear equalities (in general position) defined over n variables, one can find the point that realizes all these equalities in O(n^3 ) time.
Proof: This is solved using standard linear algebra techniques. In particular, by using Graham-Schmidt orthogonalization. This takes O(n^3 ) for a matrix of size n × n.
A naive algorithm. Lemma 2.4 implies that we can restrict ourselves to the vertices of the feasible poly- tope associated with the given LP. In particular, given an LP defined over n variables, with m inequalities, we can brute force enumerate all n tuples of inequalities. For each such tuple T , we can use Lemma 2.2, to compute the vertex vT defined by this set of inequalities. Next, we scan the inequalities in the given LP, and check whether or not the point vT complies with the inequality. If vT is feasible for all the inequalities, then it must be a vertex of the feasible polytope, and as such we compute the value of the objective function for vT. We return the best such vertex found, and by Lemma 2.4 we know that we found an optimal solution. This is a naive algorithm that takes O
((m n
n^3 + mn
= O
mn+^1 n^4
time, which is not very satisfactory since this running time bound is tight. We would prefer to have a faster algorithm that terminates faster if the given instance is “easier”.
2.1 The Simplex algorithm
Consider a vertex v of the polytope P, and let Dv be the set of inequalities that realizes v with equality, where P is the feasible polytope for the given LP. By our general position assumption, there are exactly n inequalities in Dv. Consider a subset X ⊆ Dv of size n − 1. The intersection of the n − 1 hyperplanes induced by the inequalities of X form a line . Let h be the singleton inequality of Dv \ X. Clearly, the intersection of with h is an infinite ray, where the source of the ray is the point v. Now, if we travel (a small enough distance) along this ray from v, we are still inside the feasible region (if not, there must be n + 1 inequalities that hold with equality at v, which is not possible by our general position assumption). Since we are assuming that P is bounded, it must be that this ray hits a new hyperplane induced by an inequality h′^ of the LP. Let u be the intersection point of the ray with this hyperplane. Clearly, the segment s = [v, u] is feasible and lie completely on the boundary of P (since s lies on the hyperplanes induced by the inequalities of X). As such, u is a vertex of P induced by the set of inequalities X ∪ {h′}. Since there are
( (^) n n− 1
= n subsets of size n − 1 of Dv, we conclude that v is connected to n other vertices of P. So, let us build an (implicit) graph G over the vertices of P. Two vertices v and u are connected if they are induced by two sets of n inequalities that differ in only one inequality. Clearly, given a vertex v, we can easily compute (in polynomial time) all its neighbors in G (by just computing the n rays emanating from v, and computing for every ray its other endpoint). Every vertex of G has an associate objective function value, which can be computed by substituting the coordinates of the vertex into the objective function. Thus, assume that we have a vertex v of the feasible polytope P. We compute all the neighbors of v in G and their values. If any of them improves over the objective value of v, then we set the new vertex to be the current vertex v. We repeat this process (i.e., walk on the vertices of G) till we reach a vertex which has objective value better than all its neighbors. We stop, and return it as the solution to the LP.
p (^) q
Figure 1: The simplex algorithm works by walking on the vertices of the feasible polytope, moving from a vertex to one of its neighbors that has a better objective value. In the de- picted case, the algorithm moves from p to q.
This is essentially the Simplex algorithm (we ignored some significant technicalities which we will fill-in later). We need to prove that once we get “stuck” there is no better feasible solution. Furthermore, we need to describe how we find the starting feasible vertex of P. This are two technical issues that we will resolve later when given a more algebraic description of the Simplex algorithm. The above description implies that the Simplex algorithm per- forms an implicit walk over the vertices of the feasible region, till it get stuck in the (global) minimum (assuming the task is to min- imize the objective function). The problem is that the number of vertices we might visit in the worst case is ≤
(m n
. There is cur- rently no significantly better bound known. In fact, Hirsch in 1957 conjectured that the diameter of G is only m − n (i.e., there is a path in G between any two vertices of length at most n−m). This is still an open problem (!), but even sig- nificant progress on this conjecture will not imply that any (known) variant of the Simplex algorithm run in polynomial time, since a polynomial time algorithm would need to find this “short” path. In fact, all currently known Simplex type algorithm have bad inputs for which the require exponential time in n. Nevertheless, in practice the Simplex performs reasonably well on most inputs.
