Linear Programming: Optimal Tableau and Dual Solutions - Prof. C. W. Lee, Study notes of Mathematics

Download Linear Programming: Optimal Tableau and Dual Solutions - Prof. C. W. Lee and more Study notes Mathematics in PDF only on Docsity!

Linear Programming Notes

Carl W. Lee

Department of Mathematics

University of Kentucky

Lexington, KY 40506

[email protected]

January 3, 1996

Latest Revision: Fall 2005

i

Contents

1 References

Four good references for linear programming are

1. Dimitris Bertsimas and John N. Tsitsiklis, Introduction to Linear Optimization, Athena Scientific.
2. Vaˇsek Chv´atal, Linear Programming, W.H. Freeman.
3. George L. Nemhauser and Laurence A. Wolsey, Integer and Combinatorial Optimiza- tion, Wiley.
4. Christos H. Papadimitriou and Kenneth Steiglitz, Combinatorial Optimization: Algo- rithms and Complexity, Prentice Hall.

I used some material from these sources in writing these notes. Also, some of the exercises were provided by Jon Lee and Francois Margot. Thanks in particular to Francois Margot for many useful suggestions for improving these notes.

Exercise 1.1 Find as many errors in these notes as you can and report them to me.

2 Exercises: Linear Algebra

It is important to have a good understanding of the content of a typical one-semester un- dergraduate matrix algebra course. Here are some exercises to try. Note: Unless otherwise specified, all of my vectors are column vectors. If I want a row vector, I will transpose a column vector.

Exercise 2.1 Consider the product C = AB of two matrices A and B. What is the formula for cij , the entry of C in row i, column j? Explain why we can regard the ith row of C as a linear combination of the rows of B. Explain why we can regard the jth column of C as a linear combination of the columns of A. Explain why we can regard the ith row of C as a sequence of inner products of the columns of B with a common vector. Explain why we can regard the jth column of C as a sequence of inner products of the rows of A with a common vector. Consider the block matrices [ A B C D

] and

[ E F G H

] .

Assume that the number of columns of A and C equals the number of rows of E and F , and that the number of columns of B and D equals the number of rows of G and H. Describe the product of these two matrices.

Exercise 2.2 Associated with a matrix A are four vector spaces. What are they, how can you find a basis for each, and how are their dimensions related? Give a “natural” basis for the nullspace of the matrix [A|I], where A is an m × n matrix and I is an m × m identity matrix concatenated onto A.

Exercise 2.3 Suppose V is a set of the form {Ax : x ∈ Rk}, where A is an n × k matrix. Prove that V is also a set of the form {y ∈ Rn^ : By = O} where B is an
× n matrix, and explain how to find an appropriate matrix B. Conversely, suppose V is a set of the form {y ∈ Rn^ : By = O}, where B is an
× n matrix. Prove that V is also a set of the form {Ax : x ∈ Rk}, where A is an n × k matrix, and explain how to find an appropriate matrix A.

Exercise 2.4 Consider a linear system of equations, Ax = b. What are the various elemen- tary row operations that can be used to obtain an equivalent system? What does it mean for two systems to be equivalent?

Exercise 2.5 Consider a linear system of equations, Ax = b. Describe the set of all solutions to this system. Explain how to use Gaussian elimination to determine this set. Prove that the system has no solution if and only if there is a vector y such that yT^ A = OT^ and yT^ b = 0.

Exercise 2.13 What does it mean for a symmetric matrix A to be positive semi-definite? Positive definite? If A is positive definite, describe the set {x : xT^ Ax ≤ 1 }. What is the geometrical interpretation of the eigenvectors and eigenvalues of A with respect to this set?

Exercise 2.14 Suppose E is a finite set of vectors in Rn. Let V be the vector space spanned by the vectors in E. Let I = {S ⊆ E : S is linearly independent}. Let C = {S ⊆ E : S is linearly dependent, but no proper subset of S is linearly dependent}. Let B = {S ⊆ E : S is a basis for V }. Prove the following:

1. ∅ ∈ I.
2. If S 1 ∈ I, S 2 ∈ I, and card S 2 > card S 1 , then there exists an element e ∈ S 2 \ S 1 such that S 1 ∪ {e} ∈ I.
3. If S ∈ I and S ∪ {e} is dependent, then there is exactly one subset of S ∪ {e} that is in C.
4. If S 1 ∈ B and S 2 ∈ B, then card S 1 = card S 2.
5. If S 1 ∈ B, S 2 ∈ B, and e 1 ∈ S 1 , then there exists an element e 2 ∈ S 2 such that (S 1 \ {e 1 }) ∪ {e 2 } ∈ B.
6. If S 1 ∈ C, S 2 ∈ C, e ∈ S 1 ∩ S 2 , and e′^ ∈ S 1 \ S 2 , then there is a set S 3 ∈ C such that S 3 ⊆ (S 1 ∪ S 2 ) \ {e} and e′^ ∈ S 3.

3 Introduction

3.1 Example

Consider a hypothetical company that manufactures gadgets and gewgaws.

1. One kilogram of gadgets requires 1 hour of labor, 1 unit of wood, 2 units of metal, and yields a net profit of 5 dollars.
2. One kilogram of gewgaws requires 2 hours of labor, 1 unit of wood, 1 unit of metal, and yields a net profit of 4 dollars.
3. Available are 120 hours of labor, 70 units of wood, and 100 units of metal.

What is the company’s optimal production mix? We can formulate this problem as the linear program max z = 5x 1 + 4x 2 s.t. x 1 + 2x 2 ≤ 120 x 1 + x 2 ≤ 70 2 x 1 + x 2 ≤ 100 x 1 , x 2 ≥ 0

In matrix notation, this becomes

max

[ 5 4

] [ x 1 x 2

]

s.t.

 

 

[ x 1 x 2

] ≤

 

 

[ x 1 x 2

] ≥

[ 0 0

]

which is a problem of the form max cT^ x s.t. Ax ≤ b x ≥ O We can determine the solution of this problem geometrically. Graph the set of all points that satisfy the constraints. Draw some lines for which the objective function assumes a constant value (note that these are all parallel). Find the line with the highest value of z that has nonempty intersection with the set of feasible points. In this case the optimal solution is (30, 40) with optimal value 310.

that an infeasible linear program has a bounded feasible region.

Exercise 3.1 Graphically construct some examples of each of the following types of two- variable linear programs:

1. Infeasible.
2. With a unique optimal solution.
3. With more than one optimal solution.
4. Feasible with bounded feasible region.
5. Feasible and bounded but with unbounded feasible region.
6. Unbounded.

A linear program of the form

max

∑^ n j=

cj xj

s.t.

∑^ n

j=

aij xj ≤ bi, i = 1,... , m

xj ≥ 0 , j = 1,... , n

which, in matrix form, is max cT^ x s.t. Ax ≤ b x ≥ O

is said to be in standard form. For every linear program there is an equivalent one in standard form (begin thinking about this).

3.3 Back to the Example

Suppose someone approached the Gadget and Gewgaw Manufacturing Company (GGMC), offering to purchase the company’s available labor hours, wood, and metal, at $1.50 per hour of labor, $1 per unit of wood, and $1 per unit of metal. They are willing to buy whatever amount GGMC is willing to sell. Should GGMC sell everything? This is mighty

tempting, because they would receive $350, more than what they would gain by their current manufacturing plan. However, observe that if they manufactured some gadgets instead, for each kilogram of gadgets they would lose $4.50 from the potential sale of their resources but gain $5 from the sale of the gadgets. (Note, however, that it would be better to sell their resources than make gewgaws.) So they should not accept the offer to sell all of their resources at these prices.

Exercise 3.2 In the example above, GGMC wouldn’t want to sell all of their resources at those prices. But they might want to sell some. What would be their best strategy?

Exercise 3.3 Suppose now that GGMC is offered $3 for each unit of wood and $1 for each unit of metal that they are willing to sell, but no money for hours of labor. Explain why they would do just as well financially by selling all of their resources as by manufacturing their products.

Exercise 3.4 In general, what conditions would proposed prices have to satisfy to induce GGMC to sell all of their resources? If you were trying to buy all of GGMC’s resources as cheaply as possible, what problem would you have to solve?

Exercise 3.5 If you want to purchase just one hour of labor, or just one unit of wood, or just one unit of metal, from GGMC, what price in each case must you offer to induce GGMC to sell?

5. Vary the right hand sides and consider the following linear program:

max z = x 1 + 2x 2 s.t. 3x 1 + x 2 ≤ b 1 x 1 + x 2 ≤ b 2 x 1 , x 2 ≥ 0

Carefully and completely describe the optimal value z∗(b 1 , b 2 ) as a function of the pair (b 1 , b 2 ). What kind of function is this? Optional: Use some software such as Maple to plot this function of two variables.

6. Find the best nonnegative integer solution to (P ). That is, of all feasible points for (P ) having integer coordinates, find the one with the largest objective function value.

Exercise 4.2 Consider the following linear program (P ):

max z = −x 1 − x 2 s.t. x 1 ≤ 1 / 2 (1) x 1 − x 2 ≤ − 1 / 2 (2) x 1 ≥ 0 (3) x 2 ≥ 0 (4)

Answer the analogous questions as in Exercise 4.1.

Exercise 4.3 1. Consider the following linear program (P ):

max z = 2x 1 + x 2 s.t. x 1 ≤ 2 (1) x 2 ≤ 2 (2) x 1 + x 2 ≤ 4 (3) x 1 − x 2 ≤ 1 (4) x 1 ≥ 0 (5) x 2 ≥ 0 (6)

Associated with each of the 6 constraints is a line (change the inequality to equality in the constraint). Consider each pair of constraints for which the lines are not parallel, and examine the point of intersection of the two lines. Call this pair of constraints a primal feasible pair if the intersection point falls in the feasible region for (P ). Call

this pair of constraints a dual feasible pair if the gradient of the objective function can be expressed as a nonnegative linear combination of the two outer normal vectors of the two constraints. (The movitation for this terminology will become clearer later on.) List all primal-feasible pairs of constraints, and mark the intersection point for each pair. List all dual-feasible pairs of constraints (whether primal-feasible or not), and mark the intersection point for each pair. What do you observe about the optimal point(s)?

2. Repeat the above exercise for the GGMC problem.
   

Exercise 4.4 We have observed that any two-variable linear program appears to fall into exactly one of three categories: (1) those that are infeasible, (2) those that have unbounded objective function value, and (3) those that have a finite optimal objective function value. Suppose (P ) is any two-variable linear program that falls into category (1). Into which of the other two categories can (P ) be changed if we only alter the right hand side vector b? The objective function vector c? Both b and c? Are your answers true regardless of the initial choice of (P )? Answer the analogous questions if (P ) is initially in category (2). In category (3).

Exercise 4.5 Find a two-variable linear program

(P )

max cT^ x s.t. Ax ≤ b x ≥ O

with associated integer linear program

(IP )

max cT^ x s.t. Ax ≤ b x ≥ O and integer

such that (P ) has unbounded objective function value, but (IP ) has a finite optimal objective function value. Note: “x integer” means that each coordinate xj of x is an integer.

Exercise 4.6 Prove the following: For each positive real number d there exists a two- variable linear program (P ) with associated integer linear program (IP ) such that the entries of A, b, and c are rational, (P ) has a unique optimal solution x∗, (IP ) has a unique optimal solution x∗, and the Euclidean distance between x∗^ and x∗^ exceeds d. Can you do the same with a one-variable linear program?

5 Solving Linear Programs

5.1 Matrices

Suppose A is an m × n matrix, B is an n × p matrix, and C = AB. Then

cik =

∑^ n

j=

aij bjk, i = 1,... , m, k = 1,... , p.

We can recognize this as the inner product of the ith row of A with the kth column of B:

cik = [ai 1 ,... , ain]

  

b 1 k .. . bnk

  .

We can also see that the ith row of C is a linear combination of the rows of B using as coefficients the entries in the ith row of A:

[ci 1 ,... , cip] = ai 1 [b 11 ,... , b 1 p] + · · · + ain [bn 1 ,... , bnp] ,

and the kth column of C is a linear combination of the columns of A using as coefficients the entries in the kth column of B:   

c 1 k .. . cmk

   =^ b^1 k

  

a 11 .. . am 1

   +^ · · ·^ +^ bnk

  

a 1 n .. . amn

  .

5.2 Four Vector Spaces Associated with a Matrix

Definition 5.1 Let A be an m × n matrix. The four vector spaces associated with A are:

1. The column space of A. This is the space of all linear combinations of the columns of A, columnspace(A) = {Ax : x ∈ Rn}.
2. The row space of A. This is the space of all linear combinations of the rows of A, rowspace(A) = {yT^ A : y ∈ Rm}.
3. The nullspace of A, nullspace(A) = {x ∈ Rn^ : Ax = O}.
4. The left nullspace of A, leftnullspace(A) = {y ∈ Rm^ : yT^ A = OT^ }.

To find bases for each of the four spaces, perform Gaussian elimination on A to obtain a matrix A′^ in row-reduced form. By multiplying the matrices corresponding to the various row operations, determine the square invertible matrix M such that M A = A′. The leading nonzero entries in each nonzero row of A′^ are called the pivot entries. The rows in which they appear are called the pivot rows, and the columns in which they appear are called the pivot columns. You should be able to verify the following assertions:

1. To obtain a basis for the column space of A select the columns of A corresponding to the pivot columns of A′. Note that A and A′^ do not necessarily have the same column space, but their column spaces have the same dimension, namely, the number of pivot entries.
2. The nonzero rows of A′^ form a basis for the row space of A. Hence they have the same dimension, namely, the number of pivot entries.
3. Matrices A and A′^ have the same nullspace. There is one basis vector for each nonpivot column A′ s of A′: Set ws = 1 and wj = 0 for all other nonpivot columns Aj. Then determine the unique multipliers wj for the pivot columns of A′ j to solve A′w = O. So the dimension of the nullspace equals the number of nonpivot columns.
4. Suppose the zero rows of A′^ are precisely the last k rows of A′. Then the last k rows of M form a basis for the left nullspace of A. In particular, the dimension of the left nullspace equals the number of nonpivot rows.

As an immediate consequence we have:

Theorem 5.

1. The dimension of the row space equals the dimension of the column space. This number is also called the rank of A.
2. The dimension of the column space plus the dimension of the nullspace equals the number of columns.
3. The dimension of the row space plus the dimension of the left nullspace equals the number of rows.

Definition 5.6 Associated with a digraph having no loops is the vertex-edge incidence ma- trix A. Rows are indexed by vertices, columns by edges. The entry in row v column e is − 1 if v is the tail of e, +1 if v is the head of e, and zero otherwise. (Note that other texts may interchange −1 and 1 in the definition.)

Theorem 5.7 For a digraph, the dimension of the left nullspace of A equals the number of components of G.

Proof. An element of the left nullspace is an assignment of numbers yv to vertices v such that for every edge e = uv, yv − yu = 0; i.e., yv = yu. By connectivity, the same number must be assigned to every vertex in a given component. But different components may have vertices with different numbers.
For a digraph, an element of the nullspace of A corresponds to an assignment of numbers xe to edges e ∈ E(G) such that for every vertex v,

∑

e∈δ−(v)

xe −

∑

e∈δ+(v)

xe = 0.

We are using the notation δ+(v) to denote the set of edges whose tails are v (the set of edges leaving v), and δ−(v) to denote the set of edges whose heads are v (the set of edges entering v). That is to say, for every vertex v, the sum of the numbers on edges entering v equals the sum of the numbers on edges leaving v. The above equations are known as the flow-conservation equations because by interpreting the numbers on the edges as flows, the equations state that what flows in at each vertex must flow out. (Regard negative flow as positive flow in the opposite direction.)

Theorem 5.8 If a subset of edges contains a cycle, then the corresponding subset of columns of A is dependent.

Proof. Trace the cycle in some direction, assigning +1 to edgess traversed in the forward direction, and −1 to edges traversed in the reverse direction.

Theorem 5.9 If a subset S of columns of A is dependent, then the corresponding subset of edges contains a cycle.

Proof. Let x be a nonzero solution to Ax = O. Without loss of generality assume that xe = 0 precisely when e ∈ S. ¿From the flow-conservations equations we can deduce that there can be no twig in the subgraph of G determined by the edges in S. So the set of edges of S cannot be a forest, and hence contains a cycle.

Exercise 5.10 Prove that the following are equivalent for a digraph G with at least one edge. Try using some of the properties of the dimensions of the vector spaces associated with the vertex-edge incidence matrix A of G.

1. G is a tree.
2. G is minimally connected; i.e., G is connected, but no subgraph with the same vertex set and fewer edges is connected.
3. G is maximally acyclic; i.e, G is acyclic, but no supergraph with the same vertex set and more edges is acyclic.
4. |V (G)| = |E(G)| + 1 and G is connected.
5. |V (G)| = |E(G)| + 1 and G is acyclic.

Exercise 5.11 Let A be the vertex-edge incidence matrix of a digraph G with at least one edge. Let M be any square submatrix of A, determined by selecting any sets of equal numbers of rows and columns of A, not necessarily adjacent. Prove that the determinant of M is 0, 1, or −1. Suggestion: Recall how to calculate a determinant by expansion along a column.

5.4 Systems of Equations

Definition 5.12 Let A be an m × n matrix. For a subset B ⊆ { 1 ,... , n} we let AB denote the submatrix of A consisting only of those columns indexed by B. Similarly, if x ∈ Rn^ we let xB denote the subvector of x consisting only of those components of x indexed by B.

Given an m × n matrix A and a vector b ∈ Rm^ we can determine whether or not b is in the column space of A and, if so, find a particular solution x to Ax = b using Gaussian elimination. The solution x will satisfy AB xB = b and xN = O, where N = { 1 ,... , n} \ B. We can also find a basis for {w^1 ,... , wp} for the nullspace of A. Then the complete set of solutions to Ax = b is given by

{x + t 1 w^1 + · · · + tpwp^ : t 1 ,... , tp ∈ R}.

Actually, we can do the above for any set B of indices of columns forming a basis for the column space of A. Such a solution x is called the basic solution associated with B, and the vectors w^1 ,... , wp^ are called the basic directions associated with B.