Download Overview of Simplex Algorithm & Linear Programming by Jay R. Lund, UC Davis, Winter 2002 and more Study notes Civil Engineering in PDF only on Docsity!
UC Davis Winter 2002
THE SIMPLEX ALGORITHM (in one page)
The following steps form the fundamentals of the Simplex Algorithm for solving linear programs.
Set up an "initial tableau."
Is the trial solution represented by the initial tableau optimal? If all coefficients in Row 0 are ≥ 0, then the solution is optimal and you may STOP. If any coefficients in Row 0 of the tableau are < 0, the solution represented by this tableau is not the optimal solution and the set of basic variables must be changed.
Select a new basic variable. Select as a new basic variable the variable with the most negative coefficient in Row 0. The column for this variable is the "pivot column."
Select a new non-basic variable. Divide the RHS entry of each row by the coefficient in the column of the new basic variable. The new non-basic variable is the old basic variable associated with the row having the minimum ratio. This row is called the "pivot row."
Update the tableau. Form a new tableau that has zero for each entry in the pivot column, except for the coefficient in the pivot row. The entry in both the pivot row and pivot column must be equal to one. Use linear algebra and Gaussian elimination techniques to do this. Replace the old basic variable with the new basic variable in the "Basic Variable" column. This new tableau represents a new corner-point solution.
Is this new solution optimal? If all coefficients in Row 0 of the new tableau are ≥ 0, then STOP; the solution is optimal. If any coefficients in Row 0 of the tableau are < 0, the solution represented by this tableau is not the optimal solution and the set of basic variables must be changed. GO TO STEP 3.
Reading the final tableau. The following tableau represents an optimal solution and final tableau for the problem: MAX 2X1 + X Subject To: 1) X1 + X2 ≤^4
- X2 ≤ 3
Basic Eqn. Coefficient of Variable. Variable No. Z X1 X2 S1 S2 RHS Z 0 1 0 1 2 0 8 X1 1 0 1 1 1 0 4 S2 2 0 0 1 0 1 3
The values of each basic variable is given by the RHS column. The value of each non-basic variable is zero. Z = 8, X1 = 4, S2 = 3, X2 = 0, S1 = 0. The value of the Lagrange multiplier (shadow price) for each constraint is given by the coefficient in Row 0 under the associated slack variable (S1 or S2). λ1 = 2; λ2 = 0.
The Reduced Cost of each coefficient in the objective function is given by the coefficient in Row 0 under the associated decision variable (X1 or X2). Can you obtain these results?
UC Davis Winter 2002
Some Simplex Details
The basic simplex method described earlier applies only to a relatively narrow set of linear programming problems. In particular, the problem must be a MINIMIZATION problem and all constraints must be of the form Σa (^) ij X (^) i ≤ b (^) j , i.e., all constraints must be less than or equal to constraints. There are also some ambiguities in the application of the simplex method. This lecture will try to clear up these problems.
Maximization The simplex method minimizes the value of an objective function. But note:
Max z = Min (-z)
Min z = Max (-z).
So we can always make a maximization problem into a minimization problem, or vice versa.
Ties for New Basic Variable If two or more variables have the same negative coefficient in Row 0 of the tableau, and that coefficient is the most negative coefficient in Row 0, then select any variable from those with the tie. The choice is arbitrary.
Ties for New Non-Basic Variable What if 2 or more variables tie on the minimum ratio test for becoming the new non-basic variable? Break the tie arbitrarily. Possibility of cycling problems (discussed in book). Very rare.
Unbound Objective Function (Max z →→→→ ∞∞∞∞ or Min z →→→→ ∞∞∞∞ ) An unbound solution occurs if the new non-basic variable has a RHS/pivot ratio = ∞, i.e., if Min(RHS/pivot) = ∞. For this case that value of the new basic variable is not limited by any constraint and becomes infinite.
Multiple Optima Where the optima fall on two or more corner-points, the optima will also include the edges of the feasible region between these points.
These cases will appear only when, after the STOPPING criterion has been reached, at least one non- basic variable has a coefficient of zero in Row 0. If this variable is increased (from zero) it will not affect the objective function. The other corner optimal corner point solutions are found by performing extra tableau iterations.
UC Davis Winter 2002
Negative RHS Sometimes constraints have the form:
∑a (^) ij X (^) i ≤ b (^) j , where b (^) j < 0.
This can cause the initial tableau solution (all X = 0) to be infeasible. (This problem persists if we multiply both sides by negative one and reverse the inequality and becomes similar to solving a ≥ inequality constraint.) We solve this again using the Big M method.
∑a (^) ij X (^) i + Sj - Rj = b (^) j or - ∑a (^) ij X (^) i - Sj + Rj = ABS(b (^) j )
Min z = ∑c (^) i X (^) i + M Rj ., where M is very large.
No Feasible Solution This case can only occur if there are equality constraints or ≥ inequality constraints in addition to regular ≤ constraints. Therefore, artificial variables must appear in the formulation.
No Feasible Solution Two Parallel Equality Constraints No Feasible Solution
This case is discovered if the solution to the problem has at least one artificial variable not equal to zero.
Allowing Negative Values for a Decision Variable How could we let a decision variable X (^) i have any value, including negative?
Must divide X (^) i into two new decision variables: X +i and X - i. X +i will represent X (^) i if it has a positive
value and X - i will represent negative values of X (^) i.
We then replace X (^) i in all constraints and the objective function with:
X (^) i = X +i - X - i.
We can similarly add a lower bound to the negative value of X (^) i by adding another constraint, where Li is the absolute value of the lower negative bound:
X - i ≤ Li.
This ensures that X (^) i ≥ -Li.
UC Davis Winter 2002
Implementing the Big-M^ Method
We've suggested this method a lot. What does it do to the simplex method. A simple example:
Min 5 X 1 + 2 X (^2) Subject to: X 1 ≥ 1 2 X 1 + X 2 = 4 3 X 1 + 4 X 2 ≤ 30 Change this to: Min 5 X 1 + 2 X 2 + M R 1 + M R 2 Subject to: X 1 - S 1 + R 1 = 1 2 X 1 + X 2 + R 2 = 4 3 X 1 + 4 X 2 + S 3 = 30
Note that the 3rd constraint, a ≤ constraint, does not require an artificial variable.
The first initial tableau:
Basic Equation Coefficients of Variables Variable No. z X 1 X 2 S 1 S 3 R 1 R 2 RHS 0 1 5 2 0 0 M M 0 1 0 1 0 -1 0 1 0 1 2 0 2 1 0 0 0 1 4 3 0 3 4 0 1 0 0 30
Note that the "artificial variables" R 1 and R 2 are not yet basic variables, however. (We want to start at
the origin.) They need to have a coefficient of zero in Row 0. We have to fix this before we begin the simplex method, using linear algebra operations (Gaussian elimination). Multiplying Row 1 by M and subtracting it from Row 0 makes R 1 basic, and multiplying Row 2 by M and subtracting this from Row 0
makes R 2 basic. This becomes the true initial tableau. This results in:
Basic Equation Coefficients of Variables Variable No. z X 1 X 2 S 1 S 3 R 1 R 2 RHS z 0 1 5-3M 2-M M 0 0 0 -5M R 1 1 0 1 0 -1 0 1 0 1 R 2 2 0 2 1 0 0 0 1 4 S 3 3 0 3 4 0 1 0 0 30
This tableau represents the corner-point "solution" at the origin (X 1 = X 2 = 0). This pseudo-solution is penalized by its poor performance of 5M.
Ah! That's better. Now we start the simplex method using this as our initial tableau.
UC Davis Winter 2002
Sensitivity Analysis in Linear Programming
Sensitivity Analysis: How would the solution change if parameter or coefficient values are varied?
Often for real problems parameter values are not known with certainty. Thus sensitivity analysis can tell us the relative importance of different sources of uncertainty.
Sensitivity analysis is the systematic variation of parameter and input data values in a model to assess the affect of uncertainties or variation in these variables on model results and designs based on model output.
Sensitivity analysis helps answer such questions as:
- What is the effect if parameter c (^) i, b (^) j, or a (^) ij is off by some error ε?
- If input data values are accurate within some error ε, what are possible effects on model results?
- Is further research into the value of certain parameters and input data warranted?
Since many models have large numbers of parameters and input data, conducting a complete sensitivity analysis on all parameters and data, and all possible combinations of co-varied errors, is impossible. This implies that sensitivity analysis must be somewhat judicious, concentrating on those parameters and inputs that seem most important and prone to error.
Brute Force Sensitivity Analysis We could always re-solve the linear program with new parameter values. Linear programming solutions are usually quite fast, so this is often a very reasonable option. But, here are some simpler tricks! These are also discussed in Chapter 6 of Hillier and Lieberman.
Lagrange Multipliers/Shadow Prices/Dual Prices The shadow price (λ) associated with each constraint represents the change in the value of the objective function if the RHS constant of the constraint is changed by one unit.
Note: Beware the sign of these Lagrange multipliers λ. A slight change in the formulation of the equations or in the solution method will change the sign of the Lagrange multiplier, although the absolute value should remain the same. It is common for two computer solution packages to give different signs for the shadow prices. In practice, I recommend interpreting the sign intuitively and then applying the absolute value for the magnitude. It is easy to intuit the sign; changing the constraint value to "loosen" the constraint should improve the value of the objective function.
These appear as the coefficients in Row 0, under the slack variables, in the final simplex tableau.
Slack Variables The value of the slack variable represents the amount or the "resource" remaining un-used in the constraint; i.e., the constraint can be tightened by up to the value of the slack variable before the optimal solution is changed. These values are given directly by the simplex solution.
Reduced Cost The reduced cost is the amount by which the coefficient in the objective function would have to change to make a non-basic decision variable become basic (non-zero). These values are given by the coefficients in Row 0, under the decision variables, in the final simplex tableau.
Ranges in Which the Basis is Unchanged Remember, the basis is the set of variables with non-zero values.
Often, linear program output will include a "range in which the basis is unchanged." This output indicates how much particular coefficients in the objective function, RHS constants, and occasionally coefficients in the LHS of the constraints can change before one or more of the basic variables must become non-basic, and vice versa.
UC Davis Winter 2002
Within the “range of objective function coefficient values for which the Basis is unchanged”, changes in these coefficient values will change the value of the objective function, but leave the optimal solution values for the decision variables completely unchanged.
Within the “range of the RHS constants for which the Basis is unchanged”, changes in these RHS coefficient values for binding constraints will change both the solution values and the objective function value. For non-binding constraints, the range on the tightening side should be the same as the slack variable values.
UC Davis Winter 2002
How Much can Coefficients, c (^) i , bj , and a (^) ij Change Before the Solution Moves to Another
Cornerpoint?
Consider the initial tableau for the problem:
Max Z = c T^ X S.T. A X ≤ b
Eqn. No. Z X (^) i Sj RHS 0 1 -c 0 0 1 0
m A I b ↑ ↑ coefficients in the constraints Identity matrix
The final tableau will have the form: Equation No. Z X (^) i Sj RHS [y* A - c] y* I (^) Z* = y* b = c T^ X*
B * A B * I B *b
NOTE: B is the basis matrix and y is the vector of Lagrange multiplier values. The “” indicates values at the optimal solution. B * and y together summarize all operations on the initial tableau needed to create the final optimal tableau. y* is the vector of Lagrange multiplier values at the optimal solution.
How much can we change c (^) i , bj , or a (^) ij before the Basis changes?
Case 1: If the optimal solution remains at the same cornerpoint: (Basic variables remain basic) For changes in c that keep the same Basis, the solution values (and corner point) remain the same, although the Lagrange multiplier values will change.
Let b' = b + ∆b, A ' = A + ∆ A , and c' = c + ∆c, where ∆b, ∆ A , and ∆c represent changes in b, A and c, respectively.
Then:
*'
T T
X
b
S
z c X y b
B A B B
UC Davis Winter 2002
This implies
The new final tableau has the form:
Equation No. Z X (^) i X (^) n Sj Sm RHS (^0 1) y* A '^ - c '^ y* I Z* 1 0 B * A ' B * I B *b'
m
Case 2: If the optimal solution has changed corner-points: (Basic variables have changed.)
The new simplex tableau found in Case 1 will no longer satisfy the optimality test, if the optimal corner- point solution has changed.
This happens if either another corner-point is optimal or there is now no feasible solution.
Using these insights to find changes in parameter values that keep the set of basic variables the same.
Case 3: If the solution is infeasible: Some basic variable(s) will have negative values. I.e., a decision or slack variable will have a negative value. This can be quickly found because at least one element of B * b' will be negative.
UC Davis Winter 2002
The relationship between primal and dual linear programming problems is shown by writing them as (Quirino Paris, n.d.):
Primal Dual
Max Z = c 1 x 1 + c 2 x 2 + c 3 x 3 Min R = b 1 y 1 + b 2 y 2
Subject to: Subject to:
a 11 x 1 + a 12 x 2 + a 13 x 3 ≤^ b 1 y 1 ≥^0
a 21 x 1 + a 22 x 2 + a 23 x 3 ≤ b 2 y 2 ≥ 0
x 1 ≥ 0 a 11 y 1 + a 21 y 2 ≥ c (^1) x 2 ≥ 0 a 12 y 1 + a 22 y 2 ≥ c (^2) x 3 ≥ 0 a 13 y 1 + a 23 y 2 ≥ c (^) 3.
Here, y 1 and y 2 are dual variables^ for the primal constraints^ and x 1 , x 2 , x 3 , are dual variables^ for the
dual constraints. The notion that Lagrange multipliers are in essence dual variables can be illustrated by defining the Lagrangean function for both linear programming problems. The use of the Lagrangean function in this context is of great help in understanding why and how the primal and the dual LP problems are so intimately related to each other.
We now state "two" Lagrangean functions corresponding to the primal and dual LP problems and conclude that the two functions are identical and, therefore, only one Lagrangean function exists for a pair of primal and dual linear programming problems. We use y 1 , y 2 as Lagrange multipliers for the
primal problem and x 1 , x 2 , x 3 , as Lagrange multipliers for the dual.
Lagrangean for Primal Lagrangean for Dual
max L = c 1 x 1 + c 2 x 2 + c 3 x 3 min L' = b 1 y 1 + b 2 y 2
- y 1 [b 1 - a 11 x 1 - a 12 x 2 - a 13 x 3 ] + x 1 [c 1 - a 11 y 1 - a 21 y2]
- y 2 [b 2 - a 21 x 1 - a 22 x 2 - a 23 x 3 ] + x 2 [c 2 - a 12 y 1 - a 22 y 2 ]
- x 3 [c 3 - a 13 y 1 - a 23 y 2 ]
The two Lagrangean functions just stated are actually the same. In this way, the primal and the dual LP problems are tied together by a unique Lagrangean function.
Duality Example: For the Primal Problem: Find the Dual Problem: Max Z = 3X1 + 5X2 Min Z = 10Y1 + 12Y2 + 4Y Subject to: Subject to: X1 + X2 ≤ 10 Y1 + 2Y2 + Y3 ≥ 3 2X1 + X2 ≤^12 Y1 + Y2 ≥^5 X1 ≤ 4
An economic interpretation: If the primal problem is to maximize profit subject to resource
constraints, we can interpret Yj as the contribution to profit per unit of resource j. The objective
of the dual problem, then, is to minimize the total implicit value of the resources consumed.
UC Davis Winter 2002
Yj = λj = the marginal contribution to profit of resource j.
Some Primal-Dual Relationships for LP
- Weak Duality Property If both X o^ and Yo^ are complementary feasible solutions for the primal and dual problems, respectively, then:
c X° ≤ Y° b.
- Strong Duality Property If both x* and y* are complementary optimal solutions for the primal and dual problems, respectively, then:
c X* = Y* b.
Note: Computationally, we would rather have variables than constraints.
B * summarizes all the operations performed on the simplex tableau to get the final solution.
The Final Tableau: Coefs. of Variables Basic Var. Eqn. No. Z X (^) i X (^) n Sj Sm RHS Z (^0 1) [y* A '^ - c '] [y] [Z] 1 0 [ A *] [ B ] [b] . . . m
Initial Tableau: The initial tableau consists of the stacked vector and matrix,
t
T
t = vector of Row 0 in initial tableau [c | 0 | 0]
T = matrix of Rows 1 to m in the initial Tableau [ A | I | b]
Final Tableau: The final tableau consists of the stacked vector and matrix, ^
t* T *
t* = [Z* = y* A ], a consequence of operations done to Row 0
T * = [ A * | B * | b* ]
A * = B * A b* = B * b
"Fundamental Insight" (?) X (^) i coefs λs RHS t* = t + y* T = [y* A = c | y* | y* b]
T * = B * T = [ B * A | B * | B * b ]
UC Davis Winter 2002
The LP?
Min Z = 1 1 1 1
T n n n ijt ijt wit it t i j i
Z c X c W
= = = =
∑ ∑∑ ∑
Subject To
, 1 1 1
n n ijt jt jt jkt jt j t i k
X W S X D W +
= =
∑ +^ +^ =^ ∑ +^ + ∀^ j,t (conservation of mass at all locations and times)
Djt ≥ djt , ∀ j,t (satisfy demands at all locations and times) Sit ≤ s (^) it , ∀ i,t (limited sources at all locations and times) W (^) kt ≤ bkt , ∀ k,t (limited storage capacity) X (^) ijt ≤ r (^) ijt, ∀ijt (limited route capacity on each route at each time) X (^) ijt , X (^) kjt X (^) ikt ≥ 0, ∀ i ,j,k,t (non-negativity)
Number of decision variables = Tn 2 Number of constraints = (4n + n 2 )T Number of parameters to estimate (c, c (^) w , b, r, d, and s) = 2n 2 + 4n
Transportation with Warehouses and Travel Times/Delivery Lags Let: X (^) ijt = departures from i to j at time t Aijt = arrivals from i to j at time t Lijt lag or travel time from i to j departing at time t Note: X (^) ijt = Aij,t+Lijt since departures at time t arrive at time t + L (^) ijt.
The LP?
Min Z = 1 1 1 1
T n n n ijt ijt wit it t i j i
Z c X c W
= = = =
∑ ∑∑ ∑
Subject To
, 1 1 1
n n ijt jt jt jkt jt j t i k
A W S X D W +
= =
∑ +^ +^ =^ ∑ +^ + ∀^ j,t (conservation of mass at all locations and times)
X (^) ijt = Aij,t+Lijt ∀ i,t (arrivals lage departures by L (^) ijt) Djt ≥ djt , ∀ j,t (satisfy demands at all locations and times) Sit ≤ s (^) it , ∀ i,t (limited sources at all locations and times) W (^) kt ≤ bkt , ∀ k,t (limited storage capacity) X (^) ijt ≤ r (^) ijt, ∀ijt (limited route capacity on each route at each time) X (^) ijt , X (^) kjt X (^) ikt ≥ 0, ∀ i ,j,k,t (non-negativity)
This “transportation” problem is now rather flexible and applies to lots of problems. Variations of this also can apply to other logistics problems, such as water and wastewater movements – Just consider reservoirs to be warehouses and you can model California’s water system this way.
UC Davis Winter 2002
Scheduling Problems
Simple Scheduling Problems CPM and related problems. There are n activities to schedule to complete a project. Let: Si = the starting time of activity di = duration of the activity pij = 1 if activity j is a prerequisite of activity i; equals zero otherwise.
Objective: Min. time to completion = T: The LP: Min T S.T. T ≥ Si + d (^) i , i = 1, ..., n Si ≥ p (^) ij Sj + p (^) ij dj , i = 1, ..., n and j = 1, ..., n
Scheduling where Activity Duration can be shortened at a cost: Fundamentally, this is a multi-objective optimization problem (Cohon 1978). The two objectives are 1) minimize cost and 2) minimize time to completion T. A weighting factor α can be assigned to combine and weight these two objectives in the overall optimization objective function. ↓ is max duration of activity Let di = c (^) i - a (^) i X (^) i ← X (^) i = $ spent to speed completion of activity i d (^) i ≥ b (^) i ← minimum duration of activity
Min Z = X T
n
i
∑ i +^ α = 1
S.T. T ≥ Si + d (^) i , ∀ i
Si ≥ p (^) ij Sj + p (^) ij dj , ∀ i,j, i≠j di ≥ b (^) i , ∀ i di = c (^) i - a (^) i X (^) i , ∀ i
Minimize a combination of Cost and Time-duration
Weighting Method: Here, we'll weight each of the two objectives. Making several LP runs with different weights will produce an efficient trade-off curve for the two objectives.
Change objective function to:
Min (^) α (^) ∑ X (^) i i = 1
n
+ 1 - α T.
Constraint Method: Another multi-objective optimization approach is the "constraint method". This would involve solving the cost-minimizing LP several times, with each LP run having a different constrained value for the completion time T, T ≤ α. The constant α will vary from T (^) min (resulting from the completion-time minimizing formulation) to T (^) max (the cost-minimizing solution). This "constraint method" is actually a bit more rigorous.
UC Davis Winter 2002
LP Solution of Non-Linear Objectives with Linear Constraints
Sounds contradictory. But can be done for problems with objective functions of forms:
Min f(x): (convex) Max f(x):^ (concave)
o x
or f(x)
x o
f(x)
Piece-wise Linearization In each case, a linear approximation can be made of the function for specific ranges of X:
Min f(x):
f (^) a(X) ≈ f(X)
The resulting linearized non-linear program is:
Min f (^) a(X) = f(0) +
f(d 1 ) - f(0) d 1 X^1
f(d 2 ) - f(d 1 ) d 2 - d 1 X^2
f(d 3 ) - f(d 2 ) d 3 - d 2 X^3
Subject to: X = X 1 + X 2 + X (^3) X 1 ≤ d (^1) X 2 ≤^ d 2 - d (^1) X 3 ≤ d 3 - d (^2) followed by any other constraints
Note: The coefficients of X 1 , X 2 , and X 3 are just slopes of parts of the objective function!
y = f(x)
y=fa (x)
y
d1 d2 d
X 3
X 2
X 1
X
UC Davis Winter 2002
Why it works:
LP will always make X 1 = d 1 before X 2 becomes > 0. Similarly for X 2 and X 3.
To maximize this same function,
f(x)
o x
it won't work.
The LP will want to max X 3 and keep X 1 and X 2 = 0.
Maximizing Concave Objective Functions A similar linearization approach will work for maximizing a concave function subject to linear constraints.
Max f(X):
Later, we will see how any type of single-decision-variable can be linearized, with the addition of integer-linear programming.
[Need a numerical example.]
Example of Piecewise Linearization
- Minimize a convex function, Min Z = 5X 2 S.T. constraints on X
- Maximize a concave function, Max Z = 3X 0. S.T. constraints on X
y = f(x)
y y=fa (x)
d1 d2^ d^3
X 3
X 2
X 1
X
