Download Linear Programming-Data Representation And Algorithm Design-Lecture Slides and more Slides Data Representation and Algorithm Design in PDF only on Docsity!
Linear Programming
2
Linear Programming
What is it?
! Quintessential tool for optimal allocation of scarce resources,
among a number of competing activities.
! Powerful and general problem-solving method that encompasses:
- shortest path, network flow, MST, matching
- Ax = b, 2-person zero sum games
Why significant?
! Fast commercial solvers: CPLEX, OSL.
! Powerful modeling languages: AMPL, GAMS.
! Ranked among most important scientific advances of 20
th
century.
! Widely applicable and dominates world of industry.
see ORF 307
Ex: Delta claims saving $100 million per year using LP
3
Applications
Agriculture. Diet problem.
Computer science. Compiler register allocation, data mining.
Electrical engineering. VLSI design, optimal clocking.
Energy. Blending petroleum products.
Economics. Equilibrium theory, two-person zero-sum games.
Environment. Water quality management.
Finance. Portfolio optimization.
Logistics. Supply-chain management.
Management. Hotel yield management.
Marketing. Direct mail advertising.
Manufacturing. Production line balancing, cutting stock.
Medicine. Radioactive seed placement in cancer treatment.
Operations research. Airline crew assignment, vehicle routing.
Physics. Ground states of 3 - D Ising spin glasses.
Plasma physics. Optimal stellarator design.
Telecommunication. Network design, Internet routing.
Sports. Scheduling ACC basketball, handicapping horse races.
4
Brewery Problem: A Toy LP Example
Small brewery produces ale and beer.
! Production limited by scarce resources: corn, hops, barley malt.
! Recipes for ale and beer require different proportions of resources.
How can brewer maximize profits?
! Devote all resources to ale: 34 barrels of ale! $442.
! Devote all resources to beer: 32 barrels of beer! $736.
! 7.5 barrels of ale, 29.5 barrels of beer! $776.
! 12 barrels of ale, 28 barrels of beer! $800.
Beverage
Corn
(pounds)
Malt
(pounds)
Hops
(ounces)
Beer (barrel) 15 4 20
Ale (barrel) 5 4 35
Profit
($)
23
13
Limit 480 160 1190
5
Brewery Problem
max 13 A + 23 B
s. t. 5 A + 15 B " 480
4 A + 4 B " 160
35 A + 20 B " 1190
A , B # 0
Ale Beer
Corn
Hops
Malt
Profit
6
Brewery Problem: Feasible Region
Ale
Beer
( 34 , 0 )
( 0 , 32 )
Corn
5 A + 15 B " 480
Hops
4 A + 4 B " 160
Malt
35 A + 20 B " 1190
( 12 , 28 )
( 26 , 14 )
( 0 , 0 )
7
Brewery Problem: Objective Function
13 A + 23 B = $ 800
13 A + 23 B = $ 1600
13 A + 23 B = $ 442
( 34 , 0 )
( 0 , 32 )
( 12 , 28 )
( 26 , 14 )
( 0 , 0 )
P
r o
f
i t
Ale
Beer
8
( 34 , 0 )
( 0 , 32 )
( 12 , 28 )
( 0 , 0 )
( 26 , 14 )
Brewery Problem: Geometry
Brewery problem observation. Regardless of objective function
coefficients, an optimal solution occurs at an extreme point.
extreme point
Ale
Beer
13
Simplex Algorithm
Simplex algorithm. [George Dantzig, 1947]
! Developed shortly after WWII in response to logistical problems,
including Berlin airlift.
! One of greatest and most successful algorithms of all time.
Generic algorithm.
! Start at some extreme point.
! Pivot from one extreme point to a neighboring one.
! Repeat until optimal.
How to implement? Linear algebra.
never decreasing objective function
14
Simplex Algorithm: Basis
Basis. Subset of m of the n variables.
Basic feasible solution (BFS). Set n - m nonbasic variables to 0,
solve for remaining m variables.
! Solve m equations in m unknowns.
! If unique and feasible solution! BFS.
! BFS # extreme point.
Ale
Beer
Basis
{A, B, S M
}
(12, 28)
{A, B, S C
}
(26, 14)
{B, S H
, S M
}
(0, 32)
{S H
, S M
, S C
}
(0, 0)
{A, S
H
, S
C
}
(34, 0)
!
max 13 A + 23 B
s. t. 5 A + 15 B + S
C
= 480
4 A + 4 B + S
H
= 160
35 A + 20 B + S M
= 1190
A , B , S
C
, S
H
, S
M
" 0
Infeasible
{A, B, S H
}
(19.41, 25.53)
15
Simplex Algorithm: Initialization
Basis = {S
C
, S
H
, S
M
A = B = 0
Z = 0
S
C
S
H
S
M
max Z subject to
13 A + 23 B " Z = 0
5 A + 15 B + S
C
4 A + 4 B + S
H
35 A + 20 B + S
M
A , B , S
C
, S
H
, S
M
16
max Z subject to
16
3
A "
23
15
S
C
" Z = " 736
1
3
A + B +
1
15
S
C
8
3
A "
4
15
S
C
+ S
H
85
3
A "
4
3
S
C
+ S
M
A , B , S
C
, S
H
, S
M
Basis = {B, S
H
, S
M
A = S
C
Z = 736
B = 32
S
H
S
M
Simplex Algorithm: Pivot 1
Substitute: B = 1/15 (480 – 5A – S
C
Basis = {S
C
, S
H
, S
M
A = B = 0
Z = 0
S
C
S
H
S
M
max Z subject to
13 A + 23 B " Z = 0
5 A + 15 B + S
C
4 A + 4 B + S
H
35 A + 20 B + S
M
A , B , S
C
, S
H
, S
M
17
Simplex Algorithm: Pivot 1
Why pivot on column 2?
! Each unit increase in B increases objective value by $23.
! Pivoting on column 1 also OK.
Why pivot on row 2?
! Preserves feasibility by ensuring RHS $ 0.
! Minimum ratio rule: min { 480/15, 160/4, 1190/20 }.
max Z subject to
13 A + 23 B " Z = 0
5 A + 15 B + S
C
4 A + 4 B + S
H
35 A + 20 B + S
M
A , B , S
C
, S
H
, S
M
Basis = {S
C
, S
H
, S
M
A = B = 0
Z = 0
S
C
S
H
S
M
18
Simplex Algorithm: Pivot 2
max Z subject to
16
3
A "
23
15
S
C
" Z = " 736
1
3
A + B +
1
15
S
C
8
3
A "
4
15
S
C
+ S
H
85
3
A "
4
3
S
C
+ S
M
A , B , S
C
, S
H
, S
M
max Z subject to
" S
C
" 2 S
H
" Z = " 800
B +
1
10
S
C
1
8
S
H
A "
1
10
S
C
3
8
S
H
25
6
S
C
85
8
S
H
+ S
M
A , B , S
C
, S
H
, S
M
Substitute: A = 3/8 (32 + 4/15 S
C
H
Basis = {B, S
H
, S
M
A = S
C
Z = 736
B = 32
S
H
S
M
Basis = {A, B, S
M
S
C
= S
H
Z = 800
B = 28
A = 12
S
M
19
Simplex Algorithm: Optimality
Q. When to stop pivoting?
A. When all coefficients in top row are non-positive.
Q. Why is resulting solution optimal?
A. Any feasible solution satisfies system of equations in tableaux.
! In particular: Z = 800 – S
C
H
! Thus, optimal objective value Z* " 8 00 since S
C
, S
H
! Current BFS has value 800! optimal.
Basis = {A, B, S
M
S
C
= S
H
Z = 800
B = 28
A = 12
S
M
max Z subject to
" S
C
" 2 S
H
" Z = " 800
B +
1
10
S
C
1
8
S
H
A "
1
10
S
C
3
8
S
H
25
6
S
C
85
8
S
H
+ S
M
A , B , S
C
, S
H
, S
M
20
Simplex Algorithm: Bare Bones Implementation
Construct the simplex tableaux.
A
c
I b
0 0
public class Simplex {
private double[][] a; // simplex tableaux
private int M, N;
public Simplex(double[][] A, double[] b, double[] c) {
M = b.length;
N = c.length;
a = new double[M+ 1 ][M+N+ 1 ];
for (int i = 0 ; i < M; i++)
for (int j = 0 ; j < N; j++)
a[i][j] = A[i][j];
for (int j = N; j < M + N; j++) a[j-N][j] = 1. 0 ;
for (int j = 0 ; j < N; j++) a[M][j] = c[j];
for (int i = 0 ; i < M; i++) a[i][M+N] = b[i];
m
1
n n 1
25
Simplex Algorithm: Implementation Issues
Implementation issues.
! Avoid stalling.
! Choosing the pivot.
! Numerical stability.
! Maintaining sparsity.
! Detecting infeasiblity
! Detecting unboundedness.
! Preprocessing to reduce problem size.
Commercial solvers routinely solve LPs with millions of variables and
tens of thousands of constraints.
requires fancy data structures
26
set PROD := beer ale;
set INGR := corn hops malt;
param: profit :=
ale 13
beer 23;
param: supply :=
corn 480
hops 160
malt 1190;
param amt: ale beer :=
corn 5 15
hops 4 4
malt 35 20;
LP Solvers
AMPL. [Fourer, Gay, Kernighan] An algebraic modeling language.
CPLEX solver. Industrial strength solver.
set INGR;
set PROD;
param profit {PROD};
param supply {INGR};
param amt {INGR, PROD};
var x {PROD} >= 0;
maximize total_profit:
sum {j in PROD} x[j] * profit[j];
subject to constraints {i in INGR}:
sum {j in PROD} amt[i,j] * x[j] <= supply[i];
beer.dat
beer.mod
[cos226:tucson] ~> ampl
AMPL Version 20010215 (SunOS 5.7)
ampl: model beer.mod;
ampl: data beer.dat;
ampl: solve;
CPLEX 7.1.0: optimal solution; objective 800
ampl: display x;
x [*] := ale 12 beer 28;
separate data from model
27
LP Duality: Economic Interpretation
Brewer's problem. Find optimal mix of beer and ale to maximize profits.
Entrepreneur's problem. Buy resources from brewer at min cost.
! C, H, M = unit price for corn, hops, malt.
! Brewer won't agree to sell resources if 5C + 4H + 35M < 13.
(P) max 13 A + 23 B
s. t. 5 A + 15 B " 480
4 A + 4 B " 160
35 A + 20 B " 1190
A , B # 0
(D) min 480 C + 160 H + 1190 M
s. t. 5 C + 4 H + 35 M " 13
15 C + 4 H + 20 M " 23
C , H , M " 0
A* = 12
B* = 28
OPT = 800
C* = 1
H* = 2
M* = 0
OPT = 800
28
Primal and dual LPs. Given real numbers a
ij
, b
i
, c
j
, find real numbers x
j
, y
i
that optimize (P) and (D).
Duality Theorem. [Gale-Kuhn-Tucker 1951, Dantzig-von Neumann 1947]
If (P) and (D) have feasible solutions, then max = min.
(P) max c
j
x
j
j = 1
n
s. t. a
ij
x
j
j = 1
n
b
i
1 # i # m
x
j
$ 0 1 # j # n
(D) min b i
y i
i = 1
m
s. t. a
ij
y
i
i = 1
m
c
j
1 $ j $ n
y i
0 1 $ i $ m
LP Duality
29
LP Duality: Sensitivity Analysis
Q. How much should brewer be willing to pay (marginal price) for
additional supplies of scarce resources?
A. Corn $1, hops $2, malt $0.
Q. How do I compute marginal prices (dual variables)?
A. Simplex solves primal and dual simultaneously.
Q. New product "light beer" is proposed. It requires 2 corn, 5 hops,
24 malt. How much profit must be obtained from light beer to justify
diverting resources from production of beer and ale?
A. Breakeven: 2 ($1) + 5 ($2) + 24 ($0) = $12 / barrel.
objective row of final simplex tableaux
provides optimal dual solution!
30
History
1939. Production, planning. [Kantorovich]
1947. Simplex algorithm. [Dantzig]
1950. Applications in many fields.
1975. Nobel prize in Economics. [Kantorovich and Koopmans]
1979. Ellipsoid algorithm. [Khachian]
1984. Projective scaling algorithm. [Karmarkar]
1990. Interior point methods.
200x. Approximation algorithms, large scale optimization.
31
Simplex vs. Interior Point Methods
interior point faster when polyhedron
smooth like disco ball
simplex faster when polyhedron
spiky like quartz crystal
32
Ultimate problem-solving model?
! Shortest path.
! Maximum flow.
! Assignment problem.
! Min cost flow.
! Multicommodity flow.
! Linear programming.
! Semidefinite programming.
! Integer programming (or any NP-complete problem).
Does P = NP? No universal problem-solving model exists unless P = NP.
tractable
Ultimate Problem Solving Model
intractable (conjectured)
