Download Implicit Enumeration: An Example of Branch and Bound Algorithm in Linear Programming and more Study notes Applied Mathematics in PDF only on Docsity!
8/11/2009 Vira ChankongEECS, CWRU 1
Numerical Optimization
Instructor: Vira Chankong
Electrical Engineering and Computer Science
Case Western Reserve University
Phone: 216 368 4054, Fax: 216 368 3123
A Workshop
At
Department of Mathematics
Chiang Mai University
August 4-15, 2009
8/11/2009 Vira ChankongEECS, CWRUVira ChankongEECS. CWRU 22
Session:
Discrete and Combinatorial
Optimization
Vira Chankong Vira Chankong
Case Western Reserve UniversityCase Western Reserve University Electrical Engineering and Computer ScienceElectrical Engineering and Computer Science
8/11/2009 Vira ChankongEECS, CWRU 3
NEOS Guide Optimization Tree
8/11/2009 Vira ChankongEECS, CWRU 4
Discrete Optimization Problem: An optimization problem
in which some or all the variables are discrete.
An integer programming problem (IP): A linear or
nonlinear programming problem in which all variables are
nonnegative integers
LIP: Linear IP
- Pure LIP---all variables are integers
- Mixed LIP--some variables are integers and
some are continuous
- Pure Binary IP– LIP with all variables are binary
- Mixed Binary IP-- some variables are binary and
some are continuous and/or integers
Discrete Optimization ProblemsDiscrete Optimization Problems
8/11/2009 Vira ChankongEECS, CWRU 7
Suppose that you are organizing housing accommodations for a group of four hundred university students. Space is limited and only one hundred of the students will receive places in the dormitory. To complicate matters, the Dean has provided you with a list of pairs of incompatible students, and requested that no pair from this list appear in your final choice. This is an example of what computer scientists call an NP-problem, since it is easy to check if a given choice of one hundred students proposed by a coworker is satisfactory (i.e., no pair taken from your coworker's list also appears on the list from the Dean's office), however the task of generating such a list from scratch seems to be so hard as to be completely impractical. Indeed, the total number of ways of choosing one hundred students from the four hundred applicants is greater than the number of atoms in the known universe! Thus no future civilization could ever hope to build a supercomputer capable of solving the problem by brute force; that is, by checking every possible combination of 100 students. However, this apparent difficulty may only reflect the lack of ingenuity of your programmer. In fact, one of the outstanding problems in computer science is determining whether questions exist whose answer can be quickly checked, but which require an impossibly long time to solve by any direct procedure. Problems like the one listed above certainly seem to be of this kind, but so far no one has managed to prove that any of them really are so hard as they appear, i.e., that there really is no feasible way to generate an answer with the help of a computer. Stephen Cook and Leonid Levin formulated the P (i.e., easy to find) versus NP (i.e., easy to check) problem independently in 1971.
P vs NP: A Millennium Problem
8/11/2009 Vira ChankongEECS, CWRU 8
Highlights:
- Modeling
- Solution Strategies
- Enumeration (implicit) for 0-1 IP, combinatorial
problems
- Branch-and Bound
- Heuristics and Meta Heuristics
- Constraint Programming
- Simulated Annealing, Genetic Algorithms
Discrete Optimization
8/11/2009 Vira ChankongEECS, CWRU 9
Modeling Discrete Optimization andModeling Discrete Optimization and IP: ExamplesIP: Examples
8/11/2009 Vira ChankongEECS, CWRU 10
Exam ple 5: Am bulance Location
The time in minutes it takes an ambulance to travel from one district to another is show n below. The population of each district in thousands is also show n. Find districts to locate 2 ambulances so as to maximize the number of people w ho lives w ithin 2 minutes of an ambulance. To District 1 2 3 4 5 6 7 8 1 0 3 4 6 8 9 8 10 From 2 3 0 5 4 8 6 12 9 District (^) 3 4 5 0 2 2 3 5 7 4 6 4 2 0 3 2 5 4 5 8 8 2 3 0 2 2 4 6 9 6 3 2 2 0 3 2 7 8 12 5 5 2 3 0 2 8 10 9 7 4 4 2 2 0 # of population in 1000s 8 3 5 9 4 1 7 2
8/11/2009 Vira ChankongEECS, CWRU 13
Solution (Ambulance): Decision Variables: 1 2 3 4 5 6 7 8 Locate in district i? Xj^^0 0 0 0 1 1 0 District i covered? Yj^0 0 1 1 1 1 1 Objective Function: # of people within 2 minutes of ambulance:^28 Constraints: To District? 1 2 3 4 5 6 7 8 Yi Sum[a(ij)Xj)] 1 1 0 0 0 0 0 0 0 0 0 2 mins 2 0 1 0 0 0 0 0 0 0 0 From^3 0 0 1 1 1 0 0 0 1 District^4 0 0 1 1 0 1 0 0 1 5 0 0 1 0 1 1 1 0 1 2 6 0 0 0 1 1 1 0 1 1 2 7 0 0 0 0 1 0 1 1 1 1 8 0 0 0 0 0 1 1 1 1 1
Number of ambulances^2 must equal 2
District
8/11/2009 Vira ChankongEECS, CWRU 14
A salesman in city 1 wants to visit each and every city once and returns to city 1. Find a “tour” with minimum distance.
1
2
3
4 6
5
7
8
1
1
Traveling Salesman Problem (TSP) Traveling Salesman Problem (TSP)
1
2
2
5
1
4
2
2
1
1
8/11/2009 Vira ChankongEECS, CWRU 15
TSP Model 1TSP Model 1
8 8 1 1
1 if segment from to is taken Variables: 0 otherwise
Objective Functio
n:
Constraints: For e
ij
i j ij^ ij
x i^ j i j j i c x = =
∑∑
8 1 8 1
ach city, exactly one inbound: 1, 1,..,
exactly one outbound: 1, 1,..,
i^ ij
j ij
x j
x i
=
=
∑
∑
20 20 20 1 2 20 2 20 20 20 4 20 20 2 20 1 20
20
2
20
1
1
20
5
2
1
20
1
20
20
20
1
1
20 2 5 20 20 20 2 20 20 4 20
20 1 20 1 20
20 20 2 20 1
20 20 1 20 20
20 20 20 20 20
cij
12 21 13 31 23 32 24 42
1;
No subtour of 2 citie 1; 1; 1
s
;
: x x x x x x x x
25 52 34 43 37 73 46 64 56 65 58 85 67 76 78 87
1; 1; 1; 1; 1; 1; 1; 1;
x x x x x x x x x x x x x x x x
8/11/2009 Vira ChankongEECS, CWRU 16
TSP Model 1TSP Model 1
No subtour of 3 or more citi 2; 2; 2; 2;
3; 3;
e
3;
s
3 3 3
:
; ;
x x x x x x x x x x x x
x x x x x x x x x x x x x x x x x x x x x x x x
56 67 78 85 58 87 76 65
; 3; 3; No need for constraints on more than 4 cities
x + x + x + x ≤ x + x + x + x ≤
8/11/2009 Vira ChankongEECS, CWRU 19
TSP Model 4 TSP Model 4
1
1 9 8 1
1
Variables:
Objective Function:
city visited on the tour = 1,.., 1; 1
Each occurs exactly once and they are alldifferent (
Constraints:
,
i i
i th
i y y
i
y i i y y c
y alldifferent y
=^ +
∑
.., y 8 )
cij
20 20 20 1 2 20 2 20 20 20 4 20 20 2 20 1 20
20
2
20
1
1
20
5
2
1
20
1
20
20
20
1
1
20 2 5 20 20 20 2 20 20 4 20
20 1 20 1 20
20 20 2 20 1
20 20 1 20 20
20 20 20 20 20
- alldifferent ( y 1 ,.., y 8 ) = a permutation of 1,2,..,
- The “logic-based” model is very compact and intuitively meaningful
- The only difficulty is that the variables appear as subscripts, and hence not treatable by an IP-based method
- Can be solved by a logic-based method or a heuristic (e.g genetic algorithm)
8/11/2009 Vira ChankongEECS, CWRU 20
TSP Model 5 TSP Model 5
8 1
city visited after city , = 1,..,
Begin at city 1 End at city 1 a
Variables:
Objective Function:
Constraints
nd other occurs exactly once and they are alldif
: fe
i
i i iy
i
y i i c
y
=
= ∑
1 8
rent circuit y ( ,.., y )
cij
20 20 20 1 2 20 2 20 20 20 4 20 20 2 20 1 20
20
2
20
1
1
20
5
2
1
20
1
20
20
20
1
1
20 2 5 20 20 20 2 20 20 4 20
20 1 20 1 20
20 20 2 20 1
20 20 1 20 20
20 20 20 20 20
- circuit ( y 1 ,.., y 8 ) = a permutation of (1,2,..,8) that begins and ends with 1
- This “logic-based” model is even more compact and better than Model 4, since
- There is only one variable-subscript
- The feasible set is a lot smaller, e.g. ( y 1 , y 2 , y 3 , y 4 ) = (3,4,2,1) is feasible since it gives a complete tour 1-3-2-4-1), but (3,4,1,2) is not since it gives 2 subtours 1-3-1, and 2-4-
- Solved by a logic-based method or a heuristic (e.g genetic algorithm)
8/11/2009 Vira ChankongEECS, CWRU 21
Warehouse location problem
• n warehouses
¾ cost f j of opening warehouse j
• m customers
¾ customer i has a “demand” of di units
¾ unit shipping cost c ij of serving customer i via warehouse j.
• Variables:
¾ let y j = 1 if warehouse j is opened
¾ Let x ij = amount of demand for customer i satisfied via
warehouse j.
8/11/2009 Vira ChankongEECS, CWRU 22
Model for warehouse location
- y (^) j = 1 if warehouse i is opened
y j = 0 otherwise
- x (^) ij = units supplied from i to j
subject to:
- customers get their demand
satisfied
- each warehouse is either opened
or it is not (no partial openings)
- no shipments are made from an
empty warehouse
Σ ixij = d j
yj = 0 or 1 for all j
xij ≤ d j yi for all i,j
and x ij ≥ 0
minimize Σ i,j cij xij +Σ j f i yi
8/11/2009 Vira ChankongEECS, CWRU 25
s.t. 0 ≤ x 1 ≤ 10
0 ≤ x 2 ≤ 15
0 ≤ x 3 ≤ 25
If x 1 < 10, then x 2 = 0.
If x 2 < 15, then x 3 = 0.
Minimize 2,000 x 1 +1,800 x 2 + 1,700 x 3
Let w 1 = 1 if x 1 = 10
w 1 = 0 if x 1 < 10
Let w 2 = 1 if x 2 = 15
w 2 = 0 if x 2 < 15
10 w 1 ≤ x 1 ≤ 10
15 w 2 ≤ x 2 ≤ 15 w 1
0 ≤ x 3 ≤ 25 w 2
If x 1 < 10, then x 2 = 0.
If w 1 = 0, then x 2 = 0.
If x 2 < 15, then x 3 = 0.
If w 2 = 0, then x 3 = 0.
w 1 , w 2 ∈ {0, 1}
Modeling piecewise linear functions
Note: It does not model the w ’s perfectly. ¾Suppose x 1 = 10, x 2 = x 3 = 0; ¾Then w 1 = 0 or 1 ¾Cost = 20,000, regardless
8/11/2009 Vira ChankongEECS, CWRU 26
Modeling Logical Constraints
The cash amount of $14,000 is available for investing on 6 securities, The cash required from each investment as well as the NPV of the investment is below.
What is the optimal strategy to maximize the NPV?
Note: No fractional investment is allowed for each investment.
NPV ($1000s)
Cost ($1000s)
Investment
8/11/2009 Vira ChankongEECS, CWRU 27
IP Model: 1 0
, if we invest in i 1,...,6, i , else x
⎧ = = ⎨ ⎩
Variables:
Objective: Maximize total NPV Max: 16 x 1 + 22 x 2 + 12 x 3 + 8 x 4 + 11 x 5 + 19 x 6 Constraints: Total cash available: 5 x 1 + 7 x 2 + 4 x 3 + 3 x 4 + 4 x 5 + 6 x 6 ≤ 14 Indivisibility of investment: x (^) j ∈ {0,1} for each j = 1 to 6
8/11/2009 Vira ChankongEECS, CWRU 28
Additional Constraints: Logical
- No more than 4 stocks are selected.
- If stock 2 is selected, then so is stock 1.
- If stock 1 is selected, then stock 3 is not selected.
- Either stock 4 is selected or stock 5 is selected, but not both.
8/11/2009 Vira ChankongEECS, CWRU 31
More logical constraint: Stock 1 must be selected unless the NPV of the portfolio exceeds $42,000.
If NPV < 42 then x 1 =1.
⇒ x 1 ≥ (42 – NPV)/
⇒Add constraint:
42 x 1 ≥ 42 - (16 x 1 +22 x 2 +12 x 3 +8 x 4 +11 x 5 +19 x 6 )
Note: NPV = 16 x 1 +22 x 2 +12 x 3 +8 x 4 +11 x 5 +19 x 6
8/11/2009 Vira ChankongEECS, CWRU 32
- May not be easy to model
- They are usually not as easy to solve. In fact they are generally much harder to solve than linear programming problems. Although some IPs programs are easy (some problems with millions of variables can be solved), some are hard (even ones with 100 variables)
- There are often many ways to model the same integer program (see for example various ways to model the TSP), and solvers for integer programs (such EXCEL Solver or LINDO/LINGO) are quite sensitive to how they are formulated
The Cons of Discrete Optimization/IP The Cons of Discrete Optimization/IP
8/11/2009 Vira ChankongEECS, CWRU 33
Example: Different ways of modeling the same things and their effects on solution methods:
- A: 2 x 1 + 2 x 2 + … + 2 x 50 ≤ 51, x (^) i = 0 or 1
- B: x 1 + x 2 + … + x 50 ≤ 25 B is better than A
- C: x 1 ≤ y , x 2 ≤ y , …, x 10 ≤ y, x (^) i = 0 or 1; y = 0 or 1
- D: x 1 + … + x 50 ≤ 10 y C is better than D
Why? Must know how algorithms work!!!
8/11/2009 Vira ChankongEECS, CWRU 34
Solution Methods for Solving Solution Methods for Solving Integer ProgramsInteger Programs
8/11/2009 Vira ChankongEECS, CWRU 37
A rounding technique that
sometimes is useful, and
sometimes not.
5 Solve LP (ignoreintegrality) get x=4.8, y=
and z =14.
Round, get x=5, y=0,
infeasible!
Truncate, get x=4, y=0,
and z =
Same solution value at
x=0, y=3.
Optimal is x=3, y=1, and
z =
8/11/2009 Vira ChankongEECS, CWRU 38
Methods for Solving IPs:
• Enumeration Techniques
- Complete Enumeration
- Consider all possible “solutions” and choose the best
- Implicit Enumeration
- Branch and Bound applied to pure binary IPs
- Branch and Bound/Branch-and-Cut/Branch-and-Price
- Implicitly consider all solutions, but creatively eliminating a large number of them before they are actually searched
• Cutting Plane Techniques
- Use LP to solve integer programs by adding constraints
to eliminate the fractional solutions.
• Heuristics
8/11/2009 Vira ChankongEECS, CWRU 39
Capital Budgeting Example
Investment budget = $14,
maximize 16 x 1 + 22 x 2 + 12 x 3 + 8 x 4 +11 x 5 + 19 x 6 subject to 5 x 1 + 7 x 2 + 4 x 3 + 3 x 4 +4 x 5 + 6 x 6 ≤ 14 x (^) j binary for j = 1 to 6
NPV ($1000s
Cost ($1000s)
Investment
16
8/11/2009 Vira ChankongEECS, CWRU 40
Complete Enumeration for 0-1 IPs
- Systematically considers all possible values of the decision variables. - If there are n binary variables, there are 2 n different ways. (in permutation problems: the number of ways is in the order of n !)
- Usual idea: iteratively break the problem in two. At the first iteration, we consider separately the case that x 1 = 0 and x 1 = 1.
