Scarica Ottimizzazione Combinatoria: Problemi e Algoritmi - Prof. Mansini e più Appunti in PDF di Ricerca Operativa solo su Docsity!
SUMMARY
𝑈 and 𝑏(𝐹
- Combinatorial Optimization (COP)
- 0 - 1 Knapsack Problem (KP)
- Set Covering Problem (SCP)
- Set covering
- Set partitioning
- Set packing
- Traveling Salesman Problem (TSP)
- Plant Location Problem (PLP)
- Minimum Spanning Tree (MST) Problem
- Perfect Matching Problem (PMP)
- Local Search Methods (LS)..............................................................................................................................................
- Constructive Heuristic Algorithms............................................................................................................................
- Constructive algorithm for TSP..............................................................................................................................
- Constructive (and greedy) algorithm of 0-1 KP
- Local Search Algorithms
- Iterative improvement algorithm...........................................................................................................................
- 𝑘-opt heuristic with TSP
- Kernel Search (KS)
- Iterative Kernel Search (IKS)
- Multidimensional Knapsack Problem (MKP)
- Portfolio Selection Problem with Conditional Value at Risk (CVaR)
- Tabu Search (TS).............................................................................................................................................................
- Basic algorithm of Short Term (STM)
- Tabu search with MST – Example............................................................................................................................
- Variable Neighborhood Search (VNS)
- Algorithm of Variable Neighborhood Search
- Algorithm of Variable Neighborhood Descent (VND)
- Adaptive Large Neighborhood Search (ALNS)
- Large Neighbourhood Search (LNS)
- Adaptive Large Neighborhood Search (ALNS)
- Greedy Randomized Adaptive Search Procedure (GRASP)
- GRASP with SCP problem – Example.....................................................................................................................
- Set Covering Problem with Conflicts on Sets (SCP-CS)........................................................................................
- Exact Algorithms
- Specific cuts: Weighted Independent Set Problem (WISP)...................................................................................
- Branch-and-Bound - ) TSP – Example 𝑖
- Branch-and-Cut
- Algorithm of Branch-and-Cut
- Index Selection Problem (ISP)
- ISP - Example
- TSP
- Important definition for MFP
- Max Flow Problem (MFP)
- Ford-Fulkerson Algorithm
- Ford-Fulkerson – Example
- MFP on TSP
- MFP on TSP – Example.........................................................................................................................................
- Approximation Algorithms
- TSP – Tree Algorithm................................................................................................................................................
- Maximum Independent Set Problem (MISP)
- MISP - Greedy Algorithm
- MISP – Worst Case - Example
- Minimum Vertex Cover Problem (MVCP)
- MVCP - Rounding Algorithm..............................................................................................................................
- MVCP – Worst Case - Example
- Bin Packing Problem (BPP)
- BPP - Next fit algorithm........................................................................................................................................
- BPP – Worst case - Example
- Knapsack Problem (KP)
- KP – Dantzig’s formula.........................................................................................................................................
- KP - Heuristic 1 Worst case
- Arc Routing Problems
- Basic Definitions: Undirected Graphs
- Basic Definitions: Directed Graphs
- Basic Definitions: Hamiltonian Graphs and Cycles
- Basic Definitions: Eulerian Graphs and Cycles
- Characterization of Eulerian Graphs
- Undirected Graph..................................................................................................................................................
- Directed Graph
- Chinese Postman Problem (CPP)
- PMP for CPP...........................................................................................................................................................
- PMP for CPP – Example
- Rural Postman Problem (RPP)
- URPP – Example
- URPP – Frederickson Heuristic algorithm
- Frederickson – Example
- Models used
Set packing
max ∑ 𝑐
𝑗
𝑗
𝑛
𝑗= 1
𝑗
It’s a NP-hard problem: it’s easy to find a feasible solution, but the optimal one no.
Traveling Salesman Problem (TSP)
Given:
- A set of costumers 𝑁 = { 1 , … , 𝑛} to visit;
- A setup cost, or traveling cost, 𝑐 𝑖𝑗
for going from 𝑖 to 𝑗.
We want to find the Hamiltonian cycle. So, we want to find the cyclic permutation 𝜋 =
1
𝑛
such that:
min
𝜋
𝑖𝜋
( 𝑗
)
𝑛
𝑗= 1
Where 𝑇 is the set of all cyclic permutations 𝜋 of 𝑛 items.
It’s a NP-hard problem, cause all the possible ways which it’s possible to combine costumers (cyclic
permutation) is an exponential number.
Mathematical formulation:
min ∑ 𝑐
𝑖𝑗
𝑖𝑗
(𝑖,𝑗)∈𝐴
𝑖𝑗
𝑗∈𝑁
𝑖𝑗
𝑖∈𝑁
𝑖𝑗
(𝑖,𝑗)∈𝐴(𝑆)
and
are called assignment problem : they are linear in number, but we obtain a subtour (a smaller
cycle). So, we introduce ( 3 ), that is called subtour elimination. They are in exponential number.
Plant Location Problem (PLP)
It’s NP-hard. There, the goal is to determine the optimal locations for opening facilities to efficiently serve
customers while minimizing costs.
Given:
- A set of facilities to open: 𝑁 = { 1 , … , 𝑛};
- A set of customers to serve: 𝐶 = { 1 , … , 𝑚};
Every facility has a setup cost represented by 𝑘 𝑗
and cost of connection between a customer 𝑖 and a facility 𝑗 is
represented by 𝑐 𝑖𝑗
We introduce two binary variables 𝑥 𝑖𝑗
and 𝑦
𝑖
𝑖𝑗
= 1 if plant 𝑗 serves customers 𝑖, 0 otherwise;
𝑗
= 1 if plant 𝑗 is open, 0 otherwise.
We have two formulations:
min ∑ 𝑘
𝑗
𝑗
𝑛
𝑗= 1
𝑖𝑗
𝑖𝑗
𝑛
𝑗= 1
𝑚
𝑖= 1
𝑖𝑗
𝑛
𝑗= 1
𝑖𝑗
𝑚
𝑖= 1
𝑗
𝑖𝑗
𝑗
( 1 ) states that for every costumer only 1 facility can be selected.
states that the total number of customers assigned to facility 𝑗 should be less or equal than the number
of facilities open (𝑚 ⋅ 𝑦
𝑗
). This constraint represents 𝑛 constraints.
min ∑ 𝑘
𝑗
𝑗
𝑛
𝑗= 1
𝑖𝑗
𝑖𝑗
𝑛
𝑗= 1
𝑚
𝑖= 1
𝑖𝑗
𝑛
𝑗= 1
𝑖𝑗
𝑗
𝑖𝑗
𝑗
In this way, ( 2 ) are 𝑛 ⋅ 𝑚 constraints.
The two formulations are equivalent: they provide the same integer optimal solution. The second one gives
the convex polyhedrons, so is the stronger one.
Minimum Spanning Tree (MST) Problem
Given:
- An undirected graph 𝐺 = (𝑁, 𝐸);
- A node set 𝑁, |𝑁| = 𝑛;
- An edge set 𝐸, |𝐸| = 𝑚;
- A cost 𝑐 𝑒
associated with each edge 𝑒 ∈ 𝐸
The MST looks for a spanning tree of minimum cost.
We have two different formulations:
min ∑ 𝑐
𝑒
𝑒
𝑒∈𝐸
𝑒
𝑒∈𝛿
({ 𝑖
})
𝑒
min ∑ 𝑐
𝑒
𝑒
𝑒∈𝐸
𝑒
𝑒∈𝛿({𝑖})
𝑒
𝑒
𝑒∈𝛿(𝑆)
≥ 1 𝑆 ⊂ 𝑁, |𝑆| odd ( 3 )
Both formulations share the first two constraints.
ensures that only one edge connecting person 𝑖 to other should be selected to create a pair.
( 3 ) ensures an integer solution. This is useful to prevent cycles, because in this way we avoid groups of an
odd number of persons.
The second formulation is the strongest one. Moreover, 𝑃 𝐵
= 𝐶𝐻(𝑀), where 𝑀 is the set of all vectors 𝑥
corresponding to matchings.
LOCAL SEARCH METHODS (LS)
Constructive Heuristic Algorithms
- Empty solution;
- Iterative process that progressively determine and incorporate new elements into the solution;
- At the end we have a complete and feasible solution.
Constructive algorithm for TSP
- Arbitrary vertex and its self-loop as the initial partial solution 𝑆;
- Expand 𝑆 by inserting one by one the remaining vertices, obtaining a Hamiltonian cycle. Two separate
phases:
a. Select a vertex 𝑘 not included in 𝑆, according to some (greedy) criterion:
- Nearest Neighbor : select the vertex with the minimum distance from 𝑆;
- Farthest Insertion : select the vertex with the maximum distance from 𝑆;
- Arbitrary Insertion : randomly select the vertex to insert in 𝑆;
- Cheapest Insertion : select the vertex with the minimum insertion cost.
b. Insert vertex 𝑘 in 𝑆, between two specific consecutive vertices 𝑢, 𝑣.
Constructive (and greedy) algorithm of 0-1 KP
- Sort the items in (or):
a. Non-increasing order of profit/weight ratio;
b. Non-increasing order of profit;
c. Non-decreasing order of weight.
- Add items in the knapsack until you reach the first one that exceeds the capacity.
Local Search Algorithms
- Existing feasible solution 𝑥;
- Iterative process that progressively enhance the current solution by making strategic changes ( moves ) in
, where 𝑥 ∉ 𝑁
- At the end we have a feasible solution, that is the local optimum one.
Iterative improvement algorithm
- First Improvement (FI) : the neighbors of the current solution are analysed in random order. A move is
made towards the first one that improves the current solution.
It’s good for efficiency, but not for efficacy because I could leave neighborhood early. It’s faster than BI.
Procedure FI_Simple_Descent(s) /𝑠 ∈ 𝑆 initial sol./
Found:=TRUE;
while found=TRUE do
Found:=FALSE;
for each 𝑠
′
∈ 𝑁(𝑠) do
if 𝑓
′
then
′
Found:=TRUE;
break ;
end while
return
KERNEL SEARCH (KS)
- Initialization:
- Initial kernel set: select promising variables in the kernel set:
a. Solve LP relaxation;
b. Sort variables by sorting criterion according to:
o Non increasing value of variables in LP relaxation solution;
o Non decreasing value of absolute reduced costs.
c. While sorting, we also fixing variables of the problem;
d. Taking the first variables, construct initial kernel set Λ;
e. Partion remaining variables into a sequence of buckets
- Initial solution: solve a restricted MIP on the kernel set.
- Improvement:
a. Set 𝑖 ≔ 1 ;
b. Construct set Λ ∪ 𝐵
𝑖
, with 𝐵
𝑖
current bucket;
c. Solve restricted problem 𝑀𝐼𝑃(Λ ∪ 𝐵
𝑖
) plus some constraints to find new incumbent solution and
improve kernel set. Constraints that we add are:
o 𝑓 ≥ 𝑤
∗
: the value of the objective function must be at least better than the current one;
o ∑ 𝑥
𝑗∈𝐵 𝑗
𝑗
≥ 1 : must take at least a newelement from 𝐵
𝑗
d. If 𝑀𝐼𝑃(Λ ∪ 𝐵
𝑗
) is feasible, it provides a better solution:
o Update incumbent integer solution
∗
∗
o Construct Λ
𝑖
≔ {selected variables in 𝐵
𝑗
o Update kernel set Λ ≔ Λ ∪ Λ
𝑖
. Λ is always increasing.
e. If 𝑖 < 𝑛𝑏 set 𝑖 ≔ 𝑖 + 1 and go to step (b), otherwise stop.
Iterative Kernel Search (IKS)
In Iterative Kernel Search, buckets are scrolled more than once.
Repeat until no more variables are added to Kernel Set:
- Call phase 1 + phase 2;
- Set 𝑛𝑏: = 𝑞 − 1 where 𝑞: = 𝑚𝑎𝑥{𝑖|Λ 𝑖
- Call phase 2;
- If a new variable entered the Kernel Set:
- Set 𝑛𝑏 to its initial value;
- Call phase 2 and goto Step 2.
Multidimensional Knapsack Problem (MKP)
Given:
- 𝑛 available items;
- 𝑚 dimensions of the items;
of the 𝑗 item;
𝑘
of 𝑗 item in dimension 𝑘;
in dimension 𝑖;
that is 1 if item 𝑗 is selected, 0 otherwise.
The objective function is to maximize the total profit, subject to capacity constraints for each dimension.
The mathematical formulation is:
max ∑ 𝑝
𝑗
𝑗
𝑛
𝑗= 1
𝑖𝑗
𝑘
𝑗
𝑖
𝑛
𝑗= 1
𝑗
It’s a NP-hard problem. There, reduced costs provide good signal to identify promising variables.
Portfolio Selection Problem with Conditional Value at Risk (CVaR)
The objective is to optimize the portfolio’s performance while considering risk, transaction costs and capital
constraints.
This is a NP-hard problem.
In the objective function, given:
- 𝑦 that is the Value at Risk (VaR);
𝑘
𝑇
- The probability of each scenario 𝑝 𝑡
We want to maximize the expected value of the 𝑘 worst scenarios.
Variables are:
𝑗
: represents the number of securities to buy for security 𝑗. It’s a continuous variable;
𝑗
: binary variable indicating the selection
or non-selection
of security 𝑗;
𝑡
: represents the deviation from the Value at Risk (VaR) for each period 𝑡. It’s a continuous variable;
- 𝑦: the Value at Risk (VaR), a free variable in the objective function.
Parameters are:
𝑗𝑡
: return of security 𝑗 at time 𝑡;
𝑗
: transaction cost associated with buying security 𝑗;
𝑗
: fixed cost associated with selecting security 𝑗;
𝑡
: probability of scenario 𝑡;
𝑗
: quotation for security 𝑗 at time 𝑡;
- 𝑇: total number of periods;
0
: desired minimum return;
0
and 𝐶
1
: lower and upper bounds on the total amount of capital invested in the portfolio;
- 𝑚: maximum number of selected securities;
𝑗
and 𝑢
𝑗
: parameters defining lower and upper bounds on the number of securities bought for security 𝑗.
max 𝑦 −
𝑡
𝑡
𝑇
𝑡= 1
𝑡
𝑖𝑗
𝑗
𝑗
𝑗
𝑗
𝑗
𝑛
𝑗= 1
𝑛
𝑗= 1
𝑗
𝑗
𝑗
𝑗
𝑛
𝑗= 1
𝑗
𝑗
𝑛
𝑗= 1
0
𝑗
𝑗
𝑛
𝑗= 1
0
𝑗
𝑗
1
𝑛
𝑗= 1
𝑗
𝑛
𝑗= 1
𝑗
𝑗
𝑗
𝑗
𝑗
TABU SEARCH (TS)
Basic algorithm of Short Term (STM)
- Initialize a feasible solution 𝑥̃ using a constructive method.
Set 𝑥 ≔ 𝑥̃ , where 𝑥 is the current solution, and initialize the tabu list 𝑇𝐿 ≔ 0 ;
- Generate the neighborhood 𝑁
by deciding the move;
- Find a solution, the best in the neighborhood, 𝑦 ∈ 𝑁(𝑥) such that:
- 𝑓(𝑦) is minimal, as we are dealing with a minimization problem;
- 𝑦 ≠ 𝑥, ensuring a different solution. Remind that 𝑥 ∉ 𝑁
- The solution doesn’t violate any tabu move;
- Set 𝑥 ≔ 𝑦;
- Update 𝑇𝐿 by adding the move that changes 𝑥 into 𝑦;
- If the stopping criterion isn’t satisfied, go to ( 2 ), stop otherwise.
Tabu search with MST – Example
- Initial feasible solution:
Greedy construction
Step Candidates Selection (min cost) Total weight
- Move : removing an edge and adding a different one (swap). By separating moves we obtain 2 tabu tenures.
Tabu activation rules :
- Removed edges are tabu-active for 2 iterations;
- Added edges are tabu-active for 1 iteration.
For the total of 𝑘 − 1 = 3 iterations.
- Find new solutions and update Tabu List:
TS iterations
Iteration
Tabu-active net tenure
Add Drop Weight
Iterations of a first level TS procedure
Iteration
Tabu-active net tenure
Add Drop
Move
value
Weight
∗
∗
VARIABLE NEIGHBORHOOD SEARCH (VNS)
Algorithm of Variable Neighborhood Search
- Initialization:
a. Define a finite set of neighborhood structures 𝒩
𝑘
𝑚𝑎𝑥
b. Find an initial feasible solution 𝑥;
c. Decide a stopping rule.
- Repeat until stopping rule is satisfied:
a. 𝑘 ← 1 ;
b. Repeat until 𝑘 = 𝑘
𝑚𝑎𝑥
- Shaking: randomly choose 𝑥
′
in the 𝑘-th neighborhood of 𝑥 => 𝑥
′
𝑘
- Local search: find a local optimum 𝑥
′′
starting from 𝑥
′
by using a local search method;
- Move or not:
- If the local optimum is better than the incumbent solution:
o Move from 𝑥 to 𝑥
′′
′′
o Restart the search with 𝒩
1
Note that:
- VNS is a randomly descent method using FI;
- In b.1, solution 𝑥
′
is randomly generated to avoid cycling.
Algorithm of Variable Neighborhood Descent (VND)
- Initialization:
a. Define a finite set of neighborhood structures 𝒩
𝑘
𝑚𝑎𝑥
b. Find an initial feasible solution 𝑥;
- Repeat until no improvement is obtained:
a. 𝑘 ← 1 ;
b. Repeat until 𝑘 = 𝑘
𝑚𝑎𝑥
- Neighborhoods exploration: find the best solution 𝑥
′
in the 𝑘-th neighborhood of 𝑥 => 𝑥
′
𝑘
- Move or not:
′
is better than 𝑥:
o Set 𝑥 => 𝑥 ← 𝑥
′
o Restart the search with 𝒩
1
Note that VND is a randomly descent method using BI.
GREEDY RANDOMIZED ADAPTIVE SEARCH PROCEDURE (GRASP)
There are 2 phases:
- Build a feasible solution using a greedy random adaptive function;
- Improve the current solution using a local search procedure looking for a local minimum or maximum.
Algorithm: GRASP outline
1 function Grasp (instance)
2 bestSolution ← null
3 bestObjValue ← null
4 while stopping rule is not satisfied do
5 solution ← BuildSolution (instance)
6 newSolution, newObjValue ← LocalSearch (instance, solution)
7 if newObjValue is better than bestObjValue then
8 bestSolution ← newSolution
9 bestObjValue ← newObjValue
10 end if
11 end while
12 return bestSolution, bestObjValue
13 end function
Algorithm: Greedy BuildSolution
1 function BuildSolution (instance)
2 solution ← null
3 sortedItems ← Sort(instance.items, 𝜃)
4 while solution is not feasible do
5 nextItem ← RemoveFirst (sortedItems)
6 solution ← Add (solution, nextItem)
end while
8 return solution
9 end function
To produce better solutions, we can make the building solution also adaptive:
Algorithm: Greedy and adaptive BuildSolution
1 function BuildSolution (instance)
2 solution ← null
3 while solution is not feasible do
4 sortedItems ← Sort (instance.items, 𝜃
𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛
5 nextItem ← RemoveFirst (sortedItems)
6 solution ← Add (solution, nextItem)
7 end while
8 return solution
9 end function
Since in a greedy algorithm the selection of the next element to add to the solution is deterministic, we need
to add some randomicity. To do that, we need to add Restricted Candidate List (RLC), a subset of good
candidates.
Common strategies to build the RCL are the following:
- Consider only the elements that increase the objective function value no more than 𝛼 times with respect to
the best choice (𝛼 ∈ [ 0 , ∞]). (𝛼 = 0 : purely greedy, 𝛼 = +∞: purely random);
- Consider only the 𝛽 best elements (𝛽 ∈ [ 0 , 1 ]). (𝛽 = 0 : purely greedy, 𝛽 = 1 : purely random);
- Consider at most the top 𝛾 elements (𝛾 ∈ [ 1 , 𝑛]). (𝛾 = 1 : purely greedy, 𝛾 = 𝑛: purely random).
Algorithm: RCL construction
1 function BuildRCL (items, solution, 𝜃
𝑆
2 𝐴 ← items\solution
3 𝑎 ← min
𝑆
4 𝑏 ← max{𝜃
𝑆
𝑆
6 𝑘 ← min
7 𝑅𝐶𝐿 ← Topk (𝑅𝐶𝐿, 𝑘)
return 𝑅𝐶𝐿
9 end function
At the end, the pseudocode of phase I is:
Algorithm: GRASP phase I
1 function BuildSolution (instance)
2 solution ← null
3 while solution is not feasible do
4 RCL ← BuildRCL (instance, solution, 𝜃
𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛
5 chosenItem ← ChooseRandomItem (RCL)
6 solution ← Add (solution, chosenItem)
7 end while
8 return solution
9 end function
To introduce phase II, we need to introduce a neighborhood 𝒩(𝑆) where to make a local search.
Algorithm: GRASP phase II
1 function LocalSearch (instance, solution)
2 while solution is not locally optimal in 𝒩
solution
do
newSolution ← FindBetterSolution (𝒩
solution
4 solution ← newSolution
5 end while
6 return solution
7 end function
GRASP with SCP problem – Example
To rank each subset, we choose the rank function 𝜃: 𝑁 → ℝ, 𝑗 ↦
𝑐 𝑗
|𝑈
𝑗
|
A greedy procedure to build a solution for the SCP is then:
Algorithm: SCP example: greedy BuildSolution
1 function BuildSolution (𝑈, 𝒰, {𝑐
𝑗
𝑗∈𝑁
3 while ∪
𝑗∈𝑆
𝑗
≠ 𝑈 do
𝑗 ∈ arg min
j∈N\S
𝑗
end while
7 return 𝑆
8 end function
Consider 𝑈 = { 1 , … , 16 } and 𝒰 = {𝑈 1
2
3
- Set 𝑆 = 0
