Download Greedy Methods: Prim's Algorithm and Dijkstra's Algorithm in Graph Theory - Prof. Thomas H and more Study notes Algorithms and Programming in PDF only on Docsity!
1 10/24/
CS 432: Greedy Algorithms
Topics covered for greedy algorithms
General principles
Making change
Knapsack problems
Minimum spanning trees: Prim’s, Kruskal’s
algorithms
Single-source shortest path: Dijkstra’s algorithm
Approximation algorithms
2 10/24/
Greedy Method: Overview
Optimization problems: terminology
Solutions judged on some criteria:
Objective function
Example: Sum of edge weights in path is smallest
A solution must meet certain constraints
A solution is feasible
Example: All edges in solution are in graph, form a
simple path
One (or more) feasible solutions that scores
highest (by the objective function) is the optimal
solution(s)
3 10/24/
Greedy Method: Overview
Greedy strategy:
Build solution by stages, adding one item to partial
solution found so far
At each stage, make locally optimal choice based
on the greedy rule (sometimes called the selection
function)
Locally optimal, I.e. best given what info we have now
Irrevocable, a choice can’t be un-done
Sequence of locally optimal choices leads to
globally optimal solution (hopefully)
Must prove this for a given problem!
Approximation algorithms, heuristics
4 10/24/
Making Change
Remember? We did this one in class on Day 1
Inputs:
Value N of the change to be returned
An unlimited number of coins of values d1, d2,.., dk
Output: the smallest possible set of coins that sums to
N
Objective function? Smallest set
Constraints on feasible solutions? Must sum to N
Greedy rule: choose coin of largest value that is less
than N - Sum(coins chosen so far)
Always optimal? Depends on set of coin values
5 10/24/
Algorithm 7.1.1 Greedy Coin Changing
6 10/24/
7 10/24/
Knapsack Problems
Section 7.6 in text
Inputs:
n items, each with a weight w_i and a value v_i capacity of the knapsack, C
Output:
Fractions for each of the n items, x_I Chosen to maximize total profit but not to exceed knapsack capacity
8 10/24/
Two Types of Knapsack Problem
0/1 knapsack problem
Each item is discrete. Must choose all of it or none of it.
So each x_i is 0 or 1
Greedy approach does not produce optimal solutions
But another approach, dynamic programming, does
Continuous knapsack problem
Can pick up fractions of each item
The correct selection function yields a greedy algorithm
that produces optimal results
9 10/24/
Greedy Rule for Knapsack?
Build up a partial solution by choosing x_i for one item until knapsack is full (or no more items). Which item to choose?
There are several choices. Pick one and try on this:
n = 3, C = 20
weights = (18, 15, 10)
values = (25, 24, 15)
What answer do you get?
The optimal answer is: (0, 1, 0.5), total=31. Can you verify this?
10 10/24/
Possible Greedy Rules for Knapsack
Build up a partial solution by choosing x_i for one item until knapsack is full (or no more items). Which item to choose? Maybe this: take as much as possible of the remaining item that has largest value, v_i Or maybe this: take as much as possible of the remaining items that has smallest weight, w_i Neither of these produce optimal values! The one that does “combines” these two approaches.
Use ratio of profit-to-weight
11 10/24/
Example Knapsack Problem
For this example:
n = 3, C = 20 weights = (18, 15, 10) values = (25, 24, 15)
Ratios = (25/18, 24/15, 15/10)
= (1.39, 1.6, 1.5)
The optimal answer is: (0, 1, 0.5)
12 10/24/
Minimum Spanning Tree
Problem: given a connected, undirected, weighted graph:
19 10/24/
Prim’s Algorithm
Run on example graph
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; } 20 10/24/
21 10/24/
Prim’s Algorithm
Run on example graph
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; } 22 10/24/
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; }
Prim’s Algorithm
s Pick a start vertex s
23 10/24/
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; }
Prim’s Algorithm
Red vertices have been removed from PQ
24 10/24/
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; }
Prim’s Algorithm
- Red arrows indicate parent pointers.
- Numbers in nodes are fringe weight. -? in node means node is unseen.
25 10/24/
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; }
Prim’s Algorithm
26 10/24/
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; }
Prim’s Algorithm
v
27 10/24/
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; }
Prim’s Algorithm
v
28 10/24/
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; }
Prim’s Algorithm
v
- Note update of fringe node! FringeWt better, new parent.
29 10/24/
Prim’s Algorithm
v
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; } 30 10/24/
Prim’s Algorithm
v
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; }
37 10/24/
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; }
Prim’s Algorithm
v
38 10/24/
Prim’s Algorithm
MST-Prim(G, wt)^ v init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; }
39 10/24/
Prim’s Algorithm
v
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; } 40 10/24/
Cost of Prim’s Algorithm
(Assume connected graph)
Clearly it looks at every edge, so Ω(n+m)
Is there more? Yes, priority queue operations
ExtractMin called n times
How expensive? Depends on the size of the PQ
descreaseKey could be called for each edge
How expensive is each call?
41 10/24/
Worst Case
If all nodes connected to start, then size of PQ is n-
right away.
Decreases by 1 for each node selected
Total cost is O(cost of extractMin for size n-1)
Note use of Big-Oh (not Big-Theta)
Could descreaseKey be called a lot?
Yes! Imagine an input that adds all nodes to the PQ at the
first step, and then after that calls descreaseKey every
possible time. (For you to do.)
42 10/24/
Priority Queue Costs and Prim’s
Simplest choice: unordered list PQ.ExtractMin() is just a “findMin”
Cost for one call is Θ(n)
Total cost for all n calls is Θ(n^2 )
PQ.decreaseKey() on a node finds it, changes it.
Cost for one call is Θ(n)
But, if we can index an array by vertex number, the cost
would be Θ(1).
If so, worst-case total cost is Θ(m)
Conclusion: Easy to get Θ(n^2 )
43 10/24/
Better PQ Implementations
Consider using a min-heap for the Priority Queue
PQ.ExtractMin() is O(lg n) each time
Called n times, so like Heap’s Construct: efficient!
What about PQ.decreaseKey()?
Our need: given a vertex-ID, change the value stored
But our basic heap implementation does not allow look-ups
based on vertex-ID!
Solution: Indirect heaps (see pages 142-145)
Heap structure stores indices to data in an array that doesn’t
change
Can increase or decrease key in O(lg n) after O(1) lookup
44 10/24/
Better PQ Implementations (2)
Use Indirect Heaps for the PQ
PQ.decreaseKey() is O(lg n) also
Called for each edge encountered in MST algorithm
So O(m x lg n)
Overall: Might be better Θ(n^2 ) than if m << n^2
Fibonacci heaps: an even more efficient PQ
implementation. We won’t cover these.
Θ(m + n lg n)
45 10/24/
Kruskal’s MST Algorithm
Prim’s approach:
Build one tree. Make the one tree bigger and as
good as it can be.
Kruskal’s approach
Choose the best edge possible: smallest weight
Not one tree – maintain a forest!
Each edge added will connect two trees.
Can’t form a cycle in a tree!
After adding n-1 edges, you have one tree, the
MST
46 10/24/
47 10/24/
Prim’s Algorithm
v
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; } 48 10/24/
Kruskal’s Algorithm
v
55 10/24/
MST-Prim(G, wt) init PQ to be empty; PQ.Insert(s, wt=0); parent[s] = NULL; while (PQ not empty){ v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { PQ.Insert(w, wt(v,w)); parent[w] = v; } else if (w is fringe && wt[v,w] < fringeWt(w)){ PQ.decreaseKey(w, wt[v,w]); parent[w] = v; }
Reminder: Prim’s Algorithm
56 10/24/
dijkstra(G, wt, s) init PQ to be empty; PQ.Insert(s, dist=0); parent[s] = NULL; dist[s] = 0 ; while (PQ not empty) v = PQ.ExtractMin(); for each w adj to v if (w is unseen) { dist[w] = dist[v] + wt(v,w) PQ.Insert(w, dist[w] ); parent[w] = v; } else if (w is fringe && dist[v] + wt(v,w) < dist[w] ) { dist[w] = dist[v] + wt(v,w) PQ.decreaseKey(w, dist[w]); parent[w] = v; }
Dijkstra'Algorithm
57 10/24/
Notes on Dijkstra’s Algorithm
Use dist[] to store distances from start to any
fringe or tree node
Store and calculate using distances instead of
edge-weights (like in Kruskal’s MST)
What’s the output?
Tree captured in the parent[] array Shortest distance to each node in dist[] array Trace shortest path in reverse by using parent[] to move from target back to start node, s
58 10/24/
dijkstra(adj, start, parent) { n = adj.last for i = 1 to n { key[i] = } // key is a local array key[start] = 0; predecessor[start] = 0 // the following statement initializes the // container h to the values in the array key h.init(key,n) for i = 1 to n { v = h.min_weight_index() min_cost = h.keyval(v) v = h.del() ref = adj[v] while (ref != null) { w = ref.ver if (h.isin(w) && min_cost + ref.weight < h.keyval(w)) { predecessor[w] = v h.decrease(w, min_cost+ref.weight) } // end if ref = ref.next } // end while } // end for }
59 10/24/
Correctness of These Greedy
Algorithms
Recall that the greedy approach may or may
not guarantee an optimal result
Do these produce optimal solutions?
The min weight spanning tree? Kruskal’s, Prim’s The shortest path from s? Dijkstra’s
Answer: Yes, they do.
Proofs in the text Proofs by induction, also using proof by contradiction
