Lecture 8: Greedy Graph Algorithms - Dijkstra's Algorithm and Minimum Spanning Trees, Study notes of Computer Science

A set of lecture notes from a university course on algorithm design and analysis. The notes cover the topics of dijkstra's algorithm for finding shortest paths and minimum spanning trees (mst) using kruskal's, prim's, and reverse-delete algorithms. The notes include proofs of correctness, implementations, and examples.

Typology: Study notes

Pre 2010

Uploaded on 09/24/2009

koofers-user-nwp
koofers-user-nwp 🇺🇸

10 documents

1 / 5

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
1
9/14/2007
S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayn e
Sofya Raskhodnikova
Algorithm Design and Analysis
LECTURE 8
Greedy Graph Algorithms
Shortest Paths
Dijkstra’s
Minimum Spanning Tree
Kruskal’s, Prim’s
Reverse-Delete
9/14/2007
S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayn e
Review Question
Express vVdegree(v) in terms of m, the number of
edges in the (undirected) graph.
(
Handshaking Lemma:
vVdegree(v) = 2|E|
for undirected graphs
)
When a priority queue is impemented using an unsorted
array, how long do Extract-Min and Decrease-Key take?
(Answer: Extract-Min – O(n);
Decrease-Key – O(1))
9/14/2007
S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayn e
Shortest Path Problem
Input:
Directed graph G = (V, E).
Source node s, destination node t.
for each edge e, length
(e)
= length of e.
Find: shortest directed path from s to t.
 

s
3
t
2
6
7
4
5
23
18
2
9
14
15
5
30
20
44
16
11
6
19
6
9/14/2007
S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayn e
Dijkstra's Algorithm (Greedy)
Maintain a set of explored nodes S whose shortest path
distance d(u) from s to u is known.
Initialize S = { s}, d(s) = 0.
Repeatedly choose unexplored node v which minimizes
add v to S, and set d(v) = π(v).
,)()(min)(
:),(
eudv
Suvue
+
=
=
π
 
! "# $
$

%

9/14/2007
S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayn e
Dijkstra's Algorithm (Greedy)
Maintain a set of explored nodes S whose shortest path
distance d(u) from s to u is known.
Initialize S = { s}, d(s) = 0.
Repeatedly choose unexplored node v which minimizes
add v to S, and set d(v) = π(v).
 
! "# $
$

%

,)()(min)(
:),(
eudv
Suvue
+
=
=
π
9/14/2007
S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayn e
Proof of Correctness
Invariant. For each node u S, d(u) is the length of
the shortest path from s to u.
Proof: (by induction on |S|)
Base case: |S| = 1 is trivial.
Inductive hypothesis: Assume for |S| = k 1.
Let v be next node added to S, and let (u,v) be the chosen edge.
The shortest s-u path plus (u,v) is an s-v path of length π(v).
Consider any s-v path P. We'll see that it's no shorter than π(v).
Let x-y be the first edge in P that leaves S,
and let P' be the subpath to x.
P + (x,y) has length d(x)+ (x,y)π(y)π(v)
%
#
$
&
&'
pf3
pf4
pf5

Partial preview of the text

Download Lecture 8: Greedy Graph Algorithms - Dijkstra's Algorithm and Minimum Spanning Trees and more Study notes Computer Science in PDF only on Docsity!

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne^ Sofya Raskhodnikova

Algorithm Design and Analysis LECTURE 8

Greedy Graph Algorithms • • Shortest PathsMinimum Spanning Tree • Dijkstra’s

  • • Kruskal’s, Prim’sReverse-Delete 9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne - Expressedges in the (undirected) graph.^ Review Question  vV degree ( v ) in terms of m , the number of - When a priority queue is impemented using an unsorted^ ( Handshaking Lemma:^ ^ for undirected graphs vV^ degree ( v ) = 2^ | E) | array, how long do Extract-Min and Decrease-Key take? (Answer:Decrease-Key – O(1)) Extract-Min – O(n);

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

  • Input: – Directed graph G = (V, E).^ Shortest Path Problem
  • Find: – Source node s, destination node t.– for each edge e, length shortest directed path from s to t.^ (e)^ = length of e. s^915 1427562030544182311162466319 t               9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne
    • distance d(u) from s to u is known. Maintain a set of^ Dijkstra's Algorithm (Greedy) explored nodes S whose shortest path
    • • • Initialize S = { s }, d(s) = 0.Repeatedly choose unexplored node v which minimizesadd v to S, and set d(v) = π( v ) = (^) e =min( u , v ): uS π d ((v). u )+ ( e ) ,    (^) !   "#         $ %^ ^ ^ ^ ^ $

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

  • distance d(u) from s to u is known. Maintain a set of^ Dijkstra's Algorithm (Greedy) explored nodes S whose shortest path
  • • • Initialize S = { s }, d(s) = 0.Repeatedly choose unexplored node v which minimizesadd v to S, and set d(v) = π(v).    (^) !   "#         $ π( v ) = (^) e =(min u %, v ): uSd ( u )+ ( e ) ,     $ 9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

the shortest path from s to u.^ Invariant.^ Proof of Correctness^ For each node u^ ∈^ S, d(u) is the length of

  • •^ Proof:Base case:Inductive hypothesis: – Let v be next node added to S, and let (u,v) be the chosen edge.^ (by induction on |S|) |S| = 1 is trivial. Assume for |S| = k ≥ 1.
    • The shortest s-u path plus (u,v) is an s-v path of length– Consider any s-v path P. We'– Let x-y be the first edge in P that leaves S,and let P'be the subpath to x.ll see that it's no shorter than π(v). π(v).
    • P + (x,y) has length  d(x)+ (x,y) π(y) π(v)

%^ &'^ ^ ^ & $^ #

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

  • – Next node to explore = node with minimumFor unexplored nodes, maintain^ Implementation π(v).
    • When exploring v, for each edge e = (v,w), update
  • queue of unexplored nodes, prioritized by Efficient implementation: Maintain a priority π(v).

π( w ) =min{ π( w ), π( v )+^ π(^ v ()^ e =)} e^^ =min(.^ u , v ): u ∈ Sd ( u )+( e^ ).

&# (  9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

d for [ s ] do each← d^ Implementation of Dijkstra 0 [ vv ] ∈← ∞; π V – { s [} v ] ← ∞ (G,^  )

S Q while ← ∅ ← do V Qu (^) ← ≠ ∅ EXTRACT Q keyed on is a priority queue maintaining-MIN π( Q [ v )] VS , S for ← (^) do if each S ∪ (^) π v { then [ uv };] > Adjacency-list d (^) πππππ[[ u [ v ] u ] ← π] + ←  d ([[ uuu ], (^) ] + v [) u ]( u , v ) leaving v exploreedges Implicit DECREASE-KEY

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

Demo of Dijkstra’s Algorithm

Graph withnonnegativeedge lengths: A A (^1031) C B C B (^) 4 822 7 DE DE 9 9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

Demo of Dijkstra’s Algorithm

Initialize: Q: A 0 B ∞ (^) CDEA A 103 1 BC B C (^) 4 822 7 DE DE 9 S: {}

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

Demo of Dijkstra’s Algorithm

Q: A 0 B ∞ C ∞ D ∞ E ∞ A^ A^1031 C^ B C^ B^^4 8227 DE^ DE^9

S: { A }

“A” ←←←← EXTRACT-MIN( Q ): ∞

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

Demo of Dijkstra’s Algorithm

Q: A 0 B ∞ C ∞ D ∞ E ∞ A^ A^1031 BC^^ B C^^4 8227 DE^ DE^9

S: { A }

10 3

Explore edges leaving A : ∞ ∞

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

Demo of Dijkstra’s Algorithm

Q: A 0 B ∞ C ∞ D ∞ E ∞ A^ A^1031 C^ B C^ B^^4 8227 DE^ DE^9

S: { A, C, E, B, D }

1077 3 1111 ∞ 9 ∞ 5

“D” ←←←← EXTRACT-MIN( Q ):

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

Analysis of Dijkstra

times Handshaking Lemma^ n degree times(^ while u )^ do^ Q u^ S^ for ^ ←←^ ≠ ∅^ do if each^ E S m^ XTRACT∪ (^) implicit D^ dv {[ then^ uv }] >^ Adj -M (^) dd [IN[[ vuu ECREASE](]] + Q ←) w d ([ uu ,] +-K v ) (^) EY w ’s.( u , v ) &( 0 . -  ./  , 5    1 #  0 67    4  #   ) *  2  #  $         (^)    8! (^) 9 :  9 : # 9  (^) +; "    3 "      )

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

Input: with weight function^ Minimum spanning tree (MST) A connected undirected graph w : E → . G = ( V , E )

  • For now, assume all edge weights are distinct.

Output: all vertices — of minimum weight: A spanning tree w ( T ) =( u , v ) ∈ T Tw — a tree that connects( u , v ). 9/14/2007 S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

Example of MST 6 12

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

Example of MST 6 12

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

• • Network design.Approximation algorithms for NP-hard problems. – telephone, electrical, hydraulic, TV cable, computer, road^ Applications

  • Indirect applications.^ – traveling salesperson problem, Steiner tree – max bottleneck paths– LDPC codes for error correction– image registration with Renyi entropy
    • learning salient features for real-time face verification– reducing data storage in sequencing amino acids in a protein– model locality of particle interactions in turbulent fluid flows– autoconfig protocol for Ethernet bridging to avoid cycles in anetwork
  • Cluster analysis.

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

  • order of weights. Insert edge e in T unless doing so wouldcreate a cycle. Kruskal's:^ Greedy Algorithms for MST Start with T = φ. Consider edges in ascending
  • descending order of weights. Delete edge e from T unlessdoing so would disconnect T. Reverse-Delete: Start with T = E. Consider edges in
  • outward. At each step, add to T the cheapest edge e withexactly one endpoint in T. Prim's: Start with some root node s. Grow a tree T from s 9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne
    • min weight edge with exactly one endpoint in S. Then theMST contains e. Cut property.^ Cut and Cycle Properties Let S be a subset of nodes. Let e be the
    • weight edge in C. Then the MST does not contain f. Cycle property. % Let C be a cycle, and let f be the max  <   ^   / %5      / %

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

  • Cycle: Set of edges the form (a,b),(b,c),(c,d),…,(y,z),(z,a).^ Cycles and Cuts
  • subset of edges with exactly one endpoint in S. Cut: a subset of nodes S. The corresponding  ^ <#. < ^    cutset = =  D is the  ?   = > < %<  0 ?>  @ =   > A? = 

 ?   = > 9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

  • Claim. number of edges.^ Cycle-Cut Intersection A cycle and a cutset intersect in an even
  • Proof: same number of times. (^) %A cycle has to leave and enter the cut the < B 8 %

9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

Cut property: weight edge with exactly one endpoint in S. Then the MST^ Proof of Cut Property Let S be a subset of nodes. Let e be the min

T* contains e. • Proof: – Suppose e does not belong to T.– Adding e to T creates a cycle C in T*. (exchange argument)

  • Edge e is both in the cycle C and in the cutset D corresponding to– T'– Since cS = T* there exists another edge, say f, that is in both C and D. (^) e∪ < c { e } - { f } is also a spanning tree.f, cost(T') < cost(T*). Contradiction.  % ^  5 C 9/14/2007 (^) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson, A. Smith, K. Wayne

Cycle property: weight edge in C. Then the MST T* does not contain f.^ Proof of Cycle Property Let C be a cycle in G. Let f be the max

  • – Suppose f belongs to T.– Deleting f from T creates a cut S in T*.– Edge f is both in the cycle C and in the cutset D corresponding to Proof: (exchange argument)
    • T'– Since cS^ = T^ there exists another edge, say e, that is in both C and D. (^) e∪ < c { e } - { f } is also a spanning tree.f, cost(T') < cost(T). Contradiction. 

% ^  5 C