Vivek Sarkar
Department of Computer Science
Rice University
COMP 422, Lecture 4:
Decomposition Techniques for
Parallel Algorithms
(Sections 3.1 & 3.2 of textbook)
COMP 422 Lecture 4 17 January 2008
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Decomposition Techniques for Parallel Algorithms, Slides of Parallel Computing and Programming
Data and Computation Decompositions methods and Tasks and Dependency Graphs are part of lecture
Typology: Slides
2021/2022
1 / 41
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Decomposition Techniques for Parallel Algorithms and more Slides Parallel Computing and Programming in PDF only on Docsity!
Vivek Sarkar
Department of Computer Science
Rice University
[email protected]
COMP 422, Lecture 4:
Decomposition Techniques for
Parallel Algorithms
(Sections 3.1 & 3.2 of textbook)
COMP 422 Lecture 4 17 January 2008
Recap of Lecture 3
- Interconnection Networks
—Static (direct) vs. Dynamic (indirect) networks
—Metrics: diameter, bisection width, arc connectivity, cost
—Dynamic networks: Crossbar, Omega, (Fat) Tree
—Static networks: Mesh, Torus, Hypercube
- Cache Coherence
—Invalidate vs. Update protocols
—Bus Snooping vs. Directory-based
—False sharing
- Communication Costs
—Store-and-forward vs. Cut-through
Acknowledgments for today’s lecture
- Cilk lecture by Charles Leiserson and Bradley Kuszmaul (Lecture 1, Scheduling Theory)
— http://supertech.csail.mit.edu/cilk/lecture-1.pdf
- Slides accompanying course textbook
—http://www-users.cs.umn.edu/~karypis/parbook/
Outline of Today’s Lecture
- Tasks, Dependence Graphs, Scheduling Theory
- Data and Computation Decompositions
Example: Database Query Processing Consider the execution of the query:
MODEL = ``CIVIC'' AND YEAR = 2001 AND
(COLOR = GREEN'' OR COLOR =WHITE)
on the following database:
Example: Database Query Processing The execution of the query can be divided into subtasks in various ways. Each task can be thought of as generating an intermediate table of entries that satisfy a particular clause.
Critical Path Length
- A directed path in the task dependency graph represents a sequence of tasks that must be processed one after the other.
- The longest such path determines the shortest time in which the program can be executed in parallel.
- The length of the longest path in a task dependency graph is called the critical path length.
- The ratio of the total amount of work to the critical path length is the average degree of concurrency.
Examples of Critical Path Length Consider the task dependency graphs of the two database query decompositions: Total work = 63 Crit. path length = 27 Avg. concurrency ~ 2. Total work = 64 Crit. path length = 34 Avg. concurrency ~ 1.
Upper Bounds on T P
- Greedy scheduler ==> no unenforced idleness
Performance Bound for Greedy Algorithm NOTE: performance bound approaches 1 (optimal) when one of the max terms dominates the other
Socrates Normalized Speedup
T
/T
P
T
/T
P
T
/T
T
P
= T
1
/ P + T
∞
measured speedup
T
P
= T
∞
TP
T
1
/ P
Developing Socrates
- For the competition, Socrates was to run on a 512 - processor Connection Machine Model CM5 supercomputer at the University of Illinois.
- The developers had easy access to a similar 32 -processor CM5 at MIT.
- One of the developers proposed a change to the program that produced a speedup of over 20% on the MIT machine.
- After a back-of-the-envelope calculation, the proposed “improvement” was rejected!
Outline of Today’s Lecture
- Tasks, Dependence Graphs, Scheduling Theory
- Data and Computation Decompositions
Decomposition Techniques: Patterns for Parallel Algorithms So how does one decompose a task into various subtasks? While there is no single recipe that works for all problems, we present a set of commonly used techniques that apply to broad classes of problems. These include:
- recursive decomposition
- data decomposition
- exploratory decomposition
- speculative decomposition