U
UNIVERSITY OF
NIVERSITY OF M
MASSACHUSETTS
ASSACHUSETTS A
AMHERST
MHERST •
•Department of Computer Science
Department of Computer Science
Parallel & Concurrent
Programming:
Collective
Communication
Emery Berger
CMPSCI 691W
Spring 2006
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Parallel and Concurrent Programming : Collective Communication | CMPSCI 691, Study notes of Computer Science
Material Type: Notes; Class: S-Gen Purpose Cmptn-GPU; Subject: Computer Science; University: University of Massachusetts - Amherst; Term: Spring 2006;
Typology: Study notes
Pre 2010
1 / 113
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Parallel and Concurrent Programming : Collective Communication | CMPSCI 691 and more Study notes Computer Science in PDF only on Docsity!
U
NIVERSITY OFNIVERSITY OF
MM
ASSACHUSETTSASSACHUSETTS
A
A
MHERST •MHERST
Department of Computer ScienceDepartment of Computer Science
Parallel & ConcurrentProgramming:CollectiveCommunication
Emery Berger
CMPSCI 691W
Spring 2006
U
U
NIVERSITY OFNIVERSITY OF
MM
ASSACHUSETTSASSACHUSETTS
A
A
MHERST •MHERST
Department of Computer ScienceDepartment of Computer Science
Outline
Last time:
Distributed parallel programming via message-passing
MPI
library
approach
Today:
Moving beyond point-to-point: collective communication
some slides from van de Geijn et al.
U
U
NIVERSITY OFNIVERSITY OF
MM
ASSACHUSETTSASSACHUSETTS
A
A
MHERST •MHERST
Department of Computer ScienceDepartment of Computer Science
Simple Analytical Model
To send
n
bytes:
α
= startup latency
β
= per-byte cost
α
- n
β
Naive broadcast (
p
processors):
α
(p-1) + (p-1)n
β
Minimum spanning tree
:
α
(log p + p - 1) + 2(p-1)/p * n
β
Note: does not model
contention
In practice, can avoid it
U
U
NIVERSITY OFNIVERSITY OF
MM
ASSACHUSETTSASSACHUSETTS
A
A
MHERST •MHERST
Department of Computer ScienceDepartment of Computer Science
MPI Collective Ops
MPI has rich set of collectivecommunication operations (& duals):
Synchronization
Barrier
Communication
Broadcast
Scatter, Gather
Reduce, Scan
Communicator management
U
U
NIVERSITY OFNIVERSITY OF
MM
ASSACHUSETTSASSACHUSETTS
A
A
MHERST •MHERST
Department of Computer ScienceDepartment of Computer Science
From root: Broadcast
MPI_Bcast
broadcastsingle datumacross allprocessors
A
processors
A0 A0 A0 A
processors
broadcast
U
NIVERSITY OFNIVERSITY OF
MM
ASSACHUSETTSASSACHUSETTS
A
A
MHERST •MHERST
Department of Computer ScienceDepartment of Computer Science
U
NIVERSITY OFNIVERSITY OF
MM
ASSACHUSETTSASSACHUSETTS
A
A
MHERST •MHERST
Department of Computer ScienceDepartment of Computer Science
U
NIVERSITY OFNIVERSITY OF
MM
ASSACHUSETTSASSACHUSETTS
A
A
MHERST •MHERST
Department of Computer ScienceDepartment of Computer Science
U
NIVERSITY OFNIVERSITY OF
MM
ASSACHUSETTSASSACHUSETTS
A
A
MHERST •MHERST
Department of Computer ScienceDepartment of Computer Science
U
NIVERSITY OFNIVERSITY OF
MM
ASSACHUSETTSASSACHUSETTS
A
A
MHERST •MHERST
Department of Computer ScienceDepartment of Computer Science
U
NIVERSITY OFNIVERSITY OF
MM
ASSACHUSETTSASSACHUSETTS
A
A
MHERST •MHERST
Department of Computer ScienceDepartment of Computer Science
U
NIVERSITY OFNIVERSITY OF
MM
ASSACHUSETTSASSACHUSETTS
A
A
MHERST •MHERST
Department of Computer ScienceDepartment of Computer Science
U
NIVERSITY OFNIVERSITY OF
MM
ASSACHUSETTSASSACHUSETTS
A
A
MHERST •MHERST
Department of Computer ScienceDepartment of Computer Science
U
NIVERSITY OFNIVERSITY OF
MM
ASSACHUSETTSASSACHUSETTS
A
A
MHERST •MHERST
Department of Computer ScienceDepartment of Computer Science