Introduction to
Parallel Computing
George Karypis
Parallel Programming Platforms
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Parallel Programming Platforms - Parallel Computing - Lecture Slides, Slides of Parallel Computing and Programming
This lecture is from Parallel Computing course. Its first and introductory lecture to the parallel computing world. Literally, this lecture attracted me to take parallel computing as my final year project. You will definitely love it.
Typology: Slides
2012/2013
1 / 33
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Parallel Programming Platforms - Parallel Computing - Lecture Slides and more Slides Parallel Computing and Programming in PDF only on Docsity!
Introduction toParallel Computing
George KarypisParallel Programming Platforms
Elements of a Parallel Computer
Hardware
Multiple Processors
Multiple Memories
Interconnection Network
System Software
Parallel Operating System
Programming Constructs to Express/Orchestrate Concurrency
Application Software
Parallel Algorithms
Goal:
Utilize the Hardware, System, & Application Software to either
Achieve Speedup: T p
= T
s /p
Solve problems requiring a large amount of memory.
Logical Organization Elements
Control Mechanism
SISD/SIMD/MIMD/MISD
Single/Multiple Instruction Stream& Single/Multiple Data Stream
SPMD:Single Program Multiple Data
Logical Organization Elements
Communication Model
Shared-Address Space
UMA/NUMA/ccNUMA
Message-Passing
Physical Organization Interconnection Networks (ICNs)
Provide processor-to-processor and processor-to-memoryconnections
Networks are classified as:
Dynamic
The network consists ofswitching elements that thevarious processors attach to
indirect network
Historically used to linkprocessors-to-memory
shared-memory systems Static
Consist of a number ofpoint-to-point links
direct network
Historically used to linkprocessors-to-processors
distributed-memorysystem
Static & Dynamic ICNs
Metrics and Dynamic Networks
Network Topologies Bus-BasedNetworks
Shared medium
Information is beingbroadcasted
Evaluation: Diameter: O(1) Connectivity: O(1) Bisection width: O(1) Cost: O(p)
Network Topologies Multistage Interconnection Networks
Multistage Switch Architecture Pass-through Cross-over
Blocking in a Multistage Switch Routing is done by comparing the bit-levelrepresentation of source and destination addresses.-match goes via pass-through-mismatch goes via cross-over
Network Topologies Complete and star-connected networks.
Network Topologies Hypercubes
Network Topologies Trees