Download Introduction to Parallel Processing-Parallel Processing-Lecture Slides and more Slides Parallel Computing and Programming in PDF only on Docsity!
Course Introduction
-^
With advances in computer architecture, highperformance multiprocessor computers have becomereadily available & affordable. As a result, highperformance & supercomputing is accessible to alarge segment of industry that was once restricted tomilitary research & large corporations. The course iscomprised of architecture, algorithms & programmingof multicore & Parallel computing systems. It focuseson design concepts, principles, paradigms, models,performance evaluation and real life applications.
docsity.com
Course Contents
•^
Motivation, Scope & Applications
-^
ILP: Pipelining, Superscalar & VLIW.
-^
Classification: SIMD,MIMD,SPMD, multicoreSMP, vector/array processor, MPP & Clusters
-^
Shared/distributed memory, DSM, UMA/NUMA
-^
Multithreading & message passing model
-^
Cache Coherence & Disk arrays
-^
Interconnection networks, static vs dynamic,topologies, routing, Mapping / Embedding &performance evaluation
-^
Communication operations: Broadcast,Reduction, Scatter, Gather
docsity.com
von Neumann Architecture
Comprised of four maincomponents:
MemoryControl UnitArithmetic Logic UnitInput/Output
docsity.com
serial computing
7^ Traditionally, software has been written for
serial
computation:
To be run on a single computer having a single CentralProcessing Unit (CPU);A problem is broken into a discrete series of instructions.Instructions are executed one after another.Only one instruction may execute at any moment in time.
docsity.com
parallel computing
- The compute resources can include:
- A single processor with multiple cores;– A single computer with multiple processors– An arbitrary number of computers connected
by a network;
docsity.com
parallel computing
- The computational problem usually
demonstrates characteristics such asthe ability to be:– Broken apart into discrete pieces of work that
can be solved simultaneously;
- Execute multiple program instructions at any
moment in time;
- Solved in less time with multiple compute
resources than with a single compute resource.
docsity.com
Uses for Parallel Computing:
-^
Historically, parallel computing has been considered to be "the high end ofcomputing", and has been used to model difficult scientific and engineeringproblems found in the real world
.
-^
Some examples:–
Atmosphere, Earth, Environment– Physics - applied, nuclear, particle, condensed matter, high pressure, fusion,photonics– Bioscience, Biotechnology, Genetics– Chemistry, Molecular Sciences– Geology, Seismology– Mechanical Engineering - from prosthetics to spacecraft– Electrical Engineering, Circuit Design, Microelectronics– Computer Science, Mathematics
docsity.com
Uses for Parallel Computing:
-^
Today, commercial applications provide an equal or greater driving force inthe development of faster computers. These applications require theprocessing of large amounts of data in sophisticated ways.
-^
For example:^ –^
Databases, data mining
-^
Oil exploration
-^
Web search engines,web based business services
-^
Medical imaging and diagnosis
-^
Pharmaceutical design
-^
Management of national andmulti-national corporations
-^
Financial & economic modeling
-^
Adv graphics and virtual reality,esp entertainment industry
-^
Networked video & multi-mediatechnologies
-^
Collaborative work environment
-^
Computational Fluid dynamics
-^
DNA research
docsity.com
Why Use Parallel Computing?
-^
Save time and/or money:
-^
In theory, throwing more resources at a task willshorten its time to completion, with potential costsavings.
- Parallel clusters can be built from cheap,
commodity components.
docsity.com
Why Use Parallel Computing?^ •
Solve larger problems:^ – Many problems are so large and/or complex
that it is impractical or impossible to solve themon a single computer, especially given limitedcomputer memory.
(en.wikipedia.org/wiki/Grand_Challenge) problemsrequiring PetaFLOPS and PetaBytes of computingresources.
- Web search engines/databases processing
millions of transactions per second
docsity.com
Why Use Parallel Computing?^ •
Use of non-local resources:^ – Using compute resources on a wide area
network, or even the Internet when localcompute resources are scarce.
- For example:
- SETI@home (setiathome.berkeley.edu) uses over
330,000 computers for a compute power over 528TeraFLOPS (as of August 04, 2008)
- Folding@home (folding.stanford.edu) uses over
340,000 computers for a compute power of 4.2PetaFLOPS (as of November 4, 2008)
docsity.com
Why Use Parallel
Computing?
•^
Limits to serial computing:^ – Both physical and practical reasons pose
significant constraints to simply buildingever faster serial computers:• Transmission speeds• Limits to miniaturization• Power Dissipation• Economic limitations
docsity.com
Who and What
-^
Top500.org provides statistics on parallel computing users in
the charts below:-
-^
Some things to note: Sectors may overlap ––^
for example, research may be classified research. Respondents have to choose betweenthe two.
-^
"Not Specified" is by far the largest application - probably means multiple applications.
docsity.com
The Future:
- During the past 20 years, the trends
indicated by ever faster networks,distributed systems, and multi-processorcomputer architectures (even at thedesktop level) clearly show that parallelism is the future ofcomputing
docsity.com
