1
CMSC 714
Lecture 5
MPI vs. OpenMP
and
Titanium
Alan Sussman
2
CMSC 714, Fall05 - Alan Sussman & Jeffrey K. Hollingsworth
Notes
zFirst programming assignment coming soon
– Slight change from original plan – you’ll write one program,
first using either OpenMP or MPI, then the other
zLorin Hochstein will talk at the end of class today on
his study of all of you writing parallel programs
zFirst, questions on OpenMP and UPC
– Directives vs. language extensions
3
CMSC 714, Fall05 - Alan Sussman & Jeffrey K. Hollingsworth
OpenMP + MPI
zSome applications can take advantage of both
message passing and threads
– Questions is what to do to obtain best overall performance,
without too much programming difficulty
– Choices are all MPI, all OpenMP, or both
•For both, common option is outer loop parallelized with
message passing, inner loop with directives to generate
threads
zApplications studied:
–Hydrology –CGWAVE
– Computational chemistry – GAMESS
– Linear algebra – matrix multiplication and QR factorization
– Seismic processing – SPECseis95
– Computational fluid dynamics – TLNS3D
– Computational physics - CRETIN
4
CMSC 714, Fall05 - Alan Sussman & Jeffrey K. Hollingsworth
Types of parallelism in the codes
zFor message passing parallelism (MPI)
– Parametric – coarse-grained outer loop, essentially task
parallel
– Structured domains – domain decomposition with local
operations – structured and unstructured grids
– Direct solvers – linear algebra, lots of communication and
load balancing required – message passing works well for
large systems of equations
zShared memory parallelism (OpenMP)
– Statically scheduled parallel loops – one large, or several
smaller loops, non-nested parallel
– Parallel regions – merge loops into one parallel region to
reduce overhead of directives
– Dynamic load balanced – when static scheduling leads to
load imbalance from irregular task sizes
5
CMSC 714, Fall05 - Alan Sussman & Jeffrey K. Hollingsworth
CGWAVE
zFinite elements - MPI parameter space evaluation at outer loop,
OpenMP sparse linear equation solver in inner loops
zSpeedup using 2 levels of parallelism allows modeling larger
bodies of water possible in a reasonable amount of time
zMaster-worker strategy for dynamic load balancing in MPI
part/component
zSolver for each component solves large sparse linear system
with OpenMP to parallelize
zOn SGI Origin 2000 (distributed shared memory machine), use
first touch rule to migrate data for each component to the
processor that uses it
zPerformance results show that best performance obtained using
both MPI and OpenMP, with a combination of MPI workers and
OpenMP threads that depends on the problem/grid size
– And for load balancing, a lot fewer MPI workers than components
CMSC 714
Lecture 5
MPI vs. OpenMP
and
Titanium
Alan Sussman
