Download MPI Programming: Message Passing Interface with Sanjay Rajopadhye - Prof. Sanjay V. Rajopa and more Study notes Computer Science in PDF only on Docsity!
Message Passing Interface
(Chapter 4)
Sanjay Rajopadhye
Colorado State University
Fall 2008 Week 3
Outline
Background & History
Message passing programming model
SPMD
Distributed memory
Writing MPI programs
Compiling, running and benchmarking
Background & History
Prior to 1990: vendor specific libraries
No portability
Parallel Virtual Machine (PVM) [ORNL 1989] public
release 1993
enables portable parallel programming
Parallel [sic] efforts to develop an open, portable
general purpose parallel programming: 1992-
Parallel I/O [1997]
Nearly ubiquitous on “big iron”
“Assembly language” of parallel programming
similar advantages/disadvantages
Foundation
SPMD: Single Program Multiple Data
write a single program using a special library compile it with special compiler (that knows about the library) execute “it” in an environment created by the environment variables of your session (as on bassi) with a special command on your local machine (e.g., Lam MPI) multiple instances of the executable run in parallel Each instance has its own local data (distributed memory) MPI library provides functions for coordinating and communicating amongst the instances
Example library calls
MPI_Init
MPI_Comm_rank
MPI_Comm_size
MPI_Reduce
MPI_Finalize
Example program Contrived program: don’t take (too) seriously Boolean Circuit Satisfiability
Given a Boolean circuit (i.e., a function that takes n inputs
and produces a Boolean output (using and, or and not)
2 n^ possible inputs ( n -bit sequences)
Which input combinations produce a 1 on the output (i.e.,
satisfy the circuit)?
Difficult problem in computer science (NP-complete) Solution (algorithm) NP Complete: no known polynomial time algorithm Accept approximate solutions Solve by “exhaustive” enumeration (exponential time)
our approach (we seek all solutions)
Example Circuit
Communicators Rank and Size MPI_Comm_size(communicator comm, int &size) Communicator is input size is returned Number of processes in the communicator MPI_Comm_rank(communicator comm, int &rank) Communicator is input rank is returned id of the calling process in the communicator (integer between 0 and size)
All processes have a private copy of ALL program variables
Cyclic allocation of work main(int argc, char *argv[]) { int i, id, p; void check_circuit(int, int); MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &id); MPI_Comm_size(MPI_COMM_WORLD, &p); for (i = id; i < NUMREPS; i += p) check_circuit(id, i); printf("Process %d is done.\n", id); fflush(stdout); MPI_Finalize(); return 0;} Compiling and executing
Please see
http://www.cs.colostate.edu/~cs475/LabClass/circuit/instructions.html
Execution notes
Non-deterministic order of printf outputs
Multiple prints by same process will always occur in the order they are executed. calls by different processes will be queued (importance of flush) the system may interleave the processing of the commands Extensions
In addition to printing out the solutions we want to count how
many inputs satisfy the circuit
Introduce your first (collective) communication call
MPI_Reduce: combines values in different processes into a
single result
Details Each process has a counter to maintain how many solutions it found modify check_circuit return 0 if not satisfiable return 1 if satisfiable Introduce your first (collective) communication call at the end of the loop MPI_Reduce: combines values in different processes into a single result Call after the for loop MPI_Reduce int MPI_Reduce ( void sendbuf, / ptr to first argument */ void recvbuf, / ptr to first result / int count, / number of values to combine / MPI_Datatype datatype, / type / MPI_Op op, / operator / int root, / destination */ MPI_Comm comm )
