Message Passing Programming - Parallel and Distributed Computing - Lecture Slides, Slides of Parallel Computing and Programming

Download Message Passing Programming - Parallel and Distributed Computing - Lecture Slides and more Slides Parallel Computing and Programming in PDF only on Docsity!

Message Passing

Programming with

MPI

Chapter 4

Outline

• Message-passing model
• Message Passing Interface (MPI)
• Coding MPI programs
• Compiling MPI programs
• Running MPI programs
• Benchmarking MPI programs

Task/Channel vs. Message-passing

Task/Channel Message-passing

Task Process

Explicit channels Message communication

Processes

• Number is specified at start-up time
• Remains constant throughout execution of program
• All execute same program (SPMD)
• Each has unique ID number
• Alternately performs computations and communications
• Communications and work-sharing are all explicit!

The Message Passing Interface

• Late 1980s: vendors had unique libraries
• 1989: Parallel Virtual Machine (PVM) developed at Oak Ridge National Lab
• 1992: Work on MPI standard begun
• 1994: Version 1.0 of MPI standard
• 1997: Version 2.0 of MPI standard
• Today: MPI (and PVM) are dominant message passing library standards

Circuit Satisfiability

1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Not satisfied

0

Partitioning: Functional Decomposition

 Embarrassingly parallel: No channels between tasks

Agglomeration and Mapping

• Properties of parallel algorithm
  • Fixed number of tasks
  • No communications between tasks
  • Time needed per task is variable
• Consult mapping strategy decision tree
  • Map tasks to processors in a cyclic fashion

Pop Quiz

• Assume n pieces of work, p processes, and cyclic allocation
• What is the most amount of work any process has?
• What is the least amount of work any process has?
• How many processes have the mostDocsity.com

Summary of Program Design

• Program will consider all 65,536 combinations of 16 boolean inputs
• Combinations allocated in cyclic fashion to processes
• Each process examines each of its combinations
• If it finds a satisfiable combination, it will print

Local Variables

*int main (int argc, char argv[]) { int i; int id; / Process rank / int p; / Number of processes / void check_circuit (int, int);

 Include argc and argv : they are needed

to initialize MPI

 One copy of every variable for each process

running this program

Initialize MPI

• First MPI function called by each process
• Not necessarily first executable statement
• Allows system to do any necessary setup

MPI_Init (&argc, &argv);

Communicator

MPI_COMM_WORLD

Communicator

0

2 1

3

4

5

Processes

Ranks

Communicator Name

Determine Number of Processes

• First argument is communicator
• Number of processes returned through

second argument

MPI_Comm_size (MPI_COMM_WORLD, &p);