Point-to-Point Communication - Parallel and Distributed Computing - Lecture Slides, Slides of Parallel Computing and Programming

Download Point-to-Point Communication - Parallel and Distributed Computing - Lecture Slides and more Slides Parallel Computing and Programming in PDF only on Docsity!

Point-to-point Communication

• Involves a pair of processes
• One process sends a message
• Other process receives the message

Send/Receive Not Collective

Function MPI_Recv

int MPI_Recv (

void *message, int count, MPI_Datatype datatype, int source, int tag, MPI_Comm comm, MPI_Status *status

)

Inside MPI_Send and MPI_Recv

Sending Process Receiving Process

Program Memory

System Buffer

System Buffer

Program Memory

MPI_Send (^) MPI_Recv

Return from MPI_Recv

• Function blocks until message in buffer
• If message never arrives, function never

returns

• Which leads us to …

Example

CS 491 – Parallel and Distributed Computing 8

float a, b, c; int id; MPI_Status status; …

if (id == 0) { MPI_Recv (&b, 1, MPI_FLOAT, 1, 0, MPI_COMM_WORLD, &status); MPI_Send (&a, 1, MPI_FLOAT, 1, 0, MPI_COMM_WORLD); c = (a + b) / 2.0; } else if (id == 1) { MPI_Recv (&a, 1, MPI_FLOAT, 0, 0, MPI_COMM_WORLD, &status); MPI_Send (&b, 1, MPI_FLOAT, 0, 0, MPI_COMM_WORLD); c = (a + b) / 2.0; }

Deadlock

• Deadlock : process waiting for a condition that

will never become true

• Easy to write send/receive code that

deadlocks

• Two processes: both receive before send
• Send tag doesn’t match receive tag
• Process sends message to wrong destination

process

Coding Send/Receive

if (ID == j) { … Receive from i … } … if (ID == i) { … Send to j … } …

Receive is before Send. Why does this work?

Getting Data Places

• There are many interesting ways to arrange

and distribute data for parallel use.

• Many of these follow some fairly common

“patterns” – basic structures

• MPI Standards group wanted to provide

flexible ways to distribute data

• Uses variation on the concepts of “scatter” and

“gather”

CS 491 – Parallel and Distributed Computing (^) Docsity.com^13

Collective Communications

Scatter the vectors among N processors as zpart, xpart, and ypart.

Gather the results back to the root processor when completed.

Calls can return as soon as their participation is complete.

Broadcast the coefficients to all processors.

z ax by

Gather

int MPI_Gather(void* sendbuf,

int sendcount,

MPI_Datatype sendtype,

void* recvbuf,

int recvcount,

MPI_Datatype recvtype,

int root,

MPI_Comm comm)

CS 491 – Parallel and Distributed Computing (^) Docsity.com^16

Scatterv

int MPI_Scatterv(void* sendbuf,

int *sendcounts,

int *displs,

MPI_Datatype sendtype,

void* recvbuf,

int recvcount,

MPI_Datatype recvtype,

int root, MPI_Comm comm)

CS 491 – Parallel and Distributed Computing (^) Docsity.com^17