Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
This lecture was delivered by Dr. Hanif Durad at Pakistan Institute of Engineering and Applied Sciences, Islamabad (PIEAS) for Parallel Computing course. it includes: Message, Passing, Paradigm, Building, Blocks, MPI, Passing, Interface, Topologies, Embedding, Collective, Communication
Typology: Slides
1 / 87
This page cannot be seen from the preview
Don't miss anything!
Dr. Hanif Durad
2
^ Principles of Message-Passing Programming ^ The Building Blocks: Send and Receive Operations ^ MPI: the Message Passing Interface ^ Topologies and Embedding ^ Overlapping Communication with Computation ^ Collective Communication and Computation Operations ^ Groups and Communicators
^ The paradigm is one of the oldest and most widely usedapproaches for programming parallel computers.
^ its roots can be traced back in the early days of parallel proc. ^ its wide-spread adopted ^ There are two key attributes that characterize theparadigm.
^ Assumes a partitioned address space and ^ It supports only explicit parallelization
PC5.pdf
^ the programmer is fully aware of all the costs of nonlocalinteractions, and is more likely to think about algorithms(and mappings) that minimize interactions. ^ paradigm can be efficiently implemented on a widevariety of architectures.
^ For dynamic and/or unstructured interactions thecomplexity of the code written for this type of paradigmcan be very high for this reason.
^ The programmer is responsible:
^ for analyzing the underlying serial algorithm/application and ^ Identifying ways by which he/she can decompose thecomputations and extract concurrency. ^ Programming using the paradigm tends to be hard andintellectually demanding. ^ Properly written message-passing programs can oftenachieve very high performance and scale to a very largenumber of processes.
Dr. Hanif Durad
^ tasks or subsets of tasks synchronize to performinteractions ^ between these interactions, tasks executecompletely asynchronously ^ Since the interaction happens synchronously, it isstill quite easy to reason about the program
8
Dr. Hanif Durad
^
^
^ sendbuf points to a buffer that stores the data to be sent ^ recvbuf points to a buffer that stores the data to bereceived ^ nelems is the number of data units to be sent andreceived ^ dest is the identifier of the process that receives the data ^ source is the identifier of the process that sends the data
a = 100;
3.^
send(&a, 1, 1);
4.^
a = 0;
P1 receive(&a, 1, 0)printf("%d\n", a);
What is problem with this code??
^ Blocking Non-Buffered Send/Receive. ^ Blocking Buffered Send/Receive.
6.2.1.1.1 Idling Overheads in BlockingNon-Buffered Send/Recv (1/3) ^ 3 scenarios:a)^
the send is reached before the receive is posted, b)^
the send and receive are posted around the sametime, c)^
the receive is posted before the send is reached.
Figure 6. 1Handshake for a blocking non-buffered send/receive operation.It is easy to see that in cases where sender and receiver do notreach communication point at similar times, there can beconsiderable idling overheads.
(2/3)
6.2.1.1.2 Deadlocks in BlockingNon-Buffered Send/Recv (1/2)
1.^
send(&a, 1, 1);
4.^
receive(&b, 1, 1)
P1 send(&a, 1, 0);receive(&b, 1, 0)
Deadlocks in Blocking Non-BufferedSend/Recv (2/2) ^ The send at P0 waits for the matching receive at P1 ^ The send at process P1 waits for the correspondingreceive at P0, resulting in an infinite wait. ^ Deadlocks are very easy in blocking protocols and caremust be taken to break cyclic waits of the nature outlined. ^ In the above example, this can be corrected by replacingthe operation sequence of one of the processes by areceive and a send as opposed to the other way around. ^ This often makes the code more cumbersome and buggy.
