Cache Coherence Problem - High Performance Computing - Lecture Slides, Slides of Computer Science

Download Cache Coherence Problem - High Performance Computing - Lecture Slides and more Slides Computer Science in PDF only on Docsity!

High Performance Computing

Lecture 39

2

Cache Coherence Problem

 If each processor in a shared memory

multiple processor machine has a data cache

 Potential data consistency problem: the cache coherence problem  Shared variable modification, private cache

 Objective: processes shouldn’t read `stale’

data

 Solutions

 Hardware: cache coherence mechanisms  Software: compiler assisted cache coherence

4 X: 0 X: 1 X: 0 Read X Read X X: 0 Write X= X: 1 Read X: Cache Miss P1 P

Example: Write Once Protocol.

5

Snoopy Cache Coherence and Locks

7

Lock Implementation …

 But, many processes finding L=0 will all try and do Test&Set(L) causing a burst of bus traffic  Could try and prevent all of these processes from attempting Test&Set at about the same time

repeat

while (L);

wait ( different time for each process );

until (! Test&Set (L) );

8

PARALLEL PROGRAMMING

 Recall: Flynn’s SIMD, MIMD  How would you describe a program meant to run on each of these parallel computer classes?  SIMD: Single Program Multiple Data (SPMD)  MIMD: Multiple Program Multiple Data (MPMD)  The programs could involve cooperating activities using either  Shared memory: threads with shared variables  Message passing: communication using messages  Speedup = parallel sequential

T

T

10

Programming with Message Passing

 Need

1. Mechanism to create processes to execute on different processors
2. Mechanism to send/receive message

 Must specify the identity of the

communicating process

 Example

P1: send( &x, P2) P2: receive( &y, P1)

 Message passing libraries

 PVM (1980s)  MPI (1990s)

11

MPI References

1. Using MPI

Gropp, Lusk, Skjellum www.mcs.anl.gov/mpi/usingmpi

2. MPI: The Complete Reference

Snir, Otto, Huss-Lederman, Walker, Dongarra www.netlib.org/utk/papers/mpi-book/mpi-book.html

13

Key MPI Functions and Constants

 MPI_Init (int *argc, char ***argv)  MPI_Finalize (void)  MPI_Comm_rank (MPI_COMM comm, int *rank)  MPI_Comm_size (MPI_COMM comm, int *size)  MPI_Send (void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm)  MPI_Recv (void *buf, int count, MPI_Datatype datatype, int source, int tag, MPI_Comm comm, MPI_Status *status)  MPI_CHAR, MPI_INT, MPI_LONG, MPI_BYTE  MPI_ANY_SOURCE, MPI_ANY_TAG