Download Intel Montecito - Parallel Computer Architecture - Lecture Slides and more Slides Computer Science in PDF only on Docsity!
Module 18: "TLP on Chip: HT/SMT and CMP"
Lecture 41: "Case Studies: Intel Montecito and Sun Niagara"
TLP on Chip: HT/SMT and CMP
Intel Montecito
Features
Overview
Dual threads
Thread urgency
Core arbiter
Power efficiency
Foxton technology
Die photo
Sun Niagara OR Ultrasparc T
Features
Pipeline details
Cache hierarchy
Thread selection
Module 18: "TLP on Chip: HT/SMT and CMP"
Lecture 41: "Case Studies: Intel Montecito and Sun Niagara"
Intel Montecito
Features
Dual core Itanium 2, each core dual threaded
1.7 billion transistors, 21.5 mm x 27.7 mm die
27 MB of on-chip three levels of cache
Not shared among cores
1.8+ GHz, 100 W
Single-thread enhancements
Extra shifter improves performance of crypto codes by 100%
Improved branch prediction
Improved data and control speculation recovery
Separate L2 instruction and data caches buys 7% improvement over Itanium2; four times
bigger L2I (1 MB)
Asynchronous 12 MB L3 cache
Overview
Reproduced from IEEE Micro
Dual threads
SMT only for cache, not for core resources
Simulations showed high resource utilization at core level, but low utilization of cache
Branch predictor is still shared but use thread id tags
Thread switch is implemented by flushing the pipe
More like coarse-grain multithreading
Five thread switch events
Module 18: "TLP on Chip: HT/SMT and CMP"
Lecture 41: "Case Studies: Intel Montecito and Sun Niagara"
Thread urgency
Each thread has eight urgency levels
Every L3 miss decrements urgency by one
Every L3 refill increments urgency by one until urgency reaches 5
A switch due to time quantum expiry sets the urgency of the switched thread to 7
Arrival of asynchronous interrupt for a background thread sets the urgency level of that
thread to 6
Switch from L3 miss requires urgency level to be compared also
Reproduced from IEEE Micro
Core arbiter
Reproduced from IEEE Micro
Module 18: "TLP on Chip: HT/SMT and CMP"
Lecture 41: "Case Studies: Intel Montecito and Sun Niagara"
Sun Niagara OR Ultrasparc T
Features
Eight pipelines or cores, each shared by 4 threads
32-way multithreading on a single chip
Starting frequency of 1.2 GHz, consumes 60 W
Shared 3 MB L2 cache, 4-way banked, 12-way set associative, 200 GB/s bandwidth
Single-issue six stage pipe
Target market is web service where ILP is limited, but TLP is huge (independent
transactions)
Throughput matters
Pipeline details
Reproduced from IEEE Micro
Four threads share a six-stage pipeline
Shared L1 caches and TLBs
Dedicated register file per thread
Fetches two instructions every cycle from a selected thread
Thread select logic also determines which thread’s instruction should be fed into the pipe
Although pipe is in-order, there is an 8-entry store buffer per thread (why?)
Threads may run into structural hazards due to limited number of FUs
Module 18: "TLP on Chip: HT/SMT and CMP"
Lecture 41: "Case Studies: Intel Montecito and Sun Niagara"
Cache hierarchy
L1 instruction cache
16 KB / 4-way / 32 bytes / random replacement
Fetches two instructions every cycle
If both instructions are useful, next cycle is free for icache refill
L1 data cache
8 KB / 4-way / 16 bytes/ write-through, no-allocate
On avearge 10% miss rate for target benchmarks
L2 cache extends the tag to maintain a directory for keeping the core L1s coherent
L2 cache is writeback with silent clean eviction
Thread selection
Based on long latency events such as load, divide, multiply, branch
Also based on pipeline stalls due to cache misses, traps, or structural hazards
Speculative load dependent issue with low priority
Solution of Exercise : 1
- [10 points] Suppose you are given a program that does a fixed amount of work, and some fraction s of that work must be done sequentially. The remaining portion of the work is perfectly parallelizable on P processors. Derive a formula for execution time on P processors and establish an upper bound on the achievable speedup.
Solution: Execution time on P processors, T(P) = sT(1) + (1-s)T(1)/P. Speedup = 1/(s + (1-s)/P). Upper bound is achieved when P approaches infinity. So maximum speedup = 1/s. As expected, the upper bound on achievable speedup is inversely proportional to the sequential fraction.
- [40 points] Suppose you want to transfer n bytes from a source node S to a destination node D and there are H links between S and D. Therefore, notice that there are H+1 routers in the path (including the ones in S and D). Suppose W is the node-to-network bandwidth at each router. So at S you require n/W time to copy the message into the router buffer. Similarly, to copy the message from the buffer of router in S to the buffer of the next router on the path, you require another n/W time. Assuming a store- and-forward protocol total time spent doing these copy operations would be (H+2)n/W and the data will end up in some memory buffer in D. On top of this, at each router we spend R amount of time to figure out the exit port. So the total time taken to transfer n bytes from S to D in a store-and-forward protocol is (H+2)n/W+(H+1)R. On the other hand, if you assume a cut-through protocol the critical path would just be n/W+(H+1)R. Here we assume the best possible scenario where the header routing delay at each node is exposed and only the startup n/W delay at S is exposed. The rest is pipelined. Now suppose that you are asked to compare the performance of these two routing protocols on an 8x8 grid. Compute the maximum, minimum, and average latency to transfer an n byte message in this topology for both the protocols. Assume the following values: W=3.2 GB/s and R=10 ns. Compute for n=64 and
- Note that for each protocol you will have three answers (maximum, minimum, average) for each value of n. Here GB means 10^9 bytes and not 2^30 bytes.
Solution: The basic problem is to compute the maximum, minimum, and average values of H. The rest is just about substituting the values of the parameters. The maximum value of H is 14 while the minimum is 1. To compute the average, you need to consider all possible messages, compute H for them, and then take the average. Consider S=(x0, y0) and D=(x1, y1). So H = |x0-x1| + |y0-y1|. Therefore, average H = (sum over all x0, x1, y0, y1 |x0-x1| + |y0-y1|)/(6463), where each of x0, x1, y0, y1 varies from 0 to 7. Clearly, this is same as (sum over x0, x1 |x0-x1| + sum over y0, y1 |y0-y1|)/63, which in turn is equal to 2(sum over x0, x1 |x0-x1|)/63 = 2*(sum over x0=0 to 7, x1=0 to x0 (x0-x1)+ sum over x0=0 to 7, x1=x0+1 to 7 (x1-x0))/63 = 16/3.
- [20 points] Consider a simple computation on an nxn double matrix (each element is 8 bytes) where each element A[i][j] is modified as follows. A[i][j] = A[i][j] - (A[i-1][j] + A[i+1][j] + A[i][j-1] + A[i][j+1])/4. Suppose you assign one matrix element to one processor (i.e. you have n^2 processors). Compute the total amount of data communication between processors.
Solution: Each processor requires the four neighbors i.e. 32 bytes. So total amount of data communicated is 32n^2.
- [30 points] Consider a machine running at 10^8 instructions per second on some workload with the following mix: 50% ALU instructions, 20% load instructions, 10% store instructions, and 20% branch instructions. Suppose the instruction cache miss rate is 1%, the writeback data cache miss rate is 5%, and the cache line size is 32 bytes. Assume that a store miss requires two cache line transfers, one to load the newly updated line and one to replace the dirty line at a later point in time. If the machine
Solution of Exercise : 2
[Thanks to Saurabh Joshi for some of the suggestions.]
- [30 points] For each of the memory reference streams given in the following, compare the cost of executing it on a bus-based SMP that supports (a) MESI protocol without cache-to-cache sharing, and (b) Dragon protocol. A read from processor N is denoted by rN while a write from processor N is denoted by wN. Assume that all caches are empty to start with and that cache hits take a single cycle, misses requiring upgrade or update take 60 cycles, and misses requiring whole block transfer take 90 cycles. Assume that all caches are writeback.
Solution: Stream1: r1 w1 r1 w1 r2 w2 r2 w2 r3 w3 r3 w
(a) MESI: read miss, hit, hit, hit, read miss, upgrade, hit, hit, read miss, upgrade, hit, hit. Total latency = 90+1+1+1+2(90+60+1+1) = 397 cycles (b) Dragon: read miss, hit, hit, hit, read miss, update, hit, update, read miss, update, hit, update. Total latency = 90+1+1+1+2(90+60+1+60) = 515 cycles
Stream2: r1 r2 r3 w1 w2 w3 r1 r2 r3 w3 w
(a) MESI: read miss, read miss, read miss, upgrade, readX, readX, read miss, read miss, hit, upgrade, readX. Total latency = 90+90+90+60+90+90+90+90+1+60+90 = 841 cycles (b) Dragon: read miss, read miss, read miss, update, update, update, hit, hit, hit, update, update. Total latency = 90+90+90+60+60+60+1+1+1+60+60=573 cycles
Stream3: r1 r2 r3 r3 w1 w1 w1 w1 w2 w
(a) MESI: read miss, read miss, read miss, hit, upgrade, hit, hit, hit, readX, readX. Total latency = 90+90+90+1+60+1+1+1+90+90 = 514 cycles (b) Dragon: read miss, read miss, read miss, hit, update, update, update, update, update, update. Total latency=90+90+90+1+60*6=631 cycles
[For each stream for each protocol: 5 points]
- [15 points] (a) As cache miss latency increases, does an update protocol become more or less preferable as compared to an invalidation based protocol? Explain.
Solution: If the system is bandwidth-limited, invalidation protocol will remain the choice. However, if there is enough bandwidth, with increasing cache miss latency, invalidation protocol will lose in importance.
(b) In a multi-level cache hierarchy, would you propagate updates all the way to the first-level cache? What are the alternative design choices?
Solution: If updates are not propagated to L1 caches, on an update the L1 block must be invalidated/retrieved to the L2 cache.
(c) Why is update-based protocol not a good idea for multiprogramming workloads running on SMPs?
Solution: Pack-rat. Discussed in class.
- [20 points] Assuming all variables to be initialized to zero, enumerate all outcomes possible under
sequential consistency for the following code segments.
(a) P1: A=1; P2: u=A; B=1; P3: v=B; w=A;
Solution: If u=1 and v=1, then w must be 1. So (u, v, w) = (1, 1, 0) is not allowed. All other outcomes are possible.
(b) P1: A=1; P2: u=A; v=B; P3: B=1; P4: w=B; x=A;
Solution: Observe that if u sees the new value A, v does not see the new value of B, and w sees that new value of B, then x cannot see the old value of A. So (u, v, w, x) = (1, 0, 1, 0) is not allowed. Similarly, if w sees the new value of B, x sees the old value of A, u sees the new value of A, then v cannot see the old value B. So (1, 0, 1, 0) is not allowed, which is already eliminated in the above case. All other 15 combinations are possible.
(c) P1: u=A; A=u+1; P2: v=A; A=v+1;
Solution: If v=A happens before A=u+1, then the final (u, v, A) = (0, 0, 1). If v=A happens after A=u+1, then the final (u, v, A) = (0, 1, 2). Since u and v are symmetric, we will also observe the outcome (1, 0, 2) in some cases.
(d) P1: fetch-and-inc (A) P2: fetch-and-inc (A)
Solution: The final value of A is 2.
- [30 points] Consider a quad SMP using a MESI protocol (without cache-to-cache sharing). Each processor tries to acquire a test-and-set lock to gain access to a null critical section. Assume that test- and-set instructions always go on the bus and they take the same time as the normal read transactions. The initial condition is such that processor 1 has the lock and processors 2, 3, 4 are spinning on their caches waiting for the lock to be released. Every processor gets the lock once, unlocks, and then exits the program. Consider the bus transactions related to the lock/unlock operations only.
(a) What is the least number of transactions executed to get from the initial to the final state? [ points]
Solution: 1 unlocks, 2 locks, 2 unlocks (no transaction), 3 locks, 3 unlocks (no transaction), 4 locks, 4 unlocks (no transaction). Notice that in the best possible scenario, the timings will be such that when someone is in the critical section no one will even attempt a test-and-set. So when the lock holder unlocks, the cache block will still be in its cache in M state.
(b) What is the worst-case number of transactions? [5 points]
Solution: Unbounded. While someone is holding the lock, other contending processors may keep on invalidating each other indefinite number of times.
(c) Answer the above two questions if the protocol is changed to Dragon. [15 points]
Solution: Observe that it is an order of magnitude more difficult to implement shared test-and-set locks (LL/SC-based locks are easier to implement) in a machine running an update-based protocol. In a straightforward implementation, on an unlock everyone will update the value in cache and then will try to
Solution: Multiple of 8 integers (one cache block is eight integers).
- [20 points] The following barrier implementation is wrong. Make as little change as possible to correct it.
struct bar_struct { LOCKDEC(lock); int count; // Initialized to zero int releasing; // Initialized to zero } bar;
void BARRIER (int P) { LOCK(bar.lock); bar.count++; if (bar.count == P) { bar.releasing = 1; bar.count--; } else { UNLOCK(bar.lock); while (!bar.releasing); LOCK(bar.lock); bar.count--; if (bar.count == 0) { bar.releasing = 0; } } UNLOCK(bar.lock); }
Solution: There are too many problems with this implementation. I will not list them here. The correct barrier code is given below which requires addition of one line of code. Notice that the releasing variable nicely captures the notion of sense reversal.
void BARRIER (int P) { while (bar.releasing); // New addition LOCK(bar.lock); bar.count++; if (bar.count == P) { bar.releasing = 1; bar.count--; } else { UNLOCK(bar.lock); while (!bar.releasing); LOCK(bar.lock); bar.count--; if (bar.count == 0) { bar.releasing = 0; } } UNLOCK(bar.lock); }
- [30] Suppose you want to transpose a matrix in parallel. The source matrix is A and the destination matrix is B. Both A and B are decomposed in such a way that each node gets a chunk of consecutive rows. Application-directed page placement is used to map the pages belonging to each chunk in the local memory of the respective nodes. Now the transpose can be done in two ways. The first algorithm, known as "local read algorithm", allows each node to transpose the band of rows local to it. So naturally this algorithm involves a large fraction of remote writes to matrix B. Assume that the band is wide enough so that there is no false sharing when writing to matrix B. The second algorithm, known as "local write algorithm", allows each node to transpose a band of columns of A such that all the writes to B are local. Naturally, this algorithm involves a large number of remote reads in matrix A. Assume that the algorithms are properly tiled so that the cache utilization is good. In both the cases, before doing the transpose, every node reads and writes to its local segment in A and after doing the transpose every node reads and writes to its local segment in B. Assuming an invalidation-based cache coherence protocol, briefly but clearly explain which algorithm is expected to deliver better performance. How much synchronization does each algorithm require (in terms of the number of critical sections and barriers)? Assume that the caches are of infinite capacity and that a remote write is equally expensive in all respects as a remote read because in both cases the retirement is held up for a sequentially consistent implementation.
Solution: Analysis of local read algorithm: Before the algorithm starts, the band of rows to be transposed by a processor is already in its cache. Each remote write to a cache block of B involves a 2- hop transaction (requester to home and back) without involving any invalidation. After transpose each processor has a large number of remote cache blocks in its cache. Here a barrier is needed before a processor is allowed to work on its local rows of B. After the barrier each cache read or write miss to B involves a 2-hop intervention (local home to owner and back). So in this algorithm, each processor suffers from local misses before transpose, 2-hop misses during transpose, and 2-hop misses after transpose.
Analysis of local write algorithm: Before the algorithm starts, the band of columns to be transposed by a processor is quite distributed over the system. In fact, 1/P portion of the column would be in the local cache. Before the transpose starts, a barrier is needed. In the transpose phase, each cache miss of A involves a 2-hop transaction (requester to home's cache and back). Each cache miss to B is a local miss. After the transpose the local band of rows of B is already in the cache of each processor. So they enjoy hits in this phase. So in this algorithm, each processor suffers from local misses before transpose, 2-hop misses and local misses during transpose, and hits after transpose.
Both the algorithms have the same number of misses, but the local write algorithm has more local misses and less remote misses. Both have the same synchronization requirement. Local write algorithm is better and that's what SPLASH-2 FFT implements. Cheers!
- [5+5] If a compiler reorders accesses according to WO and the underlying processor is SC, what is the consistency model observed by the programmer? What if the compiler produces SC code, but the processor is RC?
Solution: WO compiler and SC hardware: programmer sees WO because an SC processor will execute whatever is presented to it in the intuitive order. But since the instruction stream presented to it is already WO, it cannot do any better.
SC compiler and RC hardware: programmer sees RC.
- [5+5] Consider the following piece of code running on a faulty microprocessor system which does not preserve any program order other than true data and control dependence order.
LOCK (L1) load A
store A UNLOCK (L1) load B store B LOCK (L1) load C store C UNLOCK (L1)
(a) Insert appropriate WMB and MB instructions to enforce SC. Do not over-insert anything. (b) Repeat part (a) to enforce RC.
Solution: (a) MB LOCK(L1) MB load A store A WMB // Need to hold the unlock back before the store to A is done UNLOCK (L1) MB load B store B MB LOCK (L1) MB load C store C WMB // Need to hold the unlock back before the store to C is done UNLOCK (L1) MB
(b) LOCK (L1) MB // This MB wouldn't be needed if the processor was not faulty; you lose some RC advantage load A store A WMB UNLOCK (L1) load B store B LOCK (L1) MB // This MB wouldn't be needed if the processor was not faulty; you lose some RC advantage load C store C WMB UNLOCK (L1)
- [5+10] Consider implementing a directory-based protocol to keep the private L1 caches of the cores
