Portland State University
1
ECE 587/687 โ Fall 2008
Alaa R. Alameldeen
ยฉ Copyright by Alaa Alameldeen and Haitham Akkary 2008
Caches and Memory-Level
Parallelism
Portland State University
ECE 587/687
2
Portland State University โECE 587/687 โ Fall 2008
Revisiting Processor Performance
๎CPU Execution Time =
(CPU clock cycles + Memory stall cycles)
x clock cycle time
๎For each instruction:
CPI = CPI(Perfect Memory)
+ Memory stall cycles per instruction
๎With no caches, all memory requests require main
memory access
๎Very long latency (discuss)
๎Caches filter out a lot of memory access to improve
execution time
3
Portland State University โECE 587/687 โ Fall 2008
Cache Performance
๎Memory stall cycles Per Instruction =
Cache Misses per instruction x miss penalty
๎Processor Performance:
CPI = CPI(Perfect Memory)
+ miss rate x miss penalty
๎Average memory access time =
Hit ratio x Hit latency + Miss ratio x Miss penalty
๎Cache hierarchies attempt to reduce average
memory access time
4
Portland State University โECE 587/687 โ Fall 2008
Cache Performance Metrics
๎Hit ratio: #hits / #accesses
๎Miss ratio: #misses / #accesses
๎Miss rate: Misses per instruction (or 1000 inst)
๎Miss rate = miss ratio x memory accesses per inst
๎Hit time: time from request issued to cache until data is
returned to the processor
๎Depends on cache design parameters
๎Bigger caches, larger associativity, more ports increase hit
time
๎Miss penalty: depends on memory hierarchy parameters
5
Portland State University โECE 587/687 โ Fall 2008
Memory Hierarchy Example
๎Levels in memory hierarchy:
๎First-level caches
๎Usually Split I & D caches
๎Small and fast
๎Second-level caches
๎Usually on die, SRAM cells
๎Main memory
๎DRAM cells, focus on density
๎Disk
๎Usually magnetic device
๎Non volatile, slow access
Processor
L1I$ L1D$
L2 Cache
Main Memory
Disk
6
Portland State University โECE 587/687 โ Fall 2008
Basic Cache Structure
๎Array of blocks (lines)
๎Each block is usually 32-128 bytes
๎Finding a block in cache:
๎Offset: byte offset in block
๎Index: Which set in the cache is the block
located
๎Tag: Need to match address tag in cache
Tag Index Offset
Data
Address
