Memory
Memory Hierarchy
Registers
CS 21 - Computer Organization and Assembly Language Programming Lecture 5 Memory: Registers and Cache
University of the Philippines - Diliman College of Engineering Department of Computer Science
Cache
Memory
Memory Hierarchy
Outline
1
Memory
2
Memory Hierarchy
3
Registers
4
Cache
Registers
Cache
Memory
Memory Hierarchy
Outline
1
Memory
2
Memory Hierarchy
3
Registers
4
Cache
Registers
Cache
Memory
Memory Hierarchy
Memory
What is memory?
Registers
Cache
Memory
Memory Hierarchy
Registers
Memory
What is memory? in human terms, it is the totality of what you remember, your recollections
Cache
Memory
Memory Hierarchy
Registers
Memory
What is memory? in human terms, it is the totality of what you remember, your recollections largely a defining factor in who you are
Cache
Memory
Memory Hierarchy
Memory
What is memory in computers?
Registers
Cache
Memory
Memory Hierarchy
Registers
Memory
What is memory in computers? structure used to store data and instructions
Cache
Memory
Memory Hierarchy
Registers
Memory
What is memory in computers? structure used to store data and instructions exhibits the widest range of type, technology, organization, performance and cost in any feature of a computer system
Cache
Memory
Memory Hierarchy
Registers
Memory
What is memory in computers? structure used to store data and instructions exhibits the widest range of type, technology, organization, performance and cost in any feature of a computer system remember: no memory = no computer!
Cache
Memory
Memory Hierarchy
Registers
Memory
What are the types of computer memory, by location?
Cache
Memory
Memory Hierarchy
Registers
Memory
What are the types of computer memory, by location? CPU memory
Cache
Memory
Memory Hierarchy
Registers
Memory
What are the types of computer memory, by location? CPU memory Internal memory
Cache
Memory
Memory Hierarchy
Registers
Memory
What are the types of computer memory, by location? CPU memory Internal memory External memory
Cache
Memory
Memory Hierarchy
Outline
1
Memory
2
Memory Hierarchy
3
Registers
4
Cache
Registers
Cache
Memory
Memory Hierarchy
Memory Hierarchy
Three key characteristics of memory:
Registers
Cache
Memory
Memory Hierarchy
Memory Hierarchy
Three key characteristics of memory: cost
Registers
Cache
Memory
Memory Hierarchy
Memory Hierarchy
Three key characteristics of memory: cost capacity
Registers
Cache
Memory
Memory Hierarchy
Memory Hierarchy
Three key characteristics of memory: cost capacity access time Key relationships:
Registers
Cache
Memory
Memory Hierarchy
Memory Hierarchy
Three key characteristics of memory: cost capacity access time Key relationships: smaller access time, greater cost per bit
Registers
Cache
Memory
Memory Hierarchy
Memory Hierarchy
Three key characteristics of memory: cost capacity access time Key relationships: smaller access time, greater cost per bit greater capacity, greater access time
Registers
Cache
Memory
Memory Hierarchy
Registers
Memory Hierarchy
What do we want? In terms of capacity? In terms of cost? In terms of access time?
Cache
Memory
Memory Hierarchy
Registers
Memory Hierarchy
What do we want? In terms of capacity? In terms of cost? In terms of access time? Could we get all these by using a single type of memory?
Cache
Memory
Memory Hierarchy
Registers
Memory Hierarchy
What do we want? In terms of capacity? In terms of cost? In terms of access time? Could we get all these by using a single type of memory? No.
Cache
Memory
Memory Hierarchy
Registers
Memory Hierarchy
What do we want? In terms of capacity? In terms of cost? In terms of access time? Could we get all these by using a single type of memory? No. So what do we do?
Cache
Memory
Memory Hierarchy
Registers
Memory Hierarchy
What do we want? In terms of capacity? In terms of cost? In terms of access time? Could we get all these by using a single type of memory? No. So what do we do? We use a memory hierarchy
Cache
Memory Hierarchy
What is Memory Hierarchy?
A memory hierarchy uses several types of memory. The amount of memory type present in the hierarchy is inversely proportional to its cost. Thus, a memory hierarchy features very few of fast, expensive memory types and lots of slow, cheap memory types.
Memory
Memory Hierarchy
Registers
Memory Hierarchy
Important Concept The hierarchy is INVISIBLE to the user.
Cache
Memory
Memory Hierarchy
Registers
Memory Hierarchy
Important Concept The hierarchy is INVISIBLE to the user. Except register when programming in assembly language.
Cache
Memory
Memory Hierarchy
Registers
Memory Hierarchy
Important Concept In the memory hierarchy pyramid, a higher level only contains a copy of the subset of the memory below it.
Cache
Memory
Memory Hierarchy
Memory Hierarchy
What makes Memory Hierarchy possible?
Registers
Cache
Memory
Memory Hierarchy
Registers
Memory Hierarchy
What makes Memory Hierarchy possible? Spatial and Temporal Locality
Cache
Memory
Memory Hierarchy
Registers
What is spatial locality?
Spatial locality is the tendency of programs to access items from memory that are near each other. Examples: linear program execution(no branches), arrays
Cache
Memory
Memory Hierarchy
Registers
What is temporal locality?
Spatial locality is the tendency of programs to access certain items from memory again and again or multiple times. Examples: loops, counters
Cache
Memory
Memory Hierarchy
Outline
1
Memory
2
Memory Hierarchy
3
Registers
4
Cache
Registers
Cache
Memory
Memory Hierarchy
Registers
Registers
What is a register? Memory unit closest to the ALUs Registers are contained within the CPU Fastest of all memory units(implication in terms of cost?) Grouped into register files, which are then grouped between INTEGER and FLOATING POINT
Cache
Memory
Memory Hierarchy
Registers Why do we limit the number of registers?
Registers
Cache
Memory
Memory Hierarchy
Registers
Analogy in the human mind:
Registers
Cache
Memory
Memory Hierarchy
Registers
Analogy in the human mind: Short term memory
Registers
Cache
Registers Number of registers in historical processors
Memory
Memory Hierarchy
Outline
1
Memory
2
Memory Hierarchy
3
Registers
4
Cache
Registers
Cache
Memory
Memory Hierarchy
Registers
Cache
What is a cache? intended to provide speed and size an �intermediary� between main memory and register memory usual size: between 1K and 512K words
Cache
Memory
Memory Hierarchy
Registers
Cache
Key Concept 1 In systems with cache, processor could only access cache...
Cache
Memory
Memory Hierarchy
Registers
Cache
Key Concept 1 In systems with cache, processor could only access cache... also, processor could not access main memory directly
Cache
Memory
Memory Hierarchy
Registers
Cache
Key Concept 2 Cache memory is a TEMPORARY COPY of main memory.
Cache
Memory
Memory Hierarchy
Registers
Cache
Key Concept 2 Cache memory is a TEMPORARY COPY of main memory. And only a very SMALL PART of main memory is copied.
Cache
Memory
Memory Hierarchy
Registers
Cache Operation Example assume that we have a processor with cache
Cache
Memory
Memory Hierarchy
Cache Operation Example the processor executes a load instruction
Registers
Cache
Memory
Memory Hierarchy
Registers
Cache Operation Example the processor tries to load the data, but address location A not in the cache
Cache
Memory
Memory Hierarchy
Registers
Cache Operation Example the cache gets copies entry from main memory
Cache
Memory
Memory Hierarchy
Registers
Cache Operation Example processor finally gets the data it needs from the cache
Cache
Memory
Memory Hierarchy
Registers
Cache Operation Example the next instruction executed by the processor is another load A
Cache
Memory
Memory Hierarchy
Registers
Cache Operation Example but since address location A is already in the cache, main memory no longer is accessed
Cache
Memory
Memory Hierarchy
Registers
Cache Performance hit - memory access that is found in the cache miss - memory access that is NOT found in the cache hit rate - percentage of memory access that is found in the cache miss rate - percentage of memory access that is NOT found in the cache
Cache
Memory
Memory Hierarchy
Registers
Cache Performance hit - memory access that is found in the cache miss - memory access that is NOT found in the cache hit rate - percentage of memory access that is found in the cache miss rate - percentage of memory access that is NOT found in the cache Problem: A certain program has 100 memory accesses(all reads), of which 82 are found in the cache. 1
What is the hit rate?
2
What is the miss rate?
Cache
Memory
Memory Hierarchy
Registers
Cache Performance hit - memory access that is found in the cache miss - memory access that is NOT found in the cache hit rate - percentage of memory access that is found in the cache miss rate - percentage of memory access that is NOT found in the cache Problem: A certain program has 100 memory accesses(all reads), of which 82 are found in the cache. 1
What is the hit rate?
2
What is the miss rate?
Important Concept: The hit rate and miss rate partly depends on the program being ran.
Cache
Memory
Memory Hierarchy
Registers
Cache Performance
hit time - time needed to access data FOUND in the cache, includes time spent in checking the cache whether it’s there or not time spent getting data from cache to processor time that is paid by all accesses
miss penalty - time needed to transfer data from main memory to cache ¨ misses additional time p ¨aidby
Cache
Memory
Memory Hierarchy
Registers
Cache Performance
hit time - time needed to access data FOUND in the cache, includes time spent in checking the cache whether it’s there or not time spent getting data from cache to processor time that is paid by all accesses
miss penalty - time needed to transfer data from main memory to cache ¨ misses additional time p ¨aidby
Problem: A certain program has 100 memory accesses(all reads), of which 82 are found in the cache. Assuming that the hit time is 2ns, and the miss penalty is 7ns, what is the total time spent for memory access?
Cache
Memory
Memory Hierarchy
Registers
Cache Performance
Important Concept: The hit time and miss penalty depend on the hardware: cache architecture and the IC fabrication process.
Cache
Memory
Memory Hierarchy
Registers
Cache Performance
Important Concept: The hit time and miss penalty depend on the hardware: cache architecture and the IC fabrication process. Point to ponder: Miss penalty + Hitrate vs Access time if there’s no cache?
Cache
Memory
Memory Hierarchy
Cache
Elements of Cache Design Cache size Mapping Function Replacement Algorithm Write Policy
Registers
Cache
Memory
Memory Hierarchy
Elements of Cache Design: Cache Size
What is an �optimal� cache size?
Registers
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Cache Size
What is an ”optimal” cache size? Big enough that we don’t waste time accessing main memory
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Cache Size
What is an ”optimal” cache size? Big enough that we don’t waste time accessing main memory Small enough that it is still fast and cheap.
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Cache Size
What is an ”optimal” cache size? Big enough that we don’t waste time accessing main memory Small enough that it is still fast and cheap. Does a definite optimal cache size exist?
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Mapping function
What is a Mapping function?
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Mapping function
What is a Mapping function? Policy on where data from the main memory would reside on the cache.
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Mapping function
What is a Mapping function? Policy on where data from the main memory would reside on the cache. Three types of mapping function:
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Mapping function
What is a Mapping function? Policy on where data from the main memory would reside on the cache. Three types of mapping function: Direct mapping Fully Associative mapping Set Associative mapping
Cache
Memory
Memory Hierarchy
Registers
Mapping function: Direct Mapping each memory location is mapped to exactly ONE location in the cache lower bits of an address used in mapping
Cache
Memory
Memory Hierarchy
Registers
Mapping function: Fully Associative Mapping a memory location could go to ANY cache location How to know it’s in the cache? Search every valid entry!
Cache
Memory
Memory Hierarchy
Registers
Direct Mapping vs Fully Associative Mapping
Direct Mapping very high miss rate very low hit time
Fully Associative Mapping very low miss rate very high hit time
Cache
Memory
Memory Hierarchy
Analogy for Direct Mapping
Registers
Cache
Memory
Memory Hierarchy
Analogy for Fully Associative Mapping
Registers
Cache
Memory
Memory Hierarchy
Registers
Mapping function: Set Associative Mapping
hybrid between Direct Mapping and Fully Associative Mapping location in main memory could be mapped to a SELECT number of locations level of grouping determined again by lower bits when searching, you search a few tag entries if an entry could go to n locations, it’s called an n-way associative cache
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Replacement Algorithms
Deals with the questions ”Which one do we replace?” ”Which one de we kick out?”
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Replacement Algorithms
Deals with the questions ”Which one do we replace?” ”Which one de we kick out?” Question: What Replacement Algorithm is used in a Direct Mapped Cache? Is there one?
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Replacement Algorithms
Deals with the questions ”Which one do we replace?” ”Which one de we kick out?” Question: What Replacement Algorithm is used in a Direct Mapped Cache? Is there one? FIFO
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Replacement Algorithms
Deals with the questions ”Which one do we replace?” ”Which one de we kick out?” Question: What Replacement Algorithm is used in a Direct Mapped Cache? Is there one? FIFO LRU
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Replacement Algorithms
Deals with the questions ”Which one do we replace?” ”Which one de we kick out?” Question: What Replacement Algorithm is used in a Direct Mapped Cache? Is there one? FIFO LRU LFU
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Replacement Algorithms
Deals with the questions ”Which one do we replace?” ”Which one de we kick out?” Question: What Replacement Algorithm is used in a Direct Mapped Cache? Is there one? FIFO LRU LFU Random
Cache
Memory
Memory Hierarchy
Registers
Replacement Algorithms: FIFO cache entry that is oldest is replaced Assuming a 3-entry cache with FIFO as replacement algorithm:
Cache
Memory
Memory Hierarchy
Registers
Replacement Algorithms: LRU cache entry that is least recently used/accessed is replaced Assuming a 3-entry cache with LRU as replacement algorithm:
Cache
Memory
Memory Hierarchy
Registers
Replacement Algorithms: LFU cache entry that is least frequently used/accessed is replaced Assuming a 3-entry cache with LFU as replacement algorithm:
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Write policy
Deals with the question ”What happens when we modify data that’s in the cache?”
Cache
Memory
Memory Hierarchy
Registers
Elements of Cache Design: Write policy
Deals with the question ”What happens when we modify data that’s in the cache?” Write Through Write Back
Cache
Memory
Memory Hierarchy
Registers
Write policy: Write Through Everytime there’s a write or store, we write the data to the cache AND the main memory immediately PRO: simple to implement CON: write operations are very slow workaround: write buffers write buffer - takes care of the writing operation for the processor
Cache
Memory
Memory Hierarchy
Registers
Write policy: Write Back Everytime there’s a write or store 1 2
we write the data to the cache ONLY we set dirty bit tag of entry to 1
”update” to the main memory happens only once the entry needs to be replaced writeback for entry happens only when dirty bit tag is 1
Cache
Memory
Memory Hierarchy
Registers
Multicore processors and Caches
Which write policy do you think is better suited for multicore processors?
Cache