Memory Hierarchy Primer

Question 1

What’s the current general trend with data locations?

Answer 1

The quicker the location is, the less space is available because it is more expensive.

Question 2

List the memory hierarchy levels

Answer 2

Question 3

Describe how a memory hierarchy works, in general

Answer 3

• When you look for a variable in cache and you find it (cache hit), that means it is also stored in all subsequent levels of the memory hierarchy.
• If you try to access a variable in a level and cannot find it (denoted as a miss), you will have to keep navigating the subsequent levels until you find it.
• Then, you copy it to all the levels

Question 4

What’s temporal locality?

Answer 4

Cache recently accessed data. Idea: when you access a variable/data, you tend to access it again in the near future.

Question 5

What’s Spatial locality?

Answer 5

Idea: if a program accesses a variable, it is also likely to access data that is stored in a nearby memory location

Question 6

Why is it important to consider where data is stored?

Answer 6

The CPU cycles are very short which tends to lead to applications being I/O bound. This necessarily leads to the CPU being idle for a lot of the time since a large portion of time will be spent reading or writing data.

With faster computation speed being the primary goal of new computer designs and components such as the CPU and memory being expensive, there is a strong imperative to avoid I/O bound states and eliminating them can yield a more economic improvement in performance than upgrading the CPU or memory.