Caching

Question 1

Caching

Answer 1

a technique that stores copies of frequently used application data in a layer of smaller, faster memory in order to improve data retrieval times, throughput, and compute costs. Caching can exist at any level of a system, from a single CPU to a distributed cluster. And the same fundamental design principles apply, regardless of where the cache is located.

Question 2

Purpose of caching

Answer 2

Caching is based on the principle of locality, which means that programs repeatedly access data located close to each other. There are two kinds of locality:

Temporal locality, where data that has been referenced recently is likely to be referenced again (i.e. time-based locality).
Spatial locality, where data that is stored near recently referenced data is also likely to be referenced (i.e. space-based locality).
We can speed up a system by making it faster to retrieve this locally relevant data. Caching does this by storing a “cached” copy of the data, in a smaller, faster data store. For example a CPU has a cache in specialized SRAM memory on the processor so it doesn’t have to go all the way to RAM or Disk.

Question 3

When to use caching

Answer 3

• Caching is most helpful when the data you need is slow to access, possibly because of slow hardware, having to go over the network, or complex computation.
• Caching is helpful in systems where there are many requests to static or slow to change data, because repeated requests to the cache will be up to date.
• Caching can also reduce load on primary data stores, which has the downstream effect of reducing service costs to reach performant response times.
• Ultimately the features of a successful caching layer are highly situation-dependent. Creating and optimizing a cache requires tracking latency and throughput metrics, and tuning parameters to the particular data access patterns of the system.

Question 4

When to not use caching

Answer 4

-Caching isn’t helpful when it takes just as long to access the cache as it does to access the primary data.
-Caching doesn’t work as well when requests have low repetition (higher randomness), because caching performance comes from repeated memory access patterns.
-Caching isn’t helpful when the data changes frequently, as the cached version gets out of sync and the primary data store must be accessed every time.
It’s important to note that caches should not be used as permanent data storage - they are almost always implemented in volatile memory because it is faster, and should thus be considered transient.