Performance

Question 1

what is latency?

Answer 1

times it takes to send and receive a request (amount of time it takes for the customer to order)

Question 2

what is throughput?

Answer 2

amount of data that can be successfully processed in a certain time (amount of time it takes to server a specific number of customers in a certain amount of time)

Question 3

given different stages in a server what is the latency of a server?

Answer 3

sum of all stages

Question 4

given different stages in a server what is the throughput of a server?

Answer 4

min of throughput of stages

Question 5

what is utilization?

Answer 5

percentage of capacity that is being used

Question 6

what is overhead w/ examples?

Answer 6

the amount of resources that is “wasted” (usually is the use at lower-layers ex: allocation virtualization, operating systems)

Question 7

what is useful work?

Answer 7

resources spent on actual work
(usually the use at the current layer in a system ie the application)

Question 8

what is the formula for overhead as percentage?

Answer 8

overhead = capacity - useful work
percentage = overhead / useful work

Question 9

bottleneck

Answer 9

the component that restricts throughput

Question 10

if you’re optimizing for throughput what should you focus on?

Answer 10

bottle neck

Question 11

if you’re optimizing for latency what should you focus on?

Answer 11

the component with the highest latency

Question 12

what is the number one rule for optimization and why?

Answer 12

dont prematurely optimize because we are bad at predicting performance

Question 13

what is amdah’s law?

Answer 13

1/((1-p) + (p/s))

Question 14

what is the best case speedup?

Answer 14

lim s->inf = 1/(1- p)

Question 15

what are the three solutions for latency?

Answer 15

fast-path, parallelize, speculation

Question 16

what is fast-path?

Answer 16

reduce latency for some requests

Question 17

how do we parallelize (concurrency) to help with latency?

Answer 17

run independent steps in parallel (if one step does a totally different step then another step than you can run them both at the same time)

Question 18

what is speculation and what does it cost?

Answer 18

predict what work might be done, if the prediction is correct latency goes down
- trading off work for latency, if the system is working near capacity you might be trading off throughput for latency

Question 19

what are the three solution for throughput?

Answer 19

batching, dallying, concurrency or pipelining

Question 20

what is batching?

Answer 20

grouping multiple tasks or inputs together and processing them as a single batch instead of one by one

Question 21

what is dallying?

Answer 21

delaying a request, removing unnecessary waiting

Question 22

what is pipelining?

Answer 22

give each stage its own threads/memory
connect stages using bounded buffers

Question 23

what is the working set?

Answer 23

its the last k number of distinct items that were accessed
( subset of data that is currently relevant to the program’s execution)

Question 24

what is the working set hypothesis?

Answer 24

for some number k the actual working set is much smaller than k

Question 25

what is temporal locality?

Answer 25

recently accessed items will be accessed again

Question 26

what is spatial locality?

Answer 26

items closed to recently accessed items will be accessed soon

Question 27

what are the 3 different associativity’s in caches and rank them from most to least expensive?

Answer 27

— most expensive —
1. fully associative : each item can go in any slot
2. n way associative : each item can go in n locations
3. direct mapped : each item can go in one slot
— least expensive —

Question 28

what are the two mechanisms for removing an item form the cache?

Answer 28

1. write-back cache
2. write through cache

Question 29

what is write back cache?

Answer 29

updates to the cache are not immediately reflected in the memory

Question 30

what is write through cache?

Answer 30

updates to the cache are immediately reflected in the memory

Question 31

what are the three policies for removing items from the cache?

Answer 31

FIFO, LRU, Clock

Question 32

explain FIFO

Answer 32

track count of actions and when each item was added

Question 33

explain LRU

Answer 33

• the least recently used item is removed
• tracks the age of each item
• age is rest to zero when accessed
• evicts the item with the highest age

Question 34

explain the clock algorithm

Answer 34

is like a clock with a hand ticking around. Each time an item is accessed, it’s as if the clock hand is ticking past that item. When the cache is full and a new item needs to be added, the clock hand “checks” each item. If an item has been recently accessed (referenced), it is spared. If it hasn’t been accessed since the last check, it is evicted, making room for the new item.

Question 35

what does belady’s anomaly state?

Answer 35

a larger cache does not always result in better performance

Question 36

what is a compulsory miss?

Answer 36

program has never accessed the data (how many unique items have been introduced?)

Question 37

what is capacity miss?

Answer 37

the program is actively using more data than a cache can hold (how many misses would a fully associative cache of my size have)

Question 38

conflict misses

Answer 38

a program has seen the data but it was evicted by something in that data’s set