Big Data Lecture 10 Performance at Large Scales Flashcards
With 1000s of nodes in a system, what will be the performance distribution over them?
Most nodes will take around the mean time, but some nodes will take extremely long: phenomenon of <b>tail latency</b>.
CPU, Memory, Disk, Bandwith: which one is usually the bottleneck?
That depends, but it is only one of them! Almost never two or more!
When should we be using systems such as Spark or MapReduce?
When system I/O is the limit of the system! This is because on one system disk is limiting us and we need parallel writes and reads!
What are two different definitions of latency?
Some refer to latency as the time when data starts arriving, and some to the time when all of the data arrives (= + delivery time).
Prefix 0.001 in words?
Milli (m).
Prefix 0.000 001 in words?
Micro (mu).
Prefix 0.000 000 001 in words?
Nano (n).
Prefix 0.000 000 000 001 in words?
Pico (p).
What is latency of one instruction execution on CPU?
1 ns
What is the latency of fetch from main memory?
100 ns
What is the latency of fetching from new disk location?
8 ms
What is the latency of reading internet packet to US and back?
150 ms
What is the throughput of standard Fast Ethernet?
100 Mbit/s
What is the throughput of write/read to SSD?
I/O 240/270 MB/s.
What is the definition of total response time?
NAME?
What is speedup?
Old latency / new latency.
What is Amdahl’s law for speed up?
For constant problem size<br></br>1/(1 - p +p/s)<br></br>where <i>p </i>is the fraction of how much we can parallelize and <i>s</i> speedup on parallelizable part.
What is Gustafson’s law for speedup?
For constant computing power<br></br>1 - p + sp<br></br>where <i>p </i>is the fraction of how much we can parallelize and <i>s</i> speedup on parallelizable part.
How to avoid memory scale up?
Optimize classes that are instantiated billions of times, remove redundancy and inheritance, unused fields and use efficient data structures.
How to avoid Disk I/O scale up?
Use efficient formats and compression!
How to avoid I/O scale up?
Push down computation closest to the data.<br></br><br></br>Batch process (send larger packets when possible).
What is a good rule of thumb to how many nodes we should have in a system? Why?
Usually around the square root of the number of available cores and memory capacity.<br></br><br></br>We want liquidity of tasks in slots, but at the same time not to overload the network in with extreme amount of communication.
Describe the phenomenon of tail latency.
With rising number of tasks the probability of some task taking a longer than expected time goes to 1.
What is the reason for tail latency phenomena?
Queues, power limits (hyperthreading), garbage collection and energy management.
What formula describes the probability of some node taking more super long?
Probability of taking long for one node is <i>p</i>.<br></br><br></br>Probability of not taking long is <i>1 - p.</i><br></br><br></br>Hence probability of all not taking long is <i>(1 - p)^n</i>.<br></br><br></br>Hence probability of all taking long is <i>1 - (1 - p)^n</i>.<br></br><br></br>With <i>p < 1</i> this will go to 1 with <i>n</i> going to infinity.
How to solve tail latency?
<ul><li>Hedge request: duplicate calls, terminate when one finishes (duplicates load tho),</li><li>Deffered hedge requests: if a task take more than 95% of task usually take, restart it (increases load in practice by 2% and improves tail 10 times).</li></ul>
How to avoid CPU scale up?
Remove gigantic loops, do not override, do not use classes, do not cast and use exceptions.
