Big Data Lecture 10 Performance at Large Scales

Question 1

With 1000s of nodes in a system, what will be the performance distribution over them?

Answer 1

Most nodes will take around the mean time, but some nodes will take extremely long: phenomenon of <b>tail latency</b>.

Question 2

CPU, Memory, Disk, Bandwith: which one is usually the bottleneck?

Answer 2

That depends, but it is only one of them! Almost never two or more!

Question 3

When should we be using systems such as Spark or MapReduce?

Answer 3

When system I/O is the limit of the system! This is because on one system disk is limiting us and we need parallel writes and reads!

Question 4

What are two different definitions of latency?

Answer 4

Some refer to latency as the time when data starts arriving, and some to the time when all of the data arrives (= + delivery time).

Question 5

Prefix 0.001 in words?

Answer 5

Milli (m).

Question 6

Prefix 0.000 001 in words?

Answer 6

Micro (mu).

Question 7

Prefix 0.000 000 001 in words?

Answer 7

Nano (n).

Question 8

Prefix 0.000 000 000 001 in words?

Answer 8

Pico (p).

Question 9

What is latency of one instruction execution on CPU?

Answer 9

1 ns

Question 10

What is the latency of fetch from main memory?

Answer 10

100 ns

Question 11

What is the latency of fetching from new disk location?

Answer 11

8 ms

Question 12

What is the latency of reading internet packet to US and back?

Answer 12

150 ms

Question 13

What is the throughput of standard Fast Ethernet?

Answer 13

100 Mbit/s

Question 14

What is the throughput of write/read to SSD?

Answer 14

I/O 240/270 MB/s.

Question 15

What is the definition of total response time?

Answer 15

NAME?

Question 16

What is speedup?

Answer 16

Old latency / new latency.

Question 17

What is Amdahl’s law for speed up?

Answer 17

For constant problem size<br></br>1/(1 - p +p/s)<br></br>where <i>p </i>is the fraction of how much we can parallelize and <i>s</i> speedup on parallelizable part.

Question 18

What is Gustafson’s law for speedup?

Answer 18

For constant computing power<br></br>1 - p + sp<br></br>where <i>p </i>is the fraction of how much we can parallelize and <i>s</i> speedup on parallelizable part.

Question 19

How to avoid memory scale up?

Answer 19

Optimize classes that are instantiated billions of times, remove redundancy and inheritance, unused fields and use efficient data structures.

Question 20

How to avoid Disk I/O scale up?

Answer 20

Use efficient formats and compression!

Question 21

How to avoid I/O scale up?

Answer 21

Push down computation closest to the data.<br></br><br></br>Batch process (send larger packets when possible).

Question 22

What is a good rule of thumb to how many nodes we should have in a system? Why?

Answer 22

Usually around the square root of the number of available cores and memory capacity.<br></br><br></br>We want liquidity of tasks in slots, but at the same time not to overload the network in with extreme amount of communication.

Question 23

Describe the phenomenon of tail latency.

Answer 23

With rising number of tasks the probability of some task taking a longer than expected time goes to 1.

Question 24

What is the reason for tail latency phenomena?

Answer 24

Queues, power limits (hyperthreading), garbage collection and energy management.

Question 25

What formula describes the probability of some node taking more super long?

Answer 25

Probability of taking long for one node is <i>p</i>.<br></br><br></br>Probability of not taking long is <i>1 - p.</i><br></br><br></br>Hence probability of all not taking long is <i>(1 - p)^n</i>.<br></br><br></br>Hence probability of all taking long is <i>1 - (1 - p)^n</i>.<br></br><br></br>With <i>p < 1</i> this will go to 1 with <i>n</i> going to infinity.

Question 26

How to solve tail latency?

Answer 26

<ul><li>Hedge request: duplicate calls, terminate when one finishes (duplicates load tho),</li><li>Deffered hedge requests: if a task take more than 95% of task usually take, restart it (increases load in practice by 2% and improves tail 10 times).</li></ul>

Question 27

How to avoid CPU scale up?

Answer 27

Remove gigantic loops, do not override, do not use classes, do not cast and use exceptions.