Metrics and Evaluation Flashcards
What is latency?
The time to go from start to end
What is throughput?
The number of things done per second
What are benchmarks?
Programs and input data that everyone has agreed upon for performance measurements
What are the types of benchmarks?
Real apps, kernels, synthetic, and peak performance
What are the pros/cons of using real application benchmarks?
More representative, but harder to set up. Best for testing real machines
What is a kernel benchmark?
Find the most time-consuming part of an application and test that. Best for prototype testing.
What is a synthetic benchmark?
Similar to a kernel benchmark, but easier to compile. Best for design studies (when we are in an early stage of development)
What is a peak performance benchmark?
What is the best performance this machine can get? Usually only good for marketing, not a real indicator of performance. Does not use real code
How to compare the performance of two processors?
Compare average execution times - do not compute speedups and average them!
What is the Iron Law of Performance?
CPU Time=(# of instructions) ∗(cycles per instruction)∗(clock cycle time)
What is Amdahl’s Law?
Measure the speedup when an improvement is applied only to some part of a program
What is Lhadma’s Law?
Don’t make the uncommon case too slow
