Performance

Question 1

What is Parallelism?

Answer 1

Two or more processes
that execute simultaneously and
independently

Question 2

What is Concurrency?

Answer 2

Two or more processes that execute simultaneously and share at least one resource

Question 3

What is a Processor?

Answer 3

The electronic circuitry that executes instructions comprising a program. There can be many processors in a computer system e.g. graphics processor, video processor. When left unqualified, usually refers to CPU.

Question 4

What is a CPU?

Answer 4

This is (one of) the main general-purpose processor(s) in a computer system, not a specific purpose e.g. video decompression

Question 5

What is a Die?

Answer 5

Refers to the silicon wafer containing the processor (usually the CPU when no other context is given) along with other components required for interfacing e.g. memory controller.

Question 6

What is a Socket?

Answer 6

Refers to the component containing the processor die, and includes the physical connectors to plug in to the motherboard

Question 7

What is a Core?

Answer 7

(HARDWARE) A SMALL CPU OR PROCESSOR BUILT INTO A BIG CPU OR CPU SOCKET. IT CAN INDEPENDENTLY PERFORM OR PROCESS ALL COMPUTATIONAL TASKS

Question 8

What is a Thread?

Answer 8

(SOFTWARE) A SINGLE SEQUENTIAL FLOW OF CONTROL IN A PROGRAM.
IT IS THE SMALLEST UNIT THAT CAN BE MANAGED BY AN OS SCHEDULER.
EACH THREAD HAS ITS OWN PROGRAM COUNTER, REGISTERS AND STACK.

Question 9

What is a Node?

Answer 9

• Normally thought of as one “computer”
• A single motherboard
• May have:
• More than one CPU
• Each CPU will have many cores
• One or more accelerators (GPU etc.)
We use OpenMP to program within the node

Question 10

What is a Cluster/Supercomputer?

Answer 10

A collection of 100s/1000s of nodes connected through a high-speed interconnect (this is what makes it different to a server).

Question 11

What is a single precision floating-point?

Answer 11

Formally called the IEEE 754 single-precision binary floating-point format: binary32. This is a format for representing floating-point numbers in computers using a total of 32 bits.
Corresponds to the C datatype float. Also called float32/FP32.

Question 12

What is a double precision floating-point?

Answer 12

Also part of the same IEEE 754 specification as double-precision binary
floating-point format: binary64.
This represents floating-point numbers using 64 bits. Corresponds to the C datatype double. Also called float64/FP64.

Question 13

What is a Flop?

Answer 13

Abbreviation for floating-point operation.
Usually means either an addition or multiplication of two floating-point numbers but other operations could be included.
A common unit of measurement of processor speed is flops/s.
Usually has to be qualified with either single or double precision to be specific.

Question 14

What is the RPeak for a single node?

Answer 14

𝑅𝑝𝑒𝑎𝑘 = 2 ∗ 𝑊𝑣𝑒𝑐 ∗ 𝑟𝑐𝑙𝑜𝑐𝑘 ∗ 𝑛𝑐𝑜𝑟𝑒 ∗ 𝑛𝑠𝑜𝑐𝑘𝑒𝑡

Where

𝑤𝑣𝑒𝑐: Vector width
𝑟𝑐𝑙𝑜𝑐𝑘: Clock speed
𝑛𝑠𝑜𝑐𝑘𝑒𝑡 : Sockets per node
𝑛 𝑐𝑜𝑟𝑒: Cores per socket

Question 15

How to calculate observed runtime?

Answer 15

𝑟𝑜 = 𝑟𝑡 + 𝜖

Where

• 𝑟𝑜 – Observed runtime (The physical time it took to run)
• 𝑟𝑡 – True runtime (The time actually running)
• 𝜖 – Noise (Hinderances to your program running)

Question 16

Difference between observed and true runtime?

Answer 16

Your observed runtime will never be lower than the true runtime.
Therefore, reporting the minimum observed runtime will be closest to the true runtime.

Question 17

What is Arithmetic Intensity?

Answer 17

I(n) =W(n) / Q(n)

Where 𝑊 is the number of flops carried out by the program, and
𝑄 is the number of bytes transferred from memory to cache.
• Flops/byte
• Programs with low AI are called memory-bound programs
• Programs with high AI are called compute-bound programs
• For memory-bound programs, the processor spends more time waiting for the operands to be delivered from the memory.

Question 18

Hardware size in order (smallest to largest)

Answer 18

Core < Die < Socket < Node < Cluster