L16

Question 1

What are CPUs optimised for? What are GPUs optimised for?

Answer 1

CPUs optimized for
* Low latency
* Programmability

GPUs optimized for
* High throughput
* Little reuse

Question 2

What are some properties of a GPU chip?

Answer 2

Same silicon
More ALUs and registers
Less control logic and caches

Question 3

How do GPUs achieve simple control?

Answer 3

 No branch prediction/prefetching
 No out-of-order execution
 Groups of threads in lockstep (SIMT)

Question 4

How do GPUs achieve high throughput?

Answer 4

 Large number of execution units, registers, and threads
 Thread switching on long latency operations
 Fast + wide link to memory

Question 5

How can GPUs be programmed?

Answer 5

o As graphics cards (OpenGL)
o As General Purpose GPUs (CUDA, OpenMP)

Question 6

What is CUDA?

Answer 6

A general purpose GPUs
CUDA: heterogenous programming model
* Host (CPU) vs Device (GPU)
* Only for NVIDIA GPUs

Question 7

True or False.

A CUDA function cannot be both __host__ and __device___.

Answer 7

False. It can.

Question 8

What are the three function declaration prefixes in CUDA?

Answer 8

 __global__: executed on device, called from host
 __device__: executed on device, called from device
 __host__: executed on hose, called from host

Question 9

What are the 4 levels of the thread hierarchy in GPUs?

Answer 9

Thread:
 Executes the kernel once

Warp:
 Group of 32 threads executing in lockstep
 One assigned to each SP (stream processor) of the stream multiprocessor (SM)
 Zero-cost context switching

Block:
 Group of threads assigned to one SM
 Runs as a group to completion

Grid:
 All the blocks for the kernel organized in a Cartesian Grid

Question 10

Consider the following CUDA code:

MMKernel «<dim(width/32,width/32), dim3(32,32)»>(Md, Nd, Pd, width);

a. What does “dim(width/32,width/32)” specify?
b. What does “dim3(32,32)” specify?

Answer 10

a. Blocks per grid
b. Threads per block

Question 11

In GPUs, what happens if a block uses more resources than available in SM?

Answer 11

fail to launch

Question 12

True or False.

CUDA has more performance portability than OpenCL.

Answer 12

False.

OpenCL has better performance portability but is a bit more awkward to use.

Question 13

What is an alternative to CUDA?

Answer 13

OpenCL:
 Standard for programming accelerators
 Similar to CUDA’s program structure
 Similar performance considerations
 Can run in multiple different platforms using multiple different compiler

SYCL:
 High level
 Single source
 Uses buffers for memory

Question 14

Is there a way to get portable performance from GPUs to CPUs?

Answer 14

Tuned libraries, such as MKL, can be used, but code might still be suboptimal.

Question 15

How is high throughput achieved in GPUs?

Answer 15

Many ALUs and registers

Question 16

What is the role of host and device in GPU programming?

Answer 16

Host: orchestrates execution by describing how the problem maps to threads

Device: does the computation in restricted C/C++