CHAPTER 7: Parallel Processors

Question 1

multiprocessor

Answer 1

computer system with at least two processors. This computer is in contrast to a uniprocessor, which has one, and is increasingly hard to find today

Question 2

task-level parallelism (process-level parallelism)

Answer 2

utilizing multiple processors by running independent programs simultaneously

Question 3

parallel processing program

Answer 3

single program that runs on multiple processors simultaneously

Question 4

cluster

Answer 4

set of computers connected over a local area network that function as a single large multiprocessor

Question 5

multicore microprocessor

Answer 5

microprocessor containing multiple processors (“cores”) in a single integrated circuit. Virtually all microprocessors today in desktops and servers are multicore

Question 6

shared memory multiprocessor

Answer 6

parallel processor with a single physical address space

Question 7

strong scaling

Answer 7

speed-up achieved on a multiprocessor without increasing the size of the problem

Question 8

weak scaling

Answer 8

speed-up achieved on a multiprocessor while increasing the size of the problem proportionally to the increase in the number of processors

Question 9

SISD or single instruction stream, single data stream

Answer 9

uniprocessor

Question 10

MIMD or multiple instruction streams, multiple data streams

Answer 10

multiprocessor

Question 11

SPMD or single program, multiple data streams

Answer 11

conventional MIMD programming model, where a single program runs across all processors

Question 12

SIMD or single instruction stream, multiple data streams

Answer 12

same instruction is applied to many data streams, as in a vector processor

Question 13

data-level parallelism

Answer 13

parallelism achieved by performing the same operation on independent data

Question 14

vector lane

Answer 14

one or more vector functional units and a portion of the vector register file. Inspired by lanes on highways that increase traffic speed, multiple lanes execute vector operations simultaneously

Question 15

hardware multithreading

Answer 15

increasing utilization of a processor by switching to another thread when one thread is stalled

Question 16

fine-grained multithreading

Answer 16

version of hardware multithreading that implies switching between threads after every instruction

Question 17

coarse-grained multithreading

Answer 17

version of hardware multithreading that implies switching between threads only after significant events, such as a last-level cache miss

Question 18

simultaneous multithreading (SMT)

Answer 18

version of multithreading that lowers the cost of multithreading by utilizing the resources needed for multiple issue, dynamically scheduled microarchitecture

Question 19

reduction

Answer 19

function that processes a data structure and returns a single value

Question 20

OpenMP

Answer 20

API for shared memory multiprocessing in C, C++, or Fortran that runs on UNIX and Microsoft platforms. It includes compiler directives, a library, and runtime directives

Question 21

network bandwidth

Answer 21

peak transfer rate of a network; can refer to the speed of a single link or the collective transfer rate of all links in the network

Question 22

bisection bandwidth

Answer 22

bandwidth between two equal parts of a multiprocessor, this measure is for a worst case split of the multiprocessor

Question 23

fully connected network

Answer 23

network that connects processor-memory nodes by supplying a dedicated communication link between every node

Question 24

multistage network

Answer 24

network that supplies a small switch at each node

Question 25

crossbar network

Answer 25

network that allows any node to communicate with any other node in one pass through the network

Question 26

polling

Answer 26

process of periodically checking the status of an I/O device to determine the need to service the device

Question 27

PThreads

Answer 27

a UNIX API for creating and manipulating threads. It is structured as a library

Question 28

arithmetic intensity

Answer 28

ratio of floating-point operations in a program to the number of data bytes accessed by a program from main memory

Question 29

fallacy: peak performance tracks observed performance

Answer 29

Amdahl’s Law suggests how difficult it is to reach either peak; multiplying the two together multiplies the sins. The roofline model helps put peak performance in perspective.

Question 30

fallacy: peak performance tracks observed performance

Answer 30

Amdahl’s Law suggests how difficult it is to reach either peak; multiplying the two together multiplies the sins. The roofline model helps put peak performance in perspective.

Question 31

pitfall: not developing the software to take advantage of, or optimize for, a multiprocessor architecture

Answer 31

Placing locks on smaller portions of the page table effectively eliminated the problem