CHAPTER 7: Parallel Processors Flashcards
multiprocessor
computer system with at least two processors. This computer is in contrast to a uniprocessor, which has one, and is increasingly hard to find today
task-level parallelism (process-level parallelism)
utilizing multiple processors by running independent programs simultaneously
parallel processing program
single program that runs on multiple processors simultaneously
cluster
set of computers connected over a local area network that function as a single large multiprocessor
multicore microprocessor
microprocessor containing multiple processors (“cores”) in a single integrated circuit. Virtually all microprocessors today in desktops and servers are multicore
shared memory multiprocessor
parallel processor with a single physical address space
strong scaling
speed-up achieved on a multiprocessor without increasing the size of the problem
weak scaling
speed-up achieved on a multiprocessor while increasing the size of the problem proportionally to the increase in the number of processors
SISD or single instruction stream, single data stream
uniprocessor
MIMD or multiple instruction streams, multiple data streams
multiprocessor
SPMD or single program, multiple data streams
conventional MIMD programming model, where a single program runs across all processors
SIMD or single instruction stream, multiple data streams
same instruction is applied to many data streams, as in a vector processor
data-level parallelism
parallelism achieved by performing the same operation on independent data
vector lane
one or more vector functional units and a portion of the vector register file. Inspired by lanes on highways that increase traffic speed, multiple lanes execute vector operations simultaneously
hardware multithreading
increasing utilization of a processor by switching to another thread when one thread is stalled
fine-grained multithreading
version of hardware multithreading that implies switching between threads after every instruction
coarse-grained multithreading
version of hardware multithreading that implies switching between threads only after significant events, such as a last-level cache miss
simultaneous multithreading (SMT)
version of multithreading that lowers the cost of multithreading by utilizing the resources needed for multiple issue, dynamically scheduled microarchitecture
reduction
function that processes a data structure and returns a single value
OpenMP
API for shared memory multiprocessing in C, C++, or Fortran that runs on UNIX and Microsoft platforms. It includes compiler directives, a library, and runtime directives
network bandwidth
peak transfer rate of a network; can refer to the speed of a single link or the collective transfer rate of all links in the network
bisection bandwidth
bandwidth between two equal parts of a multiprocessor, this measure is for a worst case split of the multiprocessor
fully connected network
network that connects processor-memory nodes by supplying a dedicated communication link between every node
multistage network
network that supplies a small switch at each node
