Introduction Flashcards
Moore’s Law?
the number of transistors on a microchip doubles approximately every two years
What speaks against Moore’s law?
physical limits of transistor scaling, increasing power consumption and thermal issues
What is CCX
core complex = cluster of cores sharing L3 cache
What is CCD
core complex die = chiplet of 1 or multiple CCXs
What are the layers of a runtime system? (top to bottom)
- Parallel Application Code
- Parallel Runtime Library
- Treading Library
- OS
- Multi-core Processors
Amdahl’s Law?
Upper bound on the speed-up of a parallel program when parallelizing parts of it = 1/s
Issue with Amdahl’s Law?
Overly-optimistic worst case: Assumption that the limit n→∞ is meaningful hides the fact that for practical values of n, the speedup is much lower than expected
Programming Models based on Flynn’s Taxonomy
SISD, MISD, SIMD, MIMD
What is NUMA?
NUMA (Non-Uniform Memory Access) is a memory architecture used in multi-processor where memory is divided into multiple regions (nodes)
NUMA speed
Faster for local memory (current NUMA node), slow for remote memory (different NUMA node)
Name 3 Parallel Programming Languages
OpenMP, OpenACC, Co-Array Fortran
Name 3 Parallel Programming Libraries
MPI, POSIX threads, TBB (language agnostic)
What does this do:
#pragma omp parallel for reduction(operator:variable)
Ensures that each thread computes a local result, which is then combined into a single global result at the end.
What is a barrier?
a synchronization construct that ensures that all threads reach a specific point before any of them can proceed.
For which pragmas are there implicit OpenMP barriers?
parallel, for, section, single
What are task scheduling points?
specific points during program execution where the OpenMP runtime system decides whether to switch execution to another task
What means untied?
untied task can be resumed by a different thread as well
What does nowait do?
Eliminates the implicit barrier at the end of certain constructs (for, sections, single)
