Shared memory Flashcards
two concepts for memory models
- coherency: access to one signle mem location
- consistency: access to several mem location
sequential consistency
- the result of any execution is the same as if sequential
2. order of operations of each indicivudual processor is same as in program
why doesnt sequential cosnistency work in parallel programs
-use of write buffers
- compiler optimizations
- everyt write has to be propagated
we need relaxed consistency to get good eprformance
all types of consistency
processor
weak
release
sequential
processor consistency
writes by a thread are seen by all threads in the order they were issued
but different threads may see different order
enforced locally per thread
weak consistency
splits into sync and data operations
sync flushes memory pipeline
release consistency
splits sync operations in acquire and release
like a lock
before accessing a variable all acquire operations have to be done
before completing release all read/write must be done
whats openmp’s memory model?
kinda like weak
flush directive
pragma omp flush(list)
syncs data list of thread with main memory
performance issues of shmem
- thread overhead
- too little work per thread (load imbalance)
- sync overhead: less locks, global locks
- cache behaviour and locality: numa,bndwth, cache,
- thread and data locality
false sharing
2 threads access separate data on same cache line, a write on one data invalidates the cache line that then needs to be updated in the other threads, leading to resource waste
use padding in personal cache line
data locality
row wise
thread data locality
numa
first touch: first thread that touches a dat point will allocate it in closest memory
first access will allocate physical memory
UMA procon
PRO
- predictable
CON
- central memory becomes bottle neck
- high badnwth
NUMA procon
PRO
- more bwtch
- less bottlenecks
CON
- CC protocol has large overhead
- each memory access potentially invalidates data in cache
