Basics of Parallel Computing Flashcards
What are 2 types of shared memory architecture?
Uniform Memory Access and Non-Uniform Memory Access
In Uniform Memory Access, all processors are connected to the same ________
memory
In Non-Uniform Memory Access, all processors have their own ________ but are connect by a local _________
memory, interconnect
In Non-Uniform Memory Access, why is it slow for P1 to access the memory of P0?
P1 has to go through the interconnect, rather than directly accessing P0’s memory
In a _______ memory architecture, memory updates by one processor are visible to others
shared
What are some drawbacks of a shared memory architecture?
- Difficult to scale
- Adding CPUs increases traffic (CPU to memory traffic)
- Need to synchronize memory access (using locks)
In a __________ memory architecture, P1 can access P0’s data though communication protocols
distributed
What are some advantages of a distributed memory architecture?
- Scalable in term of processors and memory
- Local access is fast (as a processor’s own memory is private)
- Cost effective
What are some drawbacks of a distributed memory architecture?
- Difficult to program
- Communication/Synchronization is difficult to manage
What are 2 communication models in memory architectures?
- Shared memory
- Message passing
In Shared memory, a common ______ ______ is accessed asynchronously, with _____ used to control access
address space, locks
In Message passing, there is no shared address space and data transfer requires ___________
coordination
In a data parallel programming model, parallel operations occur over a collection of ____ _____ and each task works on a different ________ of the collection
data items, subset
Describe the task parallel programming model.
Tasks are defined based on operations that can be safely executed concurrently with other operations. An example is pipelining
In parallel programming, you must ensure __________ are correctly satisfied.
dependencies
