Parallel Computing Flashcards
What is parallel computing?
Parallel computing is the method of increasing performance in a computer by performing multiple computations in parallel.
What is the Flynn classification?
The Flynn classification is a computer classification developed by Flynn based on the number of concurrent instruction and data streams they have.
Single Instruction, Single Data (Flynn classification)
A single instruction stream, working on a single data stream. This means that one instruction stream works on one piece of data at a time.
This was the archaic method we used, which was superceded due to limits on pipelining and clock speeds.
Multiple Instruction, Single Data (Flynn classification)
Multiple instruction streams working on a single data stream.
This means that you can execute multiple instructions at once, but the same piece of data is available to all instruction streams. This classification is very rare as it is not very useful at all.
Single Instruction, Multiple Data (Flynn classification)
A single instruction stream working on multiple data streams.
This means that each clock, one instruction is applied to multiple pieces of data. This classification can be used for uses such as in matrix multiplication, as multiple matrix elements will need to be multiplied at once.
Another more relevant use is in graphics processing. Graphics cards (GPUs) use a SIMD classification.
Multiple Instruction, Multiple Data (Flynn classification)
Multiple instruction streams working on multiple data streams.
This means that each clock, multiple instructions are applied to multiple pieces of data. Or, multiple processes are carried out at once.
MIMD processors will have multiple cores, each with their own L1 cache. To communicate with eachother, they have a contextually large shared L3 cache. Some processors may also have an L2 cache for each core.
What is the difference between a multiprocessor and a multicomputer?
A multiprocessor is the result of putting two or more chips on a board. These processors communicate via shared memory variables, and when they want to use a variable, they lock it from the other.
A multicomputer, on the other hand, puts two or more multiprocessors on a rack, where each multiprocessor has its own copy of the operating system.
Due to this seperation, multicomputers need to communicate via wires, which is even more slow than how multiprocessors communicate. However, multicomputers have completely incomparable bandwidth.
What is a Graphics Processing Unit (GPU)?
A GPU is a type of microprocessor that utilises a large number of ALUs to perform parallel processing.
The GPU typically will have multiple ALUs, but only one program counter. This is because multiple instructions are not necessary, as graphics processing works via matrix manipulation.
GPUs are classified as SIMD (Single Instruction Multiple Data) computers.
Due to their lack of reliance on cache memory, however, GPUs also have very high latency. They bypass this using threads.
What is a thread and how does it work?
A thread is a unit of execution within a process. It is a separate flow of control within a program that can be scheduled by the operating system to run concurrently with other threads in the same process or in other processes.
Threads have their own stack and register state, but they share the heap and other data with the process in which they are running. This allows threads to communicate and share data with each other and with other parts of the program.
What is the difference between coarse and fine grain threading?
Coarse grain threading is a method employed primarily by CPUs. When a thread is blocked by a memory access request (another thread is using this , it will process switch to another thread.
