ECM 1413 Multiprocessors

Question 1

Improve performance by performing multiple computations in parallel. The 5 outlets for which this can be achieved

Answer 1

ILP - Pipelining
Threads
Multi-core processors
Multiprocessors
Multicomputers

Question 2

clock speeds cannot be increased indefinitely because

Answer 2

Due to physical and thermal limitations

Question 3

Flynn’s Taxonomy

Answer 3

A classification of computer architectures based on number of instruction and data streams they can handle simultaneously

Question 4

Flynn’s Taxonomy instruction stream(s)

Answer 4

• An instruction stream - sequence of instructions; arithmetic, data transfer, and branching
  ○ PC - next instruction

Question 5

Flynn’s Taxonomy data stream

Answer 5

• A data stream consists of operands (constants, variables, and memory addresses)
  ○ Registers

Question 6

The four possible combinations of data and instruction streams

Answer 6

SISD
SIMD
MISD
MIMD

Question 7

Single Instruction Single Data (SISD)

Answer 7

Sequential computer with no data or instruction parallelism
Older single-core CPUs & simple microcontrollers

Question 8

Multiple Instruction, Single Data (MISD)

Answer 8

• Multiple instructions operate on one piece of data simultaneously
• Used for fault tolerance
• Rare in practice
• Example: Space shuttle flight control systems

Question 9

Example of MISD

Answer 9

Example - Triple Modular Redundancy
* A majority vote of three independent systems or processors
* Space satellite systems often use TMR

Question 10

Single Instruction, Multiple Data (SIMD)

Answer 10

Execute the same instruction on multiple data points simultaneously

SIMD computers are good for vector operations, including image and audio processing.
SIMD computers are not suitable when the type of instruction depends on the data.

Question 11

SIMD example

Answer 11

Graphics Processing Units
* A GPU contain many ALUs to perform mathematical operations in parallel.
* A GPU is based on SIMD architecture.

• In comparison to CPUs:
  
  • GPUs have more ALUs and rely less on cache.
  • GPUs have a high throughput and higher latency.

Question 12

GPUs and Threads

Answer 12

• GPUs hide latency by utilising threads
• A process may be subdivided into multiple threads
• Threads within a process share data, heap, and code, but have their own registers and stack

Question 13

Multiple Instruction, Multiple Data (MIMD)

Answer 13

Multiple processors simultaneously execute different instructions on different pieces of data.

A multi-core processor puts many CPU cores on one chip

Question 14

MIMD Example

Answer 14

Example - EPYC 7763 Processor
The AMD EPYC 7763 processor has:
* 64 cores/128 threads
* 64 × 32 KB 8-way set associative L1 instruction cache
* 64 × 32 KB 8-way set associative L1 data cache
* 64 × 512 KB 8-way set associative L2 cache
* 256 MB of L3 cache for each group of four cores

Question 15

Multiprocessors

Answer 15

• Two or more CPUs within a single computer system
• CPUs communicate through shared memory variables.
• To modify a shared variable, the CPU locks the variable so that no other processor can modify it.

Shared memory

Question 16

Multiprocessor example

Answer 16

Dell PowerEdge R7525 Rack Server:
* 2 × AMD EPYC processors
* 2 TB memory
* 4 TB SSD

Question 17

Multi-Computers

Answer 17

• Different computers connected together
• A standard rackmount has can house multiple Dell PowerEdge R7525 Rack Servers.

Distributed memory

Question 18

Shared vs Distributed Memory

Answer 18

Shared (Multiprocessors)
- Processors communicate through the shared memory
- Faster communication

Distributed (Multicomputers)
- Processors communicate by passing messages
- Allows for higher bandwidth