Parallel Architectures

Question 1

What are three classifications of parallel computing?

Answer 1

Low-level, single instruction multiple data (SIMD) and multiple instructions multiple data

Question 2

Define low-level parallelism

Answer 2

It has an abacus serial adding algorithm and parallels adding algorithm, it’s essentially a calculation

Question 3

Define single instruction multiple data

Answer 3

Represents multiple data items at once and operate on these arrays with single instructions such as ‘‘VECTOR-ADD’’

Question 4

What did complex CISC SIMD change compared to SIMD?

Answer 4

Added specialised word length vector registers in CPU which loads them up with data

Question 5

How does SIMD operate inside the GPU?

Answer 5

DMA transfers to main RAM, smaller caches, small pipelines of two stages and thousands of ALUs, ultimately it’s optimised for data-parallel and high throughput computations

Question 6

How does SIMD operate inside the CPU?

Answer 6

Optimised for low-latency, large caches which gives quick access to data, complex pipelines of total 30 stages but fewer ALUs

Question 7

What’s a work-group?

Answer 7

Group of Pes within a CU all executing the same instruction, single instruction

Question 8

What’s a work item?

Answer 8

Processing element executing it on one datum, multiple data

Question 9

What’s a kernel?

Answer 9

A program of SIMD instructions given to a work-group

Question 10

How does the GPU implement itself?

Answer 10

Through OpenCL C, kernel code is compiled into GPU’s own assembly language and machine code

Question 11

What exactly is MIMD?

Answer 11

Related to vectors idea of architecture a.k.a. ‘‘Very Long Instructions Words’’ = VLIW

Question 12

Do all CPU’s have the same address space?

Answer 12

Yes

Question 13

What’s a cache coherency?

Answer 13

Some levels are shared, others not, tricky to get cache write right e.g. L1 cache needs to notify each other on changes and refreshes

Question 14

What does NUMA (non-uniform memory access) do?

Answer 14

1. Single address space shared by processors
2. But access time differ
3. Used in supercomputers

Question 15

What’s a ‘blue gene’ supercomputer?

Answer 15

It’s based on custom chips, it’s task is to protein folding problems and it has a very hierarchical architecture

Question 16

What’s a Message Passing Interface?

Answer 16

A standard library for processes to send and receive discrete messages to one another

Question 17

How does MPI (message passing interface) pass like?

Answer 17

Through bus, mesh, shared memory communication

Question 18

What’s a “Beowolf” cluster architecture?

Answer 18

It’s a group of commodity PCs linked through regular network and MPI, one master node, other nodes slaves, run one program and master calls slaves to do parallel work

Question 19

What’s grid computing?

Answer 19

1. Hundred or more maintained identical PCs
2. No shared memory
3. Ethernet connection
4. Shared network discs

Question 20

What’s cloud computing?

Answer 20

Like grids but the much weaker organisation

Question 21

What’s map reducing?

Answer 21

Many cloud computing projects all evolved to use similar structures, splitting tasks into chunks, usually, each running the same code on different data

Question 22

What does map reducing work?

Answer 22

Map each chunk to worker computer, send back results, reduce these results to a single result

Question 23

What does ‘SpiNNaker’ do?

Answer 23

The task to simulate biological neurons at brain-like scales

Question 24

Does instruction-less parallelism need a CPU?

Answer 24

Nope, it doesn’t