Lecture 2

Question 1

What’s the point of parallel computing

Answer 1

Distributing the workload across multiple processors or cores, the overall computational power is increased.

Question 2

Parallelism vs concurrency

Answer 2

Parallel computing focuses on executing multiple tasks simultaneously on multiple processors or cores to improve performance and solve larger computational problems efficiently.

Concurrency deals with managing and coordinating multiple tasks or processes, often within a single processor, to enhance responsiveness and resource utilisation.

Question 3

Task vs Data parallelism

Answer 3

Task: Different processors or cores work on different tasks concurrently.

Data: Distributing the data across multiple processors or cores and performing the same operation on each subset of the data concurrently.

Question 4

Shared memory vs distributed memory

Answer 4

Shared memory: all processors have access to a common memory address space.
Distributed memory: all processors have their own private memory space.

Question 5

Concurrent vs parallel vs distributed computing

Answer 5

Concurrent - a program is one in which multiple tasks can be in progress at any instant.
Parallel - a program is one in which multiple tasks cooperate closely to solve a problem.
Distributed - a program may need to cooperate with other programs to solve a problem.

Question 6

What are the two main parts of a CPU?

Answer 6

Control unit - responsible for deciding which instruction in a program should be executed (the boss).
ALU - executing the logic (the worker).

Question 7

What is a process?

Answer 7

Instance of a program in execution

Question 8

What is a thread?

Answer 8

Contained within processes. Allows programmers to divide their programs into independent tasks.
A light-weight process.

Question 9

SPMD?

Answer 9

Single program multiple data
SPMD program consists of a single executable that can behave as if it were multiple different programs through the use of conditional branches.
If (im thread process i) do this; else do that;

Question 10

What’s MPI?

Answer 10

Message passing interface.
Standard communication for distributed memory systems.
Basically asynchronous (blocking and non-blocking).
Basic comms primitives do not inherently give reliable delivery but can be defined to cope with it (e.g. transfer-acknowledge vs request-transfer-acknowledge)

SPMD model (all tasks run the same program)