Parallelism Design

Question 1

In what ways can you decompose a problem into parallel components?

Answer 1

Data parallelism
Task parallelism
Pipelines
Mixed solutions

Question 2

What is data parallelism?

Answer 2

Breaking a large data set into smaller subsets that can be processed concurrently on multiple processors.

Question 3

What is task parallelism

Answer 3

Breaking a large task into smaller sub-tasks that can be executed concurrently on multiple processors

Question 4

What is course grained parallelism?

Answer 4

• A program is split into large tasks to be executed in parallel.
• Good for data parallelism as all threads execute the same instructions. No load imbalance

Question 5

What is fine grained parallelism?

Answer 5

• A program is split into small tasks to be executed in parallel.
• Fine with data parallelism as well, no load imbalance.
• Good for task parallelism/pipelining (with planning), allows the program to be split into tasks to reduce load imbalance and increase throughput.
• I.e. if addressing takes two timesteps because for 1 timestep, we write the address, and for another we stamp the letter, why not do more?

Question 6

What is mixed solution in terms of granularity?

Answer 6

• Differing number of threads depending on how long a task takes to complete