Lecture 3: Parallel Programming Using Shared Memory

Question 1

What happens with virtual memory?

Answer 1

With virtual memory, the process gets the illusion it has access to 100% of the memory

Question 2

True or false

Two programs run in 2 different processes, i.e. 2 different address space

Answer 2

True

Each process has its own address space so there is no interference

Question 3

What is a thread?

Answer 3

A thread is a sequence of instructions executed on a CPU core. A process can consist of one or more threads

Question 4

True or false

Threads in the same process have different address spaces

Answer 4

False

All threads in the same process share the same address space

Question 5

How do threads communicate?

Answer 5

Threads communicate using shared memory

Question 6

Match the c/c++ thread function

1. Use pthread_create()
2. pthread_exit()
3. pthread_join()

A. to create and launch a thread
B. to wait for another thread to finish
C. to have the calling thread exit

Answer 6

1. A
2. C
3. B

Question 7

What parameter can pthread_create take?

Answer 7

Takes as parameter the function to run and optionally its arguments i.e. children threads

Question 8

What are the two ways of defining a thread in java?

Answer 8

1. Class inherits from java.lang.Thread
2. Class implements java.lang.Runnable. Lets you inherit from something else than Thread

Question 9

Match the java thread function

1. run()
2. Thread.start()
3. Thread.join()

A. to wait for the thread to complete
B. defines what the thread does when it starts running
C. gets the thread going

Answer 9

1. B
2. C
3. A

Question 10

How is the order of execution of threads determined?

Answer 10

No control over the order of execution! The OS scheduler decides, it’s nondeterministic

Question 11

What is Data Parallelism? And when does it work best?

Answer 11

Divide computation into (nearly) equal sized chunks. Works best when there are no data dependencies between chunks.

Question 12

Where is Data Parallelism used?

Answer 12

1. in multithreading
2. at the instruction level in vector/array (SIMD) processors: CPUs (SSE, AVX)
3. in GPGPUs

Question 13

Give an example of Data Parallelism

Answer 13

Matrix multiply of n x n matrices

Question 14

What is OpenMP used for and how is it useful?

Answer 14

OpenMP is used to achieve automatic parallelisation. Very low programmer effort

Question 15

Explain the difference between implicit and explicit parallelism

Answer 15

Explicit parallelism:
The programmer explicitly spells out what should be done in parallel/sequence. Using threads or other high level notations e.g. OpenMP

Implicit parallelism:
No effort from the programmer, the system works out parallelism by itself. Done for example by some language able to make strong assumption about data sharing e.g. pure functions in functional languages

Question 16

What are the issues with automatic parallelisation?

Answer 16

1. Works for simple programs, but dependency analysis can be very hard.
2. Automatic parallelisation can be limited by data dependencies
3. Must be conservative. If you cannot be certain that parallel version computes correct result, can’t parallelise

Question 17

How can this function be paralleised

for (int i=0; i<n-3; i++)
a[i] = a[i+3] + b[i];

Answer 17

1. Positive offset: we read at each iteration what was in the array before the loop started
2. We never read a value computed by the loop itself

Can parallelise by making a new version of array a: parrallel_for(int i=0; i<n-3; i++)
new_a[i] = a[i+3] + b[i];
a = new_a;

Question 18

How can this function be paralleised

for (int i = 5 ; i < n ; i++) {
a[i] += a[i-5] * 2 ;
}

Answer 18

limit parallelism to 5

Question 19

How do threads use Shared Memory?

Answer 19

Read and write in the same memory location e.g. variable, buffer through:
1. Global variable
2. Pointers and references

Question 20

How can threads communicate of there was no shared memory?

Answer 20

Need to communicate via message passing. Close to a distributed system

Question 21

Is shared memory implicit or explicit?

Answer 21

implicit