MPI

Question 1

MPI_init(NULL, NULL)

Answer 1

• Decide which processes get which
  rank
• Allocating storage for message
  buffers
• Define a communicator that
  consist of all the processes started
  by the user at program start up.
  This communicator is called
  MPI_COM_WORLD.

Question 2

MPI_Finalize()

Answer 2

• Says that any resources allocated for MPI can be freed

Question 3

MPI_Comm_size(MPI_COMM_WORLD, &comm_sz)

Answer 3

• Sets comm_sz to the number of processors

Question 4

MPI_Comm_rank(MPI_COMM_WORLD, &my_rank)

Answer 4

• sets my_rank to the process number

Question 5

MPI_Send(snd, buffer, snd_sz, snd_type, dest, tag, comm)

Answer 5

• Greeting is pointer to block of memory containing contents of the message ( a string rated)
• Second argument is number of items in the buffer
• Third argument: type of items in the buffer (in this case char)
• 4th argument is the rank of the destination process
• 5th argument is the tag: a nonnegative integer’s used to distinguish two messages.
• 6th argument: communicate or (in most cases MPI_Comm_World

Question 6

MPI_Recv(rcv_buf, rcv_sz, rcv_type, src, tag, comm, &status)

Answer 6

• Greeting is pointer to block of memory containing contents of the received message ( a string rated)
• Second argument is number of items in the buffer
• Third argument: type of items in the buffer (in this case char)
• 4th argument is the rank of the source process from which the message is sent
• 5th argument is the tag: a nonnegative integers used to distinguish two messages. (must match tag of send
  operation)
• 6th argument: communicator (in most cases MPI_Comm_World
• 7th argument: either. A Variable of type MPI_Status or a special constant called MPI_STATUS_IGNORE)

Question 7

MPI_Status* status

Answer 7

• The status variable has 3 elements that it can receive
• status.MPI_SOURCE
• status.MPI_TAG
• status.MPI_ERROR

Question 8

MPI communication modes

Answer 8

Synchronous
* Only completes when receive completed

Buffered
* Always completes irrespective of wheter the receive has completed

Standard
* Either synchronous or buffered, runtime system decides

Ready Send
* Always completes irrespective of whether the receive has completed

Receive
* Completes when a message has arrived

Question 9

Blocking forms

Answer 9

Standard Send
* MPI_Send

Synchronous Send
* MPI_Ssend

Buffered Send
* MPI_Bsend

Ready Send
* MPI_Rsend

Receive
* MPI_Recv

Question 10

Non-blocking forms

Answer 10

Standard Send
* MPI_Isend

Synchronous Send
* MPI_Issend

Buffered Send
* MPI_Ibsend

Ready Send
* MPI_Irsend

Receive
* MPI_Irecv

Question 11

Ready Send (MPI_Rsend)

Answer 11

• Completes immediately
• Guaranteed to succeed normally
  
  • if matching receive has already been posted
• If receiver not ready
  
  • message may be dropped and an error may occur
• Non-blocking ready send
  
  • has no advantage over blocking ready sent

Question 12

Synchronous Send (MPI_Ssend)

Answer 12

MPI_Ssend is guaranteed to block until the matching
receive starts.

Question 13

MPI_Sendrecv

Answer 13

• An alternative to scheduling the communications ourselves.
• Carries out a blocking send and a receive in a single call.
• The dest and the source can be the same or different.
• Especially useful because MPI schedules the communications so that the program won’t hang or crash.

Question 14

Testing for completion

Answer 14

• If operation has completed:
  ~~~
  MPI_Test
  MPI_Wait
  MPI_Testany
  MPI_Waitany
  MPI_Testsome
  ~~~

Question 15

MPI_Reduce

Answer 15

We can use MPI_Reduce to get input data from all other processes and perform an operation on them.
Operations available:
* MPI_MAX
* MPI_MIN
* MPI_SUM
* MPI_PROD
* MPI_LAND
* MPI_BAND
* MPI_LOR
* MPI_BOR
* MPI_LXOR
* MPI_BXOR
* MPI_MAXLOC
* MPI_MINLOC

Question 16

Collective vs Point-Point communcation

Answer 16

Collective:
* All processes in communicator must call the same collective function
* For example, a program that attempts to match a call to MPI_Reduce on one process with a call to
MPI_Recv on another process is erroneous, and, in all likelihood, the program will hang or crash.
* Arguments passed by each process must be “compatible.”
* For example, if one process passes in 0 as the dest_process and another passes in 1, then the
outcome of a call to MPI_Reduce is erroneous, and, once again, the program is likely to hang or
crash.
* All processes need to pass an argument to all parameters even if not used
* E.g., output parameter only used on proc 0
* Matched on basis of communicator and order called
* While point-to-point was matched on basis of tags and communicator
* Collective communicator don’t use tags so they are matched on only basis of communicator and order they
are called.

Question 17

MPI_Allreduce

Answer 17

This stores reduction result at all the processes. Its
implemented as a reduce operation followed by a
broadcast operation.
Useful in a situation in which all of the processes
need the result of a global sum in order to complete
some larger computation.

Question 18

MPI_Bcast

Answer 18

Data belonging to a single process is sent to all of the processes in the communicator.

Question 19

MPI_Scatter

Answer 19

MPI_Scatter can be used in a function that reads in an entire vector on process 0 but only sends the
needed components to each of the other processes

Question 20

MPI_Gather

Answer 20

Collect all of the components of the vector onto process 0, and then process 0 can process all of the
components.

Question 21

MPI_Allgather

Answer 21

Concatenates the contents of each process’ send_buf_p and stores this in each process’ recv_buf_p.
As usual, recv_count is the amount of data being received from each process.